2023-07-21T21:43:06.378 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T21:43:06.379 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi --verbose --archive /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/orig.config.yaml 2023-07-21T21:43:06.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T21:43:11.538 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716 branch: wip-rishabh-2023Jul13-base-2 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7347716' kernel: client: branch: testing flavor: default kdb: true sha1: c685da6f56cc77f2cfb9c4221be3960522616caf ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-base-2 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-base-2 sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-base-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi066.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi149.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 79491cb69dd37a1eafb93b3471998174bdfa2fd9 timestamp: 2023-07-21_21:30:53 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-21T21:43:11.539 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa; will attempt to use it 2023-07-21T21:43:11.540 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks 2023-07-21T21:43:11.540 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-21T21:43:11.542 INFO:teuthology.task.internal:Checking packages... 2023-07-21T21:43:11.573 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea' 2023-07-21T21:43:11.573 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-21T21:43:11.574 INFO:teuthology.packaging:ref: None 2023-07-21T21:43:11.574 INFO:teuthology.packaging:tag: None 2023-07-21T21:43:11.574 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-base-2 2023-07-21T21:43:11.575 INFO:teuthology.packaging:sha1: bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T21:43:11.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-base-2 2023-07-21T21:43:17.177 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4735.gbffe049d 2023-07-21T21:43:17.177 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-21T21:43:17.188 INFO:teuthology.task.internal:no buildpackages task found 2023-07-21T21:43:17.188 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-21T21:43:17.194 INFO:teuthology.task.internal:Saving configuration 2023-07-21T21:43:17.215 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-21T21:43:17.226 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T21:43:27.260 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:36:15.023638', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T21:43:32.287 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:36:15.022389', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T21:43:41.451 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:36:15.020912', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T21:43:41.451 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-21T21:43:41.461 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-21T21:43:41.461 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-21T21:43:41.461 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-21T21:43:41.462 INFO:teuthology.run_tasks:Running task console_log... 2023-07-21T21:43:54.040 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd5a24ff5e0>, signals=[15]) 2023-07-21T21:43:54.041 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-21T21:43:54.050 INFO:teuthology.task.internal:Opening connections... 2023-07-21T21:43:54.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T21:43:54.052 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:43:54.125 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2023-07-21T21:43:54.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:43:54.194 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2023-07-21T21:43:54.195 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:43:54.264 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-21T21:43:54.270 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2023-07-21T21:43:54.292 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2023-07-21T21:43:54.293 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2023-07-21T21:43:54.350 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T21:43:54.350 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T21:43:54.350 INFO:teuthology.orchestra.run.smithi066.stdout:ID="rhel" 2023-07-21T21:43:54.350 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="fedora" 2023-07-21T21:43:54.350 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="8.6" 2023-07-21T21:43:54.350 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el8" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T21:43:54.351 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T21:43:54.352 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T21:43:54.352 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T21:43:54.352 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T21:43:54.352 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T21:43:54.353 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2023-07-21T21:43:54.375 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2023-07-21T21:43:54.394 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2023-07-21T21:43:54.395 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2023-07-21T21:43:54.451 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T21:43:54.452 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T21:43:54.452 INFO:teuthology.orchestra.run.smithi104.stdout:ID="rhel" 2023-07-21T21:43:54.452 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="fedora" 2023-07-21T21:43:54.452 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="8.6" 2023-07-21T21:43:54.452 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el8" 2023-07-21T21:43:54.452 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T21:43:54.453 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T21:43:54.454 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T21:43:54.454 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T21:43:54.454 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T21:43:54.454 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2023-07-21T21:43:54.475 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2023-07-21T21:43:54.494 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2023-07-21T21:43:54.494 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2023-07-21T21:43:54.551 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T21:43:54.551 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T21:43:54.551 INFO:teuthology.orchestra.run.smithi149.stdout:ID="rhel" 2023-07-21T21:43:54.552 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="fedora" 2023-07-21T21:43:54.552 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="8.6" 2023-07-21T21:43:54.552 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el8" 2023-07-21T21:43:54.552 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T21:43:54.552 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2023-07-21T21:43:54.552 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T21:43:54.553 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T21:43:54.554 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T21:43:54.554 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2023-07-21T21:43:54.575 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-21T21:43:54.591 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-21T21:43:54.623 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-21T21:43:54.624 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2023-07-21T21:43:54.626 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2023-07-21T21:43:54.628 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2023-07-21T21:43:54.644 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-21T21:43:54.650 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-21T21:43:54.650 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2023-07-21T21:43:54.684 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2023-07-21T21:43:54.686 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2023-07-21T21:43:54.712 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-21T21:43:54.806 INFO:teuthology.run_tasks:Running task kernel... 2023-07-21T21:43:54.822 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'} 2023-07-21T21:43:54.824 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-07-21T21:43:54.825 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-07-21T21:43:54.825 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T21:43:54.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:43:54.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-07-21T21:43:56.034 DEBUG:teuthology.task.kernel:sha1 for client.0 is c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T21:43:56.034 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "c685da6f56cc77f2cfb9c4221be3960522616caf"... 2023-07-21T21:43:56.034 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2023-07-21T21:43:56.055 INFO:teuthology.orchestra.run.smithi066.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T21:43:56.055 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T21:43:56.056 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-21T21:43:56.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum upgrade -y linux-firmware 2023-07-21T21:43:57.272 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-07-21T21:43:57.272 INFO:teuthology.orchestra.run.smithi066.stdout:Unable to read consumer identity 2023-07-21T21:43:59.311 INFO:teuthology.orchestra.run.smithi066.stdout:Copr repo for python3-asyncssh owned by ceph 4.1 kB/s | 3.5 kB 00:00 2023-07-21T21:44:01.720 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 6.5 MB/s | 16 MB 00:02 2023-07-21T21:44:05.403 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 411 kB/s | 24 kB 00:00 2023-07-21T21:44:06.205 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-07-21T21:44:06.206 INFO:teuthology.orchestra.run.smithi066.stdout:Nothing to do. 2023-07-21T21:44:06.207 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-07-21T21:44:06.304 INFO:teuthology.task.kernel:Downloading kernel c685da6f56cc77f2cfb9c4221be3960522616caf on client.0... 2023-07-21T21:44:06.304 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T21:44:09.854 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ 2023-07-21T21:44:09.854 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-21T21:44:09.856 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi066.front.sepia.ceph.com to complete... 2023-07-21T21:44:52.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21 21:44:52 URL:https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm [380683892/380683892] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-21T21:44:52.103 INFO:teuthology.orchestra.run.smithi066.stderr:FINISHED --2023-07-21 21:44:52-- 2023-07-21T21:44:52.103 INFO:teuthology.orchestra.run.smithi066.stderr:Total wall clock time: 42s 2023-07-21T21:44:52.103 INFO:teuthology.orchestra.run.smithi066.stderr:Downloaded: 1 files, 363M in 41s (8.76 MB/s) 2023-07-21T21:44:52.104 INFO:teuthology.task.kernel:Installing kernel c685da6f56cc77f2cfb9c4221be3960522616caf on client.0... 2023-07-21T21:44:52.105 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-21T21:44:53.770 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying... ######################################## 2023-07-21T21:44:53.827 INFO:teuthology.orchestra.run.smithi066.stdout:Preparing... ######################################## 2023-07-21T21:44:53.923 INFO:teuthology.orchestra.run.smithi066.stdout:Updating / installing... 2023-07-21T21:45:34.559 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-6.5.0_rc1_gc685da6f56cc-1 ######################################## 2023-07-21T21:45:58.420 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-21T21:45:58.421 DEBUG:teuthology.orchestra.run.smithi066:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-21T21:45:58.473 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.473 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/config-6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/boot/vmlinuz-6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto 2023-07-21T21:45:58.474 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-21T21:45:58.475 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-21T21:45:58.476 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu 2023-07-21T21:45:58.477 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm.ko 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ansi_cprng.ko 2023-07-21T21:45:58.478 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/anubis.ko 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/arc4.ko 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_memcpy.ko 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_pq.ko 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_tx.ko 2023-07-21T21:45:58.479 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_xor.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/raid6test.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authenc.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authencesn.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blake2b_generic.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_common.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_generic.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/camellia_generic.ko 2023-07-21T21:45:58.480 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast5_generic.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast6_generic.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast_common.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ccm.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cmac.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crc32_generic.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_engine.ko 2023-07-21T21:45:58.481 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_user.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cts.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/des_generic.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/dh_generic.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/echainiv.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/essiv.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/fcrypt.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/khazad.ko 2023-07-21T21:45:58.482 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4hc.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/md4.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/michael_mic.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcbc.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcrypt.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/rmd160.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/seed.ko 2023-07-21T21:45:58.483 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/serpent_generic.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tcrypt.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tea.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_common.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_generic.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/vmac.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/wp512.ko 2023-07-21T21:45:58.484 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xcbc.ko 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xor.ko 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xxhash_generic.ko 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_pad.ko 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/custom_method.ko 2023-07-21T21:45:58.485 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/ec_sys.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/platform_profile.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbs.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbshc.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/video.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/acard-ahci.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ahci_platform.ko 2023-07-21T21:45:58.486 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ata_generic.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/libahci_platform.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_acpi.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ali.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_amd.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_artop.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atiixp.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atp867x.ko 2023-07-21T21:45:58.487 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd640.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd64x.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cypress.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_efar.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt366.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt37x.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-21T21:45:58.488 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it8213.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it821x.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_jmicron.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_marvell.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_mpiix.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_netcell.ko 2023-07-21T21:45:58.489 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ninja32.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87410.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87415.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_oldpiix.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_opti.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_optidma.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pcmcia.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-21T21:45:58.490 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_piccolo.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_rdc.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sch.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_serverworks.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sil680.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sis.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sl82c105.ko 2023-07-21T21:45:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_triflex.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_via.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pdc_adma.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_inic162x.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_mv.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_nv.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_promise.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_qstor.ko 2023-07-21T21:45:58.492 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil24.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sis.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_svw.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sx4.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_uli.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_via.ko 2023-07-21T21:45:58.493 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_vsc.ko 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay/ks0108.ko 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma 2023-07-21T21:45:58.494 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma/bcma.ko 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe/aoe.ko 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/brd.ko 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd/drbd.ko 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/floppy.ko 2023-07-21T21:45:58.495 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/loop.ko 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/nbd.ko 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk/null_blk.ko 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/pktcdvd.ko 2023-07-21T21:45:58.496 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/rbd.ko 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/virtio_blk.ko 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkfront.ko 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram/zram.ko 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char 2023-07-21T21:45:58.497 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hangcheck-timer.ko 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/via-rng.ko 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi 2023-07-21T21:45:58.498 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/lp.ko 2023-07-21T21:45:58.499 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave/mwave.ko 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ppdev.ko 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tlclk.ko 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm.ko 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-21T21:45:58.500 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/uv_mmtimer.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/virtio_console.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-21T21:45:58.501 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp.ko 2023-07-21T21:45:58.502 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-aes.ko 2023-07-21T21:45:58.503 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-sha.ko 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca/dca.ko 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-21T21:45:58.504 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/amd64_edac.ko 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/e752x_edac.ko 2023-07-21T21:45:58.505 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/edac_mce_amd.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3000_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3200_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5100_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5400_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7300_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7core_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i82975x_edac.ko 2023-07-21T21:45:58.506 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/ie31200_edac.ko 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/sb_edac.ko 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/x38_edac.ko 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/edd.ko 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-ich.ko 2023-07-21T21:45:58.507 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-appleir.ko 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-aureal.ko 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-axff.ko 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-dr.ko 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elecom.ko 2023-07-21T21:45:58.508 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elo.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-emsff.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gaff.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gt683r.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gyration.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtekff.ko 2023-07-21T21:45:58.509 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-hyperv.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-icade.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-keytouch.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-kye.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lcpower.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-led.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lenovo.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-21T21:45:58.510 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-multitouch.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-ortek.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-penmount.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-petalynx.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-picolcd.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-pl.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-plantronics.ko 2023-07-21T21:45:58.511 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-primax.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-rmi.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-common.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-21T21:45:58.512 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-saitek.ko 2023-07-21T21:45:58.513 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-samsung.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sjoy.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sony.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-speedlink.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-steelseries.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sunplus.ko 2023-07-21T21:45:58.514 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tivo.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tmff.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-topseed.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-twinhan.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-uclogic.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-waltop.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-wiimote.ko 2023-07-21T21:45:58.515 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-xinmo.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zpff.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zydacron.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/uhid.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/wacom.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_balloon.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_utils.ko 2023-07-21T21:45:58.516 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_vmbus.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru3.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7414.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7418.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adc128d818.ko 2023-07-21T21:45:58.517 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1025.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1026.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1029.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1031.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm9240.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ads7828.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7410.ko 2023-07-21T21:45:58.518 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7411.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7462.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7470.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7475.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7x10.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/amc6821.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/applesmc.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asb100.ko 2023-07-21T21:45:58.519 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asc7621.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/atxp1.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/coretemp.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dme1737.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds1621.ko 2023-07-21T21:45:58.520 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds620.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc1403.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc6w201.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71805f.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71882fg.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f75375s.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fam15h_power.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fschmd.ko 2023-07-21T21:45:58.521 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g760a.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g762.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl518sm.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl520sm.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5500_temp.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5k_amb.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmaem.ko 2023-07-21T21:45:58.522 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmpex.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina209.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina2xx.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/it87.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k10temp.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k8temp.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lineage-pem.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm63.ko 2023-07-21T21:45:58.523 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm73.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm75.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm77.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm78.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm80.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm83.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm85.ko 2023-07-21T21:45:58.524 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm87.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm90.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm92.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm93.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95234.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95241.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95245.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc2945.ko 2023-07-21T21:45:58.525 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4151.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4215.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4222.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4245.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4260.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4261.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max16065.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1619.ko 2023-07-21T21:45:58.526 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1668.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max197.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6639.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6650.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6697.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/mcp3021.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6683.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775-core.ko 2023-07-21T21:45:58.527 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct7802.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87360.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87427.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pcf8591.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-21T21:45:58.528 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-21T21:45:58.529 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/powr1220.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5627.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5636.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht15.ko 2023-07-21T21:45:58.530 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht21.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/shtc1.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sis5595.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47b397.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m1.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m192.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/thmc50.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp102.ko 2023-07-21T21:45:58.531 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp103.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp401.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp421.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via-cputemp.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via686a.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt1211.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt8231.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627ehf.ko 2023-07-21T21:45:58.532 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627hf.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83781d.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83791d.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83792d.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83793.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83795.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l785ts.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l786ng.ko 2023-07-21T21:45:58.533 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-21T21:45:58.534 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-21T21:45:58.535 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-21T21:45:58.536 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-dev.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-mux.ko 2023-07-21T21:45:58.537 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-smbus.ko 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-stub.ko 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_core.ko 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-21T21:45:58.538 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4 2023-07-21T21:45:58.539 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-21T21:45:58.540 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-21T21:45:58.541 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/gameport.ko 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/lightning.ko 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/ns558.ko 2023-07-21T21:45:58.542 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/joydev.ko 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/appletouch.ko 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/bcm5974.ko 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/cyapatp.ko 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-21T21:45:58.543 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/sermouse.ko 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/altera_ps2.ko 2023-07-21T21:45:58.544 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/arc_ps2.ko 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/serio_raw.ko 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/sparse-keymap.ko 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds 2023-07-21T21:45:58.545 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-blinkm.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lm3530.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lp3944.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-ss4200.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-21T21:45:58.546 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache 2023-07-21T21:45:58.547 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache/bcache.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-bio-prison.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache-smq.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-crypt.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-delay.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-flakey.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-log-userspace.ko 2023-07-21T21:45:58.548 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-multipath.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-queue-length.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-raid.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-round-robin.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-service-time.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-switch.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-thin-pool.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-verity.ko 2023-07-21T21:45:58.549 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/faulty.ko 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/linear.ko 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/multipath.ko 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid0.ko 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid1.ko 2023-07-21T21:45:58.550 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid10.ko 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid456.ko 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptbase.ko 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptctl.ko 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptfc.ko 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptlan.ko 2023-07-21T21:45:58.551 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptsas.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptscsih.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptspi.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_ich.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_sch.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/mfd-core.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/sm501.ko 2023-07-21T21:45:58.552 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/viperboard.ko 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/vx855.ko 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/wl1273-core.ko 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds9802als.ko 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds990x.ko 2023-07-21T21:45:58.553 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/bh1770glc.ko 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710/cb710.ko 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/at24.ko 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/max6875.ko 2023-07-21T21:45:58.554 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/enclosure.ko 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/hpilo.ko 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29003.ko 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29020.ko 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-21T21:45:58.555 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-me.ko 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-txe.ko 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei.ko 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-21T21:45:58.556 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_7xx1.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_core.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tsl2550.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_balloon.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-21T21:45:58.557 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding/bonding.ko 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-21T21:45:58.558 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dummy.ko 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/eql.ko 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-21T21:45:58.559 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-21T21:45:58.560 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-21T21:45:58.561 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-21T21:45:58.562 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-21T21:45:58.563 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-21T21:45:58.564 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-21T21:45:58.565 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic 2023-07-21T21:45:58.566 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-21T21:45:58.567 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dnet.ko 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet 2023-07-21T21:45:58.568 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ethoc.ko 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/fealnx.ko 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e 2023-07-21T21:45:58.569 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb 2023-07-21T21:45:58.570 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/jme.ko 2023-07-21T21:45:58.571 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-21T21:45:58.572 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-21T21:45:58.573 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-21T21:45:58.574 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-21T21:45:58.575 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-21T21:45:58.576 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis 2023-07-21T21:45:58.577 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro 2023-07-21T21:45:58.578 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-21T21:45:58.579 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-21T21:45:58.580 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/geneve.ko 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-21T21:45:58.581 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ifb.ko 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvlan.ko 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvtap.ko 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio 2023-07-21T21:45:58.582 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mii.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/net_failover.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/netconsole.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/nlmon.ko 2023-07-21T21:45:58.583 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/amd.ko 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-21T21:45:58.584 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm87xx.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/broadcom.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/cicada.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/davicom.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/dp83640.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/et1011c.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/icplus.ko 2023-07-21T21:45:58.585 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/lxt.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/marvell.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/micrel.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/national.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/phylink.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/qsemi.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/realtek.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/smsc.ko 2023-07-21T21:45:58.586 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/ste10Xp.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/vitesse.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_async.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-21T21:45:58.587 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppoe.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppox.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pptp.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slhc.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slip.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/sungem_phy.ko 2023-07-21T21:45:58.588 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tap.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_random.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-21T21:45:58.589 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tun.ko 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/veth.ko 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/virtio_net.ko 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan/vxlan.ko 2023-07-21T21:45:58.590 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netfront.ko 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb/ntb.ko 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme-core.ko 2023-07-21T21:45:58.591 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme.ko 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport.ko 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_cs.ko 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_pc.ko 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_serial.ko 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug 2023-07-21T21:45:58.592 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/xen-pcifront.ko 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/i82092.ko 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/pd6729.ko 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-21T21:45:58.593 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acerhdf.ko 2023-07-21T21:45:58.594 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-21T21:45:58.595 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/hdaps.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel_ips.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-21T21:45:58.596 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi.ko 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap 2023-07-21T21:45:58.597 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps_parport.ko 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/pps_core.ko 2023-07-21T21:45:58.598 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp.ko 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp_kvm.ko 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-21T21:45:58.599 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-21T21:45:58.600 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-em3027.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-max6900.ko 2023-07-21T21:45:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-21T21:45:58.602 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-x1205.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-9xxx.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-sas.ko 2023-07-21T21:45:58.603 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-xxxx.ko 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/BusLogic.ko 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/a100u2w.ko 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/advansys.ko 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-21T21:45:58.604 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/am53c974.ko 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/atp870u.ko 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa 2023-07-21T21:45:58.605 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa/bfa.ko 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ch.ko 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor 2023-07-21T21:45:58.606 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dc395x.ko 2023-07-21T21:45:58.607 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dmx3191d.ko 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-21T21:45:58.608 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esp_scsi.ko 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic/fnic.ko 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hpsa.ko 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hptiop.ko 2023-07-21T21:45:58.609 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hv_storvsc.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/initio.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ipr.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ips.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci/isci.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-21T21:45:58.610 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc/libfc.ko 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi.ko 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas/libsas.ko 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-21T21:45:58.611 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid.ko 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-21T21:45:58.612 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvumi.ko 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pmcraid.ko 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla1280.ko 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx 2023-07-21T21:45:58.613 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/raid_class.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_debug.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-21T21:45:58.614 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ses.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/st.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/stex.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-21T21:45:58.615 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/virtio_scsi.ko 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/wd719x.ko 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb/ssb.ko 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-21T21:45:58.616 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_file.ko 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_iblock.ko 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_mod.ko 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_pscsi.ko 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_user.ko 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc 2023-07-21T21:45:58.617 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-21T21:45:58.618 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-21T21:45:58.619 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_gsm.ko 2023-07-21T21:45:58.620 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_hdlc.ko 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/nozomi.ko 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/arc_uart.ko 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-21T21:45:58.621 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/synclink_gt.ko 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio.ko 2023-07-21T21:45:58.622 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_aec.ko 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_cif.ko 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_pci_generic.ko 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_sercos3.ko 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-acm.ko 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-21T21:45:58.623 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usblp.ko 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usbtmc.ko 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/mdc800.ko 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/microtek.ko 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc 2023-07-21T21:45:58.624 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/adutux.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/appledisplay.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi26.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi62.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ezusb.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/idmouse.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/iowarrior.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-21T21:45:58.625 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ldusb.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/legousbtower.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usb3503.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usblcd.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-21T21:45:58.626 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/uss720.ko 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/yurex.ko 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy/phy-generic.ko 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/aircable.ko 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ark3116.ko 2023-07-21T21:45:58.627 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ch341.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cp210x.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cyberjack.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/empeg.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-21T21:45:58.628 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_ti.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipaq.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipw.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ir-usb.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan.ko 2023-07-21T21:45:58.629 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mct_u232.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7720.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7840.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/navman.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/omninet.ko 2023-07-21T21:45:58.630 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/opticon.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/option.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/oti6858.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/pl2303.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcaux.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcserial.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/quatech2.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/safe_serial.ko 2023-07-21T21:45:58.631 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/sierra.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ssu100.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/symbolserial.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_debug.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-21T21:45:58.632 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/visor.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/whiteheat.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/uas.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-21T21:45:58.633 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-karma.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-21T21:45:58.634 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/usb-storage.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-21T21:45:58.635 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio.ko 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost.ko 2023-07-21T21:45:58.636 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_net.ko 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_scsi.ko 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/apple_bl.ko 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/lcd.ko 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-21T21:45:58.637 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_balloon.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_mmio.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_ring.ko 2023-07-21T21:45:58.638 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2482.ko 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2490.ko 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-21T21:45:58.639 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-21T21:45:58.640 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/wire.ko 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/hpwdt.ko 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/i6300esb.ko 2023-07-21T21:45:58.641 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ib700wdt.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ibmasr.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it87_wdt.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/machzwd.ko 2023-07-21T21:45:58.642 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/nv_tco.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/softdog.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/via_wdt.ko 2023-07-21T21:45:58.643 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/wdt_pci.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/xen_wdt.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-evtchn.ko 2023-07-21T21:45:58.644 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntalloc.ko 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntdev.ko 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-privcmd.ko 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs 2023-07-21T21:45:58.645 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p/9p.ko 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs/affs.ko 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs/befs.ko 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/binfmt_misc.ko 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs 2023-07-21T21:45:58.646 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs/btrfs.ko 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles/cachefiles.ko 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph/ceph.ko 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda/coda.ko 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs 2023-07-21T21:45:58.647 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs/cramfs.ko 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm/dlm.ko 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs/f2fs.ko 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat 2023-07-21T21:45:58.648 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/fat.ko 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/msdos.ko 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/vfat.ko 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache/fscache.ko 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/cuse.ko 2023-07-21T21:45:58.649 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/fuse.ko 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2/gfs2.ko 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs/hfs.ko 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus/hfsplus.ko 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs 2023-07-21T21:45:58.650 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs/isofs.ko 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs/jfs.ko 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd/lockd.ko 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix/minix.ko 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs 2023-07-21T21:45:58.651 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs/netfs.ko 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-21T21:45:58.652 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfs.ko 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv3.ko 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv4.ko 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/grace.ko 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/nfs_acl.ko 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd/nfsd.ko 2023-07-21T21:45:58.653 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2/nilfs2.ko 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-celtic.ko 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-centeuro.ko 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-croatian.ko 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-cyrillic.ko 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-gaelic.ko 2023-07-21T21:45:58.654 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-greek.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-iceland.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-inuit.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-roman.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-romanian.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-turkish.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1250.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1251.ko 2023-07-21T21:45:58.655 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1255.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp737.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp775.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp850.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp852.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp855.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp857.ko 2023-07-21T21:45:58.656 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp860.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp861.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp862.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp863.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp864.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp865.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp866.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp869.ko 2023-07-21T21:45:58.657 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp874.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp932.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp936.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp949.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp950.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_euc-jp.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-1.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-13.ko 2023-07-21T21:45:58.658 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-14.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-15.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-2.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-3.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-4.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-5.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-6.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-7.ko 2023-07-21T21:45:58.659 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-9.ko 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-r.ko 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-ru.ko 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-u.ko 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_utf8.ko 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-21T21:45:58.660 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2.ko 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-21T21:45:58.661 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs/overlay.ko 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore/ramoops.ko 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs/reiserfs.ko 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs/romfs.ko 2023-07-21T21:45:58.662 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client/cifs.ko 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_arc4.ko 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_md4.ko 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs/squashfs.ko 2023-07-21T21:45:58.663 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv/sysv.ko 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf/udf.ko 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs/ufs.ko 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs 2023-07-21T21:45:58.664 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs/xfs.ko 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu/rcutorture.ko 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/torture.ko 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib 2023-07-21T21:45:58.665 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/asn1_encoder.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc-itu-t.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc8.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libarc4.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libdes.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/libcrc32c.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lru_cache.ko 2023-07-21T21:45:58.666 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4_compress.ko 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4hc_compress.ko 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math/cordic.ko 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6/raid6_pq.ko 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon 2023-07-21T21:45:58.667 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_bm.ko 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_fsm.ko 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_kmp.ko 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd/zstd_compress.ko 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm/hwpoison-inject.ko 2023-07-21T21:45:58.668 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/6lowpan.ko 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_dest.ko 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_fragment.ko 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_hop.ko 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-21T21:45:58.669 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_mobility.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_routing.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_udp.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/garp.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/mrp.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/p8022.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/psnap.ko 2023-07-21T21:45:58.670 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/stp.ko 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q/8021q.ko 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet.ko 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_fd.ko 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_rdma.ko 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_virtio.ko 2023-07-21T21:45:58.671 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv/batman-adv.ko 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/br_netfilter.ko 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/bridge.ko 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-21T21:45:58.672 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-21T21:45:58.673 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-21T21:45:58.674 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtables.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph/libceph.ko 2023-07-21T21:45:58.675 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/failover.ko 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/pktgen.ko 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp.ko 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_diag.ko 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv4.ko 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv6.ko 2023-07-21T21:45:58.676 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver/dns_resolver.ko 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/dsa_core.ko 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_brcm.ko 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_dsa.ko 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_none.ko 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_trailer.ko 2023-07-21T21:45:58.677 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154.ko 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ah4.ko 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/esp4.ko 2023-07-21T21:45:58.678 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/fou.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/gre.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/inet_diag.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_gre.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_tunnel.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_vti.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipcomp.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipip.ko 2023-07-21T21:45:58.679 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-21T21:45:58.680 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-21T21:45:58.681 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_bic.ko 2023-07-21T21:45:58.682 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_dctcp.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_diag.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_highspeed.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_htcp.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_hybla.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_illinois.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_lp.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_scalable.ko 2023-07-21T21:45:58.683 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_vegas.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_veno.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_westwood.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_yeah.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tunnel4.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_diag.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_tunnel.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-21T21:45:58.684 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ah6.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/esp6.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/fou6.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_tunnel.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_vti.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ipcomp6.ko 2023-07-21T21:45:58.685 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-21T21:45:58.686 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-21T21:45:58.687 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-21T21:45:58.688 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/sit.ko 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/tunnel6.ko 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key/af_key.ko 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_core.ko 2023-07-21T21:45:58.689 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_eth.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip6.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_netlink.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ppp.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc/llc.ko 2023-07-21T21:45:58.690 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154/mac802154.ko 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls/mpls_gso.ko 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set.ko 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-21T21:45:58.691 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-21T21:45:58.692 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs 2023-07-21T21:45:58.693 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-21T21:45:58.694 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conncount.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack.ko 2023-07-21T21:45:58.695 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-21T21:45:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_log_syslog.ko 2023-07-21T21:45:58.697 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_irc.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_sip.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_tables.ko 2023-07-21T21:45:58.698 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_log.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_chain_nat.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_compat.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_ct.ko 2023-07-21T21:45:58.699 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fib.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_hash.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_limit.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_log.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_masq.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_nat.ko 2023-07-21T21:45:58.700 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_numgen.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_queue.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_quota.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_redir.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject_inet.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_socket.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_tunnel.ko 2023-07-21T21:45:58.701 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_AUDIT.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CT.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_DSCP.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HL.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HMARK.ko 2023-07-21T21:45:58.702 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LED.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LOG.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NETMAP.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFLOG.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_RATEEST.ko 2023-07-21T21:45:58.703 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_SECMARK.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TEE.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TPROXY.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TRACE.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_addrtype.ko 2023-07-21T21:45:58.704 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_bpf.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cgroup.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cluster.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_comment.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connbytes.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlabel.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlimit.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connmark.ko 2023-07-21T21:45:58.705 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_conntrack.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cpu.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dccp.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_devgroup.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dscp.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ecn.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_esp.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hashlimit.ko 2023-07-21T21:45:58.706 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_helper.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hl.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipcomp.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_iprange.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipvs.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_l2tp.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_length.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_limit.ko 2023-07-21T21:45:58.707 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mac.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mark.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_multiport.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nat.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nfacct.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_osf.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_owner.ko 2023-07-21T21:45:58.708 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_physdev.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_pkttype.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_policy.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_quota.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_rateest.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_realm.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_recent.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_sctp.ko 2023-07-21T21:45:58.709 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_set.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_socket.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_state.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_statistic.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_string.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_tcpmss.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_time.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_u32.ko 2023-07-21T21:45:58.710 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink/netlink_diag.ko 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh/nsh.ko 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/openvswitch.ko 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-geneve.ko 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-gre.ko 2023-07-21T21:45:58.711 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-vxlan.ko 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet/af_packet_diag.ko 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds.ko 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_rdma.ko 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_tcp.ko 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched 2023-07-21T21:45:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_csum.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_gact.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_ipt.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_mirred.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_nat.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_pedit.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_police.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_simple.ko 2023-07-21T21:45:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_skbedit.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_vlan.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_basic.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_bpf.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_flow.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_fw.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_route.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_u32.ko 2023-07-21T21:45:58.714 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_cmp.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_ipset.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_meta.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_nbyte.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_text.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_u32.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_choke.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_codel.ko 2023-07-21T21:45:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_drr.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_fq.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_gred.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hfsc.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hhf.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_htb.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_ingress.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio.ko 2023-07-21T21:45:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio_lib.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_multiq.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_netem.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_pie.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_plug.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_prio.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_qfq.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_red.ko 2023-07-21T21:45:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfb.ko 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfq.ko 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_tbf.ko 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_teql.ko 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp.ko 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp_diag.ko 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc 2023-07-21T21:45:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/sunrpc.ko 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc 2023-07-21T21:45:58.719 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/diag.ko 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/tipc.ko 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix/unix_diag.ko 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock.ko 2023-07-21T21:45:58.720 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-21T21:45:58.721 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys/trusted.ko 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib/irqbypass.ko 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias.bin 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin 2023-07-21T21:45:58.722 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.bin 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.modinfo 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep.bin 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.devname 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.order 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.softdep 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols 2023-07-21T21:45:58.723 INFO:teuthology.orchestra.run.smithi066.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols.bin 2023-07-21T21:45:58.724 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.724 DEBUG:teuthology.orchestra.run.smithi066:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-21T21:45:58.752 INFO:teuthology.orchestra.run.smithi066.stdout:postinstall scriptlet (using /bin/sh): 2023-07-21T21:45:58.752 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc -a -r /boot/System.map-6.5.0-rc1-gc685da6f56cc ]; then 2023-07-21T21:45:58.752 INFO:teuthology.orchestra.run.smithi066.stdout:cp /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T21:45:58.752 INFO:teuthology.orchestra.run.smithi066.stdout:cp /boot/System.map-6.5.0-rc1-gc685da6f56cc /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/installkernel 6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-21T21:45:58.753 INFO:teuthology.orchestra.run.smithi066.stdout:new-kernel-pkg --remove 6.5.0-rc1-gc685da6f56cc --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-gc685da6f56cc.img 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-install remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:fi 2023-07-21T21:45:58.755 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -qi grub2-tools 2023-07-21T21:45:58.833 INFO:teuthology.orchestra.run.smithi066.stdout:Name : grub2-tools 2023-07-21T21:45:58.833 INFO:teuthology.orchestra.run.smithi066.stdout:Epoch : 1 2023-07-21T21:45:58.834 INFO:teuthology.orchestra.run.smithi066.stdout:Version : 2.02 2023-07-21T21:45:58.834 INFO:teuthology.orchestra.run.smithi066.stdout:Release : 123.el8 2023-07-21T21:45:58.834 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: x86_64 2023-07-21T21:45:58.834 INFO:teuthology.orchestra.run.smithi066.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-21T21:45:58.834 INFO:teuthology.orchestra.run.smithi066.stdout:Group : System Environment/Base 2023-07-21T21:45:58.834 INFO:teuthology.orchestra.run.smithi066.stdout:Size : 9265450 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:License : GPLv3+ 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Relocations : (not relocatable) 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Packager : Red Hat, Inc. 2023-07-21T21:45:58.835 INFO:teuthology.orchestra.run.smithi066.stdout:Vendor : Red Hat, Inc. 2023-07-21T21:45:58.836 INFO:teuthology.orchestra.run.smithi066.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-21T21:45:58.836 INFO:teuthology.orchestra.run.smithi066.stdout:Summary : Support tools for GRUB. 2023-07-21T21:45:58.836 INFO:teuthology.orchestra.run.smithi066.stdout:Description : 2023-07-21T21:45:58.836 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T21:45:58.836 INFO:teuthology.orchestra.run.smithi066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-21T21:45:58.836 INFO:teuthology.orchestra.run.smithi066.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-21T21:45:58.837 INFO:teuthology.orchestra.run.smithi066.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-21T21:45:58.837 INFO:teuthology.orchestra.run.smithi066.stdout:hardware devices. 2023-07-21T21:45:58.837 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T21:45:58.837 INFO:teuthology.orchestra.run.smithi066.stdout:This subpackage provides tools for support of all platforms. 2023-07-21T21:45:58.838 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-21T21:45:58.838 INFO:teuthology.task.kernel:Updating grub on smithi066 to boot 6.5.0-rc1-gc685da6f56cc 2023-07-21T21:45:58.839 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-21T21:46:02.109 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2023-07-21T21:46:04.382 INFO:teuthology.orchestra.run.smithi066.stderr:done 2023-07-21T21:46:04.383 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2023-07-21T21:46:04.400 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.30NRiyV0M4 2023-07-21T21:46:04.401 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.30NRiyV0M4 2023-07-21T21:46:04.473 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.30NRiyV0M4 2023-07-21T21:46:04.631 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.30NRiyV0M4 is 6KB 2023-07-21T21:46:04.652 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.30NRiyV0M4 2023-07-21T21:46:04.668 DEBUG:teuthology.orchestra.run.smithi066:> sudo /bin/ls /boot/loader/entries || true 2023-07-21T21:46:04.755 INFO:teuthology.orchestra.run.smithi066.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-21T21:46:04.755 INFO:teuthology.orchestra.run.smithi066.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-21T21:46:04.755 INFO:teuthology.orchestra.run.smithi066.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc.conf 2023-07-21T21:46:04.756 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc 2023-07-21T21:46:04.874 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2023-07-21T21:46:34.877 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-07-21T21:46:34.878 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T21:46:34.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:46:53.424 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-07-21T21:47:02.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T21:47:02.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:47:05.488 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-07-21T21:47:17.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T21:47:17.490 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:47:20.560 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2023-07-21T21:47:35.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T21:47:35.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:48:40.566 DEBUG:teuthology.orchestra.remote:timed out 2023-07-21T21:48:58.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T21:48:58.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T21:49:03.930 DEBUG:teuthology.orchestra.run.smithi066:> true 2023-07-21T21:49:04.344 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T21:49:04.344 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc1-gc685da6f56cc"... 2023-07-21T21:49:04.345 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2023-07-21T21:49:04.388 INFO:teuthology.orchestra.run.smithi066.stdout:6.5.0-rc1-gc685da6f56cc 2023-07-21T21:49:04.388 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-gc685da6f56cc vs 6.5.0-rc1-gc685da6f56cc 2023-07-21T21:49:04.389 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-21T21:49:04.389 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-21T21:49:05.389 DEBUG:teuthology.parallel:result is None 2023-07-21T21:49:05.390 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-21T21:49:05.397 INFO:teuthology.task.internal:Creating test directory... 2023-07-21T21:49:05.398 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T21:49:05.402 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T21:49:05.405 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T21:49:05.446 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-21T21:49:05.454 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-21T21:49:05.467 INFO:teuthology.task.internal:Creating archive directory... 2023-07-21T21:49:05.467 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T21:49:05.472 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T21:49:05.476 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T21:49:05.674 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-21T21:49:05.683 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-21T21:49:05.683 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T21:49:05.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:05.702 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T21:49:05.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:05.722 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T21:49:05.726 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T21:49:05.744 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T21:49:05.807 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:49:05.817 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:49:05.829 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:49:05.844 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:49:05.857 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:49:05.893 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T21:49:05.896 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-21T21:49:05.906 INFO:teuthology.task.internal:Configuring sudo... 2023-07-21T21:49:05.906 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T21:49:05.910 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T21:49:05.912 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T21:49:05.978 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-21T21:49:05.991 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-21T21:49:05.992 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T21:49:05.996 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T21:49:05.998 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T21:49:06.035 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:11.154 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:11.272 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:16.354 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:16.430 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T21:49:16.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T21:49:16.506 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:21.560 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:21.706 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:25.522 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:25.556 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T21:49:25.557 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T21:49:25.647 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:25.701 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T21:49:25.758 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:25.806 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T21:49:25.849 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T21:49:25.849 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T21:49:25.940 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2023-07-21T21:49:25.945 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2023-07-21T21:49:25.947 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2023-07-21T21:49:26.015 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T21:49:26.034 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T21:49:26.067 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T21:49:26.494 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-21T21:49:26.504 INFO:teuthology.task.internal:Starting timer... 2023-07-21T21:49:26.505 INFO:teuthology.run_tasks:Running task pcp... 2023-07-21T21:49:26.530 INFO:teuthology.run_tasks:Running task selinux... 2023-07-21T21:49:26.541 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-21T21:49:26.542 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2023-07-21T21:49:26.698 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2023-07-21T21:49:26.701 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2023-07-21T21:49:26.765 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2023-07-21T21:49:26.767 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2023-07-21T21:49:26.829 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2023-07-21T21:49:26.831 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-21T21:49:26.831 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2023-07-21T21:49:26.899 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2023-07-21T21:49:26.899 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2023-07-21T21:49:26.922 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2023-07-21T21:49:26.922 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2023-07-21T21:49:26.943 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2023-07-21T21:49:26.943 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2023-07-21T21:49:26.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T21:49:27.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:27.020 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T21:49:27.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:27.048 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T21:49:27.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T21:49:27.077 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-21T21:49:27.078 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2023-07-21T21:49:27.150 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2023-07-21T21:49:27.177 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2023-07-21T21:49:27.205 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-21T21:49:27.220 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-21T21:49:27.227 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-21T21:49:32.364 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-21T21:49:32.386 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-21T21:49:32.388 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-21T22:02:58.557 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2023-07-21T22:02:58.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T22:02:58.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:03:03.656 DEBUG:teuthology.orchestra.run.smithi066:> true 2023-07-21T22:03:03.761 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2023-07-21T22:03:03.761 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-21T22:03:03.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:03:08.831 DEBUG:teuthology.orchestra.run.smithi104:> true 2023-07-21T22:03:08.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2023-07-21T22:03:08.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2023-07-21T22:03:08.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T22:03:13.975 DEBUG:teuthology.orchestra.run.smithi149:> true 2023-07-21T22:03:14.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2023-07-21T22:03:14.048 INFO:teuthology.run_tasks:Running task clock... 2023-07-21T22:03:14.058 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-21T22:03:14.059 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T22:03:14.059 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:03:14.065 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T22:03:14.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:03:14.067 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T22:03:14.067 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:03:14.100 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T22:03:14.116 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T22:03:14.122 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T22:03:14.137 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T22:03:14.146 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T22:03:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T22:03:14.205 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2023-07-21T22:03:14.215 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2023-07-21T22:03:14.218 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2023-07-21T22:03:14.227 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2023-07-21T22:03:14.233 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T22:03:14.242 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T22:03:14.247 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T22:03:14.256 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T22:03:14.279 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2023-07-21T22:03:14.310 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2023-07-21T22:03:14.321 INFO:teuthology.orchestra.run.smithi149.stderr:bash: ntpq: command not found 2023-07-21T22:03:14.325 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:03:14.325 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2023-07-21T22:03:14.325 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.326 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.326 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.326 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.332 INFO:teuthology.orchestra.run.smithi104.stderr:bash: ntpq: command not found 2023-07-21T22:03:14.336 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:03:14.336 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2023-07-21T22:03:14.336 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.337 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.337 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.337 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.345 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T22:03:14.381 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T22:03:14.488 INFO:teuthology.orchestra.run.smithi066.stderr:bash: ntpq: command not found 2023-07-21T22:03:14.495 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:03:14.496 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2023-07-21T22:03:14.496 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.496 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.496 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.496 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:03:14.497 INFO:teuthology.run_tasks:Running task pexec... 2023-07-21T22:03:14.507 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-21T22:03:14.508 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:03:14.509 DEBUG:teuthology.orchestra.run.smithi066:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T22:03:14.509 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi104.front.sepia.ceph.com 2023-07-21T22:03:14.510 DEBUG:teuthology.orchestra.run.smithi104:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T22:03:14.510 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi149.front.sepia.ceph.com 2023-07-21T22:03:14.510 DEBUG:teuthology.orchestra.run.smithi149:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T22:03:14.878 INFO:teuthology.orchestra.run.smithi104.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:14.882 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:15.030 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:15.681 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:05:58 ago on Fri 21 Jul 2023 09:57:17 PM UTC. 2023-07-21T22:03:15.705 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:05:46 ago on Fri 21 Jul 2023 09:57:29 PM UTC. 2023-07-21T22:03:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:31 ago on Fri 21 Jul 2023 10:00:44 PM UTC. 2023-07-21T22:03:17.554 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2023-07-21T22:03:17.555 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2023-07-21T22:03:17.555 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2023-07-21T22:03:17.555 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2023-07-21T22:03:17.555 INFO:teuthology.orchestra.run.smithi104.stdout:Resetting modules: 2023-07-21T22:03:17.555 INFO:teuthology.orchestra.run.smithi104.stdout: container-tools 2023-07-21T22:03:17.556 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:17.556 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2023-07-21T22:03:17.556 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2023-07-21T22:03:17.556 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:17.593 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2023-07-21T22:03:17.594 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2023-07-21T22:03:17.594 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2023-07-21T22:03:17.594 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2023-07-21T22:03:17.594 INFO:teuthology.orchestra.run.smithi149.stdout:Resetting modules: 2023-07-21T22:03:17.595 INFO:teuthology.orchestra.run.smithi149.stdout: container-tools 2023-07-21T22:03:17.595 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:17.595 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2023-07-21T22:03:17.595 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2023-07-21T22:03:17.595 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:18.016 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-07-21T22:03:18.018 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2023-07-21T22:03:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2023-07-21T22:03:18.018 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2023-07-21T22:03:18.018 INFO:teuthology.orchestra.run.smithi066.stdout:Resetting modules: 2023-07-21T22:03:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools 2023-07-21T22:03:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:18.019 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2023-07-21T22:03:18.019 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2023-07-21T22:03:18.019 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:18.531 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:06:01 ago on Fri 21 Jul 2023 09:57:17 PM UTC. 2023-07-21T22:03:18.572 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:05:49 ago on Fri 21 Jul 2023 09:57:29 PM UTC. 2023-07-21T22:03:19.361 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:34 ago on Fri 21 Jul 2023 10:00:44 PM UTC. 2023-07-21T22:03:21.628 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2023-07-21T22:03:21.737 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2023-07-21T22:03:22.173 INFO:teuthology.orchestra.run.smithi104.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:22.223 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-07-21T22:03:22.306 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:22.880 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:22.941 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:06:05 ago on Fri 21 Jul 2023 09:57:17 PM UTC. 2023-07-21T22:03:23.094 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:05:53 ago on Fri 21 Jul 2023 09:57:29 PM UTC. 2023-07-21T22:03:23.886 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:39 ago on Fri 21 Jul 2023 10:00:44 PM UTC. 2023-07-21T22:03:24.660 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2023-07-21T22:03:24.755 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================================ 2023-07-21T22:03:24.755 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2023-07-21T22:03:24.755 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================================ 2023-07-21T22:03:24.755 INFO:teuthology.orchestra.run.smithi104.stdout:Installing group/module packages: 2023-07-21T22:03:24.755 INFO:teuthology.orchestra.run.smithi104.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T22:03:24.755 INFO:teuthology.orchestra.run.smithi104.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2023-07-21T22:03:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout:Installing module profiles: 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout: container-tools/common 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout:Enabling module streams: 2023-07-21T22:03:24.757 INFO:teuthology.orchestra.run.smithi104.stdout: container-tools rhel8 2023-07-21T22:03:24.758 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:24.758 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2023-07-21T22:03:24.758 INFO:teuthology.orchestra.run.smithi104.stdout:============================================================================================================ 2023-07-21T22:03:24.758 INFO:teuthology.orchestra.run.smithi104.stdout:Install 10 Packages 2023-07-21T22:03:24.758 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:24.759 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 18 M 2023-07-21T22:03:24.759 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 65 M 2023-07-21T22:03:24.759 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2023-07-21T22:03:24.830 INFO:teuthology.orchestra.run.smithi104.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-07-21T22:03:24.832 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2023-07-21T22:03:24.847 INFO:teuthology.orchestra.run.smithi104.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 276 kB/s | 25 kB 00:00 2023-07-21T22:03:24.864 INFO:teuthology.orchestra.run.smithi104.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 382 kB/s | 41 kB 00:00 2023-07-21T22:03:24.881 INFO:teuthology.orchestra.run.smithi104.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 974 kB/s | 49 kB 00:00 2023-07-21T22:03:24.915 INFO:teuthology.orchestra.run.smithi104.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 7.6 MB/s | 524 kB 00:00 2023-07-21T22:03:24.927 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================================ 2023-07-21T22:03:24.927 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2023-07-21T22:03:24.927 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================================ 2023-07-21T22:03:24.927 INFO:teuthology.orchestra.run.smithi149.stdout:Installing group/module packages: 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T22:03:24.928 INFO:teuthology.orchestra.run.smithi149.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout:Installing module profiles: 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout: container-tools/common 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:24.929 INFO:teuthology.orchestra.run.smithi149.stdout:Enabling module streams: 2023-07-21T22:03:24.930 INFO:teuthology.orchestra.run.smithi149.stdout: container-tools rhel8 2023-07-21T22:03:24.930 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:24.930 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2023-07-21T22:03:24.930 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================================ 2023-07-21T22:03:24.930 INFO:teuthology.orchestra.run.smithi149.stdout:Install 10 Packages 2023-07-21T22:03:24.930 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:24.931 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 18 M 2023-07-21T22:03:24.931 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 65 M 2023-07-21T22:03:24.931 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2023-07-21T22:03:24.940 INFO:teuthology.orchestra.run.smithi104.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T22:03:24.982 INFO:teuthology.orchestra.run.smithi104.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.1 MB/s | 213 kB 00:00 2023-07-21T22:03:25.005 INFO:teuthology.orchestra.run.smithi149.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.2 MB/s | 94 kB 00:00 2023-07-21T22:03:25.030 INFO:teuthology.orchestra.run.smithi149.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 247 kB/s | 25 kB 00:00 2023-07-21T22:03:25.055 INFO:teuthology.orchestra.run.smithi149.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 324 kB/s | 41 kB 00:00 2023-07-21T22:03:25.066 INFO:teuthology.orchestra.run.smithi104.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-07-21T22:03:25.081 INFO:teuthology.orchestra.run.smithi149.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-07-21T22:03:25.122 INFO:teuthology.orchestra.run.smithi149.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-07-21T22:03:25.148 INFO:teuthology.orchestra.run.smithi149.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-21T22:03:25.173 INFO:teuthology.orchestra.run.smithi149.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-21T22:03:25.174 INFO:teuthology.orchestra.run.smithi104.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 30 MB/s | 7.0 MB 00:00 2023-07-21T22:03:25.257 INFO:teuthology.orchestra.run.smithi149.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-07-21T22:03:25.257 INFO:teuthology.orchestra.run.smithi104.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 20 MB/s | 8.0 MB 00:00 2023-07-21T22:03:25.258 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:03:25.258 INFO:teuthology.orchestra.run.smithi104.stdout:Total 36 MB/s | 18 MB 00:00 2023-07-21T22:03:25.374 INFO:teuthology.orchestra.run.smithi149.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 31 MB/s | 7.0 MB 00:00 2023-07-21T22:03:25.457 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2023-07-21T22:03:25.474 INFO:teuthology.orchestra.run.smithi149.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-07-21T22:03:25.474 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:03:25.474 INFO:teuthology.orchestra.run.smithi149.stdout:Total 33 MB/s | 18 MB 00:00 2023-07-21T22:03:25.475 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2023-07-21T22:03:25.475 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2023-07-21T22:03:25.658 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2023-07-21T22:03:25.662 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2023-07-21T22:03:25.676 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2023-07-21T22:03:25.696 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2023-07-21T22:03:25.696 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2023-07-21T22:03:25.880 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2023-07-21T22:03:25.884 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2023-07-21T22:03:25.952 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-07-21T22:03:26.057 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================================ 2023-07-21T22:03:26.057 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2023-07-21T22:03:26.057 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================================ 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout:Installing group/module packages: 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T22:03:26.058 INFO:teuthology.orchestra.run.smithi066.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T22:03:26.059 INFO:teuthology.orchestra.run.smithi066.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T22:03:26.059 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2023-07-21T22:03:26.059 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T22:03:26.059 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T22:03:26.059 INFO:teuthology.orchestra.run.smithi066.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout:Installing module profiles: 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools/common 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling module streams: 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout: container-tools rhel8 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:26.060 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2023-07-21T22:03:26.061 INFO:teuthology.orchestra.run.smithi066.stdout:============================================================================================================ 2023-07-21T22:03:26.061 INFO:teuthology.orchestra.run.smithi066.stdout:Install 10 Packages 2023-07-21T22:03:26.061 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:26.062 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 18 M 2023-07-21T22:03:26.062 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 65 M 2023-07-21T22:03:26.062 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2023-07-21T22:03:26.129 INFO:teuthology.orchestra.run.smithi066.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 362 kB/s | 25 kB 00:00 2023-07-21T22:03:26.179 INFO:teuthology.orchestra.run.smithi066.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 344 kB/s | 41 kB 00:00 2023-07-21T22:03:26.205 INFO:teuthology.orchestra.run.smithi066.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 650 kB/s | 94 kB 00:00 2023-07-21T22:03:26.207 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2023-07-21T22:03:26.230 INFO:teuthology.orchestra.run.smithi066.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 491 kB/s | 49 kB 00:00 2023-07-21T22:03:26.272 INFO:teuthology.orchestra.run.smithi066.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.5 MB/s | 524 kB 00:00 2023-07-21T22:03:26.297 INFO:teuthology.orchestra.run.smithi066.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-21T22:03:26.331 INFO:teuthology.orchestra.run.smithi066.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-07-21T22:03:26.389 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2023-07-21T22:03:26.401 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T22:03:26.448 INFO:teuthology.orchestra.run.smithi066.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-07-21T22:03:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T22:03:26.567 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:03:26.640 INFO:teuthology.orchestra.run.smithi066.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 20 MB/s | 7.0 MB 00:00 2023-07-21T22:03:26.715 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:03:26.740 INFO:teuthology.orchestra.run.smithi066.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 15 MB/s | 8.0 MB 00:00 2023-07-21T22:03:26.741 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:03:26.741 INFO:teuthology.orchestra.run.smithi066.stdout:Total 27 MB/s | 18 MB 00:00 2023-07-21T22:03:26.779 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T22:03:26.935 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T22:03:26.973 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T22:03:27.086 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2023-07-21T22:03:27.111 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2023-07-21T22:03:27.111 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2023-07-21T22:03:27.137 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T22:03:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T22:03:27.511 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T22:03:27.651 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2023-07-21T22:03:27.656 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2023-07-21T22:03:28.075 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T22:03:28.236 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2023-07-21T22:03:28.282 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T22:03:28.577 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T22:03:28.757 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:03:28.959 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T22:03:29.028 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T22:03:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T22:03:29.165 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T22:03:29.283 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T22:03:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T22:03:29.357 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T22:03:29.505 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T22:03:29.518 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:03:29.656 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T22:03:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:03:29.786 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:03:29.786 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T22:03:29.786 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:03:29.786 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T22:03:29.787 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T22:03:29.787 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T22:03:29.787 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T22:03:29.787 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T22:03:29.787 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T22:03:29.787 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T22:03:29.972 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T22:03:29.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T22:03:29.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T22:03:30.517 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T22:03:30.979 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T22:03:30.980 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:06:13 ago on Fri 21 Jul 2023 09:57:17 PM UTC. 2023-07-21T22:03:31.135 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T22:03:31.135 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:06:01 ago on Fri 21 Jul 2023 09:57:29 PM UTC. 2023-07-21T22:03:31.528 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T22:03:31.934 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T22:03:32.120 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T22:03:32.344 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:03:32.748 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T22:03:32.749 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T22:03:32.749 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T22:03:32.749 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T22:03:32.749 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T22:03:32.749 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T22:03:32.749 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T22:03:32.750 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T22:03:32.750 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T22:03:32.750 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T22:03:34.199 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T22:03:34.199 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:49 ago on Fri 21 Jul 2023 10:00:44 PM UTC. 2023-07-21T22:03:34.498 INFO:teuthology.orchestra.run.smithi104.stdout:Installed products updated. 2023-07-21T22:03:34.523 INFO:teuthology.orchestra.run.smithi149.stdout:Installed products updated. 2023-07-21T22:03:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:34.736 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2023-07-21T22:03:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.736 INFO:teuthology.orchestra.run.smithi104.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:03:34.737 INFO:teuthology.orchestra.run.smithi104.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T22:03:34.738 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:03:34.738 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2023-07-21T22:03:34.740 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T22:03:34.741 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T22:03:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:03:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T22:03:34.742 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:03:34.742 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2023-07-21T22:03:34.886 DEBUG:teuthology.parallel:result is None 2023-07-21T22:03:34.924 DEBUG:teuthology.parallel:result is None 2023-07-21T22:03:37.457 INFO:teuthology.orchestra.run.smithi066.stdout:Installed products updated. 2023-07-21T22:03:37.700 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:37.700 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T22:03:37.701 INFO:teuthology.orchestra.run.smithi066.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:37.702 INFO:teuthology.orchestra.run.smithi066.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T22:03:37.702 INFO:teuthology.orchestra.run.smithi066.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T22:03:37.702 INFO:teuthology.orchestra.run.smithi066.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T22:03:37.702 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:03:37.702 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-07-21T22:03:37.921 DEBUG:teuthology.parallel:result is None 2023-07-21T22:03:37.921 INFO:teuthology.run_tasks:Running task install... 2023-07-21T22:03:37.930 DEBUG:teuthology.task.install:project ceph 2023-07-21T22:03:37.931 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'}} 2023-07-21T22:03:37.931 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'} 2023-07-21T22:03:37.931 INFO:teuthology.task.install:Using flavor: default 2023-07-21T22:03:37.938 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-21T22:03:37.938 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-21T22:03:37.939 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T22:03:37.939 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:03:37.942 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T22:03:37.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:03:37.943 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'tag': None, 'wait_for_package': False} 2023-07-21T22:03:37.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:03:38.516 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T22:03:38.516 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T22:03:38.522 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T22:03:38.522 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T22:03:38.530 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/ 2023-07-21T22:03:38.530 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4735.gbffe049d 2023-07-21T22:03:38.993 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T22:03:38.994 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:03:38.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T22:03:38.998 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T22:03:38.999 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:03:38.999 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T22:03:39.005 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T22:03:39.006 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:03:39.006 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T22:03:39.028 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T22:03:39.029 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T22:03:39.036 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T22:03:39.037 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T22:03:39.078 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T22:03:39.078 DEBUG:teuthology.orchestra.run.smithi066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T22:03:39.095 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T22:03:39.107 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T22:03:39.174 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T22:03:39.184 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T22:03:39.191 DEBUG:teuthology.orchestra.run.smithi066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T22:03:39.240 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2023-07-21T22:03:39.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2023-07-21T22:03:39.249 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2023-07-21T22:03:39.251 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2023-07-21T22:03:39.294 DEBUG:teuthology.orchestra.run.smithi066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T22:03:39.367 INFO:teuthology.orchestra.run.smithi066.stdout:check_obsoletes = 1 2023-07-21T22:03:39.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum clean all 2023-07-21T22:03:39.630 INFO:teuthology.orchestra.run.smithi104.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:39.644 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:39.893 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:39.975 INFO:teuthology.orchestra.run.smithi104.stdout:44 files removed 2023-07-21T22:03:39.991 INFO:teuthology.orchestra.run.smithi149.stdout:44 files removed 2023-07-21T22:03:40.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T22:03:40.047 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T22:03:40.299 INFO:teuthology.orchestra.run.smithi066.stdout:44 files removed 2023-07-21T22:03:40.364 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T22:03:40.384 INFO:teuthology.orchestra.run.smithi104.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:40.394 INFO:teuthology.orchestra.run.smithi149.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:40.866 INFO:teuthology.orchestra.run.smithi066.stdout:Updating Subscription Management repositories. 2023-07-21T22:03:42.513 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 78 kB/s | 81 kB 00:01 2023-07-21T22:03:42.533 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 72 kB/s | 81 kB 00:01 2023-07-21T22:03:42.871 INFO:teuthology.orchestra.run.smithi066.stdout:ceph packages for x86_64 71 kB/s | 81 kB 00:01 2023-07-21T22:03:43.511 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 9.8 kB/s | 9.4 kB 00:00 2023-07-21T22:03:43.530 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 9.8 kB/s | 9.4 kB 00:00 2023-07-21T22:03:43.865 INFO:teuthology.orchestra.run.smithi066.stdout:ceph noarch packages 9.8 kB/s | 9.4 kB 00:00 2023-07-21T22:03:44.483 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 1.6 kB/s | 1.5 kB 00:00 2023-07-21T22:03:44.528 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-21T22:03:44.866 INFO:teuthology.orchestra.run.smithi066.stdout:ceph source packages 1.5 kB/s | 1.5 kB 00:00 2023-07-21T22:03:45.103 INFO:teuthology.orchestra.run.smithi149.stdout:Copr repo for python3-asyncssh owned by ceph 5.8 kB/s | 3.5 kB 00:00 2023-07-21T22:03:45.136 INFO:teuthology.orchestra.run.smithi104.stdout:Copr repo for python3-asyncssh owned by ceph 5.9 kB/s | 3.5 kB 00:00 2023-07-21T22:03:45.576 INFO:teuthology.orchestra.run.smithi066.stdout:Copr repo for python3-asyncssh owned by ceph 5.0 kB/s | 3.5 kB 00:00 2023-07-21T22:03:49.077 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 4.0 MB/s | 16 MB 00:03 2023-07-21T22:03:49.088 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 3.9 MB/s | 16 MB 00:03 2023-07-21T22:03:51.942 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 16 MB 00:06 2023-07-21T22:03:52.536 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 380 kB/s | 24 kB 00:00 2023-07-21T22:03:52.711 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 221 kB/s | 24 kB 00:00 2023-07-21T22:03:53.429 INFO:teuthology.orchestra.run.smithi104.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-07-21T22:03:53.637 INFO:teuthology.orchestra.run.smithi149.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-07-21T22:03:56.334 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 430 kB/s | 24 kB 00:00 2023-07-21T22:03:57.528 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-07-21T22:04:01.814 INFO:teuthology.orchestra.run.smithi149.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-07-21T22:04:01.964 INFO:teuthology.orchestra.run.smithi104.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-07-21T22:04:07.093 INFO:teuthology.orchestra.run.smithi066.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 47 MB 00:01 2023-07-21T22:04:12.793 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:12.794 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:12.808 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:12.809 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:12.810 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:04:12.811 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:04:12.813 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:04:12.816 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:04:12.817 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:04:12.846 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:12.847 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:12.862 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:12.863 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:12.865 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:04:12.866 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:04:12.868 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:04:12.871 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:04:12.872 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:04:12.947 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2023-07-21T22:04:12.960 INFO:teuthology.orchestra.run.smithi104.stdout:=========================================================================================================================== 2023-07-21T22:04:12.960 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2023-07-21T22:04:12.960 INFO:teuthology.orchestra.run.smithi104.stdout:=========================================================================================================================== 2023-07-21T22:04:12.960 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2023-07-21T22:04:12.960 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:04:12.961 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:04:12.962 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:04:12.963 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:04:12.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2023-07-21T22:04:12.965 INFO:teuthology.orchestra.run.smithi104.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:04:12.966 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:04:12.967 INFO:teuthology.orchestra.run.smithi104.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:04:12.968 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:04:12.969 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:04:12.970 INFO:teuthology.orchestra.run.smithi104.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:04:12.971 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:04:12.971 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:04:12.971 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:04:12.971 INFO:teuthology.orchestra.run.smithi104.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:04:12.971 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:04:12.971 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:04:12.972 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:04:12.973 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:04:12.974 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:04:12.975 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:04:12.975 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:04:12.975 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:04:12.975 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:04:12.975 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:04:12.975 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:04:12.976 INFO:teuthology.orchestra.run.smithi104.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:04:12.977 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout:Enabling module streams: 2023-07-21T22:04:12.978 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql 10 2023-07-21T22:04:12.979 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:04:12.979 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2023-07-21T22:04:12.979 INFO:teuthology.orchestra.run.smithi104.stdout:=========================================================================================================================== 2023-07-21T22:04:12.979 INFO:teuthology.orchestra.run.smithi104.stdout:Install 128 Packages 2023-07-21T22:04:12.979 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 3 Packages 2023-07-21T22:04:12.979 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:04:12.980 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 205 M 2023-07-21T22:04:12.980 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2023-07-21T22:04:13.006 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2023-07-21T22:04:13.016 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================================================== 2023-07-21T22:04:13.016 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================================================== 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:04:13.017 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:04:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:04:13.019 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:04:13.020 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:04:13.021 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2023-07-21T22:04:13.022 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:04:13.023 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:04:13.024 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:04:13.025 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:04:13.026 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:04:13.027 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:04:13.028 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:13.029 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:04:13.030 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:04:13.031 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:04:13.032 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:04:13.033 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout:Enabling module streams: 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql 10 2023-07-21T22:04:13.034 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:04:13.035 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2023-07-21T22:04:13.035 INFO:teuthology.orchestra.run.smithi149.stdout:=========================================================================================================================== 2023-07-21T22:04:13.035 INFO:teuthology.orchestra.run.smithi149.stdout:Install 128 Packages 2023-07-21T22:04:13.035 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 3 Packages 2023-07-21T22:04:13.035 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:04:13.036 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 205 M 2023-07-21T22:04:13.036 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2023-07-21T22:04:13.513 INFO:teuthology.orchestra.run.smithi104.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 13 kB/s | 6.4 kB 00:00 2023-07-21T22:04:14.336 INFO:teuthology.orchestra.run.smithi149.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 10 kB/s | 6.4 kB 00:00 2023-07-21T22:04:14.930 INFO:teuthology.orchestra.run.smithi104.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 598 kB/s | 848 kB 00:01 2023-07-21T22:04:15.197 INFO:teuthology.orchestra.run.smithi104.stdout:(3/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 2.5 MB/s | 5.4 MB 00:02 2023-07-21T22:04:15.230 INFO:teuthology.orchestra.run.smithi104.stdout:(4/131): ceph-immutable-object-cache-18.0.0-473 552 kB/s | 165 kB 00:00 2023-07-21T22:04:15.415 INFO:teuthology.orchestra.run.smithi149.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 786 kB/s | 848 kB 00:01 2023-07-21T22:04:15.682 INFO:teuthology.orchestra.run.smithi149.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 621 kB/s | 165 kB 00:00 2023-07-21T22:04:15.756 INFO:teuthology.orchestra.run.smithi104.stdout:(5/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 2.9 MB/s | 1.5 MB 00:00 2023-07-21T22:04:16.207 INFO:teuthology.orchestra.run.smithi104.stdout:(6/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-07-21T22:04:16.807 INFO:teuthology.orchestra.run.smithi104.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 4.4 MB/s | 4.6 MB 00:01 2023-07-21T22:04:18.173 INFO:teuthology.orchestra.run.smithi149.stdout:(4/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 940 kB/s | 2.3 MB 00:02 2023-07-21T22:04:18.431 INFO:teuthology.orchestra.run.smithi104.stdout:(8/131): ceph-radosgw-18.0.0-4735.gbffe049d.el8 8.3 MB/s | 13 MB 00:01 2023-07-21T22:04:18.573 INFO:teuthology.orchestra.run.smithi104.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 172 kB/s | 24 kB 00:00 2023-07-21T22:04:19.617 INFO:teuthology.orchestra.run.smithi066.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:19.618 INFO:teuthology.orchestra.run.smithi066.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T22:04:19.634 INFO:teuthology.orchestra.run.smithi066.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:19.635 INFO:teuthology.orchestra.run.smithi066.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T22:04:19.636 INFO:teuthology.orchestra.run.smithi066.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T22:04:19.637 INFO:teuthology.orchestra.run.smithi066.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T22:04:19.640 INFO:teuthology.orchestra.run.smithi066.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T22:04:19.643 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T22:04:19.645 INFO:teuthology.orchestra.run.smithi066.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T22:04:19.798 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2023-07-21T22:04:19.811 INFO:teuthology.orchestra.run.smithi066.stdout:=========================================================================================================================== 2023-07-21T22:04:19.812 INFO:teuthology.orchestra.run.smithi066.stdout: Package Arch Version Repository Size 2023-07-21T22:04:19.812 INFO:teuthology.orchestra.run.smithi066.stdout:=========================================================================================================================== 2023-07-21T22:04:19.812 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2023-07-21T22:04:19.812 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T22:04:19.812 INFO:teuthology.orchestra.run.smithi066.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T22:04:19.812 INFO:teuthology.orchestra.run.smithi066.stdout: ceph x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.4 k 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 5.4 M 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 848 k 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 165 k 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 1.5 M 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 129 k 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 1.7 M 2023-07-21T22:04:19.813 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 7.4 M 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 47 k 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 13 M 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 48 M 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 258 k 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 164 k 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 258 k 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 18 k 2023-07-21T22:04:19.814 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 31 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 771 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 139 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T22:04:19.815 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 197 k 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 370 k 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 364 k 2023-07-21T22:04:19.816 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 107 k 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 94 k 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.4 M 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 179 k 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrading: 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T22:04:19.817 INFO:teuthology.orchestra.run.smithi066.stdout: librados2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.7 M 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 3.6 M 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 22 M 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 23 k 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 2.3 M 2023-07-21T22:04:19.818 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 240 k 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 4.6 M 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 18 M 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4735.gbffe049d.el8 ceph-noarch 15 k 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 24 k 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T22:04:19.819 INFO:teuthology.orchestra.run.smithi066.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 137 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T22:04:19.820 INFO:teuthology.orchestra.run.smithi066.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 473 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2 x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 6.7 M 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T22:04:19.821 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T22:04:19.822 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 45 k 2023-07-21T22:04:19.823 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common x86_64 2:18.0.0-4735.gbffe049d.el8 ceph 119 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T22:04:19.824 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T22:04:19.825 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T22:04:19.826 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T22:04:19.827 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T22:04:19.828 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T22:04:19.829 INFO:teuthology.orchestra.run.smithi066.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling module streams: 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql 10 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout:=========================================================================================================================== 2023-07-21T22:04:19.830 INFO:teuthology.orchestra.run.smithi066.stdout:Install 128 Packages 2023-07-21T22:04:19.831 INFO:teuthology.orchestra.run.smithi066.stdout:Upgrade 3 Packages 2023-07-21T22:04:19.831 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:04:19.831 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 205 M 2023-07-21T22:04:19.831 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2023-07-21T22:04:20.149 INFO:teuthology.orchestra.run.smithi149.stdout:(5/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 803 kB/s | 1.5 MB 00:01 2023-07-21T22:04:20.584 INFO:teuthology.orchestra.run.smithi066.stdout:(1/131): ceph-18.0.0-4735.gbffe049d.el8.x86_64. 13 kB/s | 6.4 kB 00:00 2023-07-21T22:04:21.556 INFO:teuthology.orchestra.run.smithi104.stdout:(10/131): ceph-test-18.0.0-4735.gbffe049d.el8.x 16 MB/s | 48 MB 00:02 2023-07-21T22:04:21.690 INFO:teuthology.orchestra.run.smithi104.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 1.9 MB/s | 258 kB 00:00 2023-07-21T22:04:21.759 INFO:teuthology.orchestra.run.smithi066.stdout:(2/131): ceph-fuse-18.0.0-4735.gbffe049d.el8.x8 722 kB/s | 848 kB 00:01 2023-07-21T22:04:21.848 INFO:teuthology.orchestra.run.smithi104.stdout:(12/131): libcephfs-devel-18.0.0-4735.gbffe049d 196 kB/s | 31 kB 00:00 2023-07-21T22:04:22.009 INFO:teuthology.orchestra.run.smithi066.stdout:(3/131): ceph-immutable-object-cache-18.0.0-473 663 kB/s | 165 kB 00:00 2023-07-21T22:04:22.024 INFO:teuthology.orchestra.run.smithi104.stdout:(13/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 4.3 MB/s | 771 kB 00:00 2023-07-21T22:04:22.166 INFO:teuthology.orchestra.run.smithi104.stdout:(14/131): libcephsqlite-18.0.0-4735.gbffe049d.e 969 kB/s | 137 kB 00:00 2023-07-21T22:04:22.308 INFO:teuthology.orchestra.run.smithi104.stdout:(15/131): librados-devel-18.0.0-4735.gbffe049d. 981 kB/s | 139 kB 00:00 2023-07-21T22:04:22.467 INFO:teuthology.orchestra.run.smithi104.stdout:(16/131): libradosstriper1-18.0.0-4735.gbffe049 2.9 MB/s | 473 kB 00:00 2023-07-21T22:04:23.068 INFO:teuthology.orchestra.run.smithi066.stdout:(4/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 1.8 MB/s | 5.4 MB 00:02 2023-07-21T22:04:23.091 INFO:teuthology.orchestra.run.smithi149.stdout:(6/131): ceph-base-18.0.0-4735.gbffe049d.el8.x8 587 kB/s | 5.4 MB 00:09 2023-07-21T22:04:23.142 INFO:teuthology.orchestra.run.smithi104.stdout:(17/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 10 MB/s | 6.7 MB 00:00 2023-07-21T22:04:23.292 INFO:teuthology.orchestra.run.smithi104.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 298 kB/s | 45 kB 00:00 2023-07-21T22:04:23.434 INFO:teuthology.orchestra.run.smithi104.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 839 kB/s | 119 kB 00:00 2023-07-21T22:04:23.576 INFO:teuthology.orchestra.run.smithi104.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.4 MB/s | 197 kB 00:00 2023-07-21T22:04:23.584 INFO:teuthology.orchestra.run.smithi066.stdout:(5/131): ceph-mds-18.0.0-4735.gbffe049d.el8.x86 1.5 MB/s | 2.3 MB 00:01 2023-07-21T22:04:23.626 INFO:teuthology.orchestra.run.smithi066.stdout:(6/131): ceph-mgr-18.0.0-4735.gbffe049d.el8.x86 2.8 MB/s | 1.5 MB 00:00 2023-07-21T22:04:23.734 INFO:teuthology.orchestra.run.smithi104.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 2.3 MB/s | 370 kB 00:00 2023-07-21T22:04:23.893 INFO:teuthology.orchestra.run.smithi104.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 2.2 MB/s | 364 kB 00:00 2023-07-21T22:04:24.064 INFO:teuthology.orchestra.run.smithi104.stdout:(23/131): python3-rgw-18.0.0-4735.gbffe049d.el8 640 kB/s | 107 kB 00:00 2023-07-21T22:04:24.218 INFO:teuthology.orchestra.run.smithi104.stdout:(24/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 597 kB/s | 94 kB 00:00 2023-07-21T22:04:24.727 INFO:teuthology.orchestra.run.smithi104.stdout:(25/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 6.8 MB/s | 3.4 MB 00:00 2023-07-21T22:04:24.877 INFO:teuthology.orchestra.run.smithi104.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-21T22:04:25.027 INFO:teuthology.orchestra.run.smithi104.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 154 kB/s | 23 kB 00:00 2023-07-21T22:04:25.220 INFO:teuthology.orchestra.run.smithi104.stdout:(28/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x8 2.0 MB/s | 18 MB 00:09 2023-07-21T22:04:25.244 INFO:teuthology.orchestra.run.smithi104.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 594 kB/s | 129 kB 00:00 2023-07-21T22:04:25.282 INFO:teuthology.orchestra.run.smithi149.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 920 kB/s | 4.6 MB 00:05 2023-07-21T22:04:25.368 INFO:teuthology.orchestra.run.smithi066.stdout:(7/131): ceph-mon-18.0.0-4735.gbffe049d.el8.x86 2.6 MB/s | 4.6 MB 00:01 2023-07-21T22:04:25.512 INFO:teuthology.orchestra.run.smithi104.stdout:(30/131): ceph-common-18.0.0-4735.gbffe049d.el8 1.8 MB/s | 22 MB 00:12 2023-07-21T22:04:25.895 INFO:teuthology.orchestra.run.smithi104.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 628 kB/s | 240 kB 00:00 2023-07-21T22:04:26.112 INFO:teuthology.orchestra.run.smithi104.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 219 kB/s | 47 kB 00:00 2023-07-21T22:04:26.270 INFO:teuthology.orchestra.run.smithi104.stdout:(33/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 1.6 MB/s | 1.7 MB 00:01 2023-07-21T22:04:26.362 INFO:teuthology.orchestra.run.smithi104.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 6.6 MB/s | 7.4 MB 00:01 2023-07-21T22:04:26.388 INFO:teuthology.orchestra.run.smithi104.stdout:(35/131): ceph-prometheus-alerts-18.0.0-4735.gb 53 kB/s | 15 kB 00:00 2023-07-21T22:04:26.455 INFO:teuthology.orchestra.run.smithi104.stdout:(36/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.4 MB/s | 258 kB 00:00 2023-07-21T22:04:26.572 INFO:teuthology.orchestra.run.smithi104.stdout:(37/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 98 kB/s | 18 kB 00:00 2023-07-21T22:04:26.988 INFO:teuthology.orchestra.run.smithi104.stdout:(38/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 262 kB/s | 164 kB 00:00 2023-07-21T22:04:27.055 INFO:teuthology.orchestra.run.smithi104.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 122 kB/s | 59 kB 00:00 2023-07-21T22:04:27.064 INFO:teuthology.orchestra.run.smithi066.stdout:(8/131): ceph-common-18.0.0-4735.gbffe049d.el8. 3.2 MB/s | 22 MB 00:06 2023-07-21T22:04:27.205 INFO:teuthology.orchestra.run.smithi104.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 408 kB/s | 306 kB 00:00 2023-07-21T22:04:27.244 INFO:teuthology.orchestra.run.smithi066.stdout:(9/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x86 4.9 MB/s | 18 MB 00:03 2023-07-21T22:04:27.269 INFO:teuthology.orchestra.run.smithi066.stdout:(10/131): ceph-selinux-18.0.0-4735.gbffe049d.el 119 kB/s | 24 kB 00:00 2023-07-21T22:04:27.389 INFO:teuthology.orchestra.run.smithi104.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 238 kB/s | 44 kB 00:00 2023-07-21T22:04:27.436 INFO:teuthology.orchestra.run.smithi066.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 1.5 MB/s | 258 kB 00:00 2023-07-21T22:04:27.573 INFO:teuthology.orchestra.run.smithi104.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 128 kB/s | 75 kB 00:00 2023-07-21T22:04:27.595 INFO:teuthology.orchestra.run.smithi066.stdout:(12/131): libcephfs-devel-18.0.0-4735.gbffe049d 196 kB/s | 31 kB 00:00 2023-07-21T22:04:27.598 INFO:teuthology.orchestra.run.smithi104.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 877 kB/s | 476 kB 00:00 2023-07-21T22:04:27.624 INFO:teuthology.orchestra.run.smithi104.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 791 kB/s | 185 kB 00:00 2023-07-21T22:04:27.749 INFO:teuthology.orchestra.run.smithi104.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 108 kB/s | 16 kB 00:00 2023-07-21T22:04:27.774 INFO:teuthology.orchestra.run.smithi104.stdout:(46/131): python3-cachetools-3.1.1-4.el8.noarch 164 kB/s | 33 kB 00:00 2023-07-21T22:04:27.799 INFO:teuthology.orchestra.run.smithi104.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 984 kB/s | 173 kB 00:00 2023-07-21T22:04:27.903 INFO:teuthology.orchestra.run.smithi066.stdout:(13/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 2.4 MB/s | 771 kB 00:00 2023-07-21T22:04:27.950 INFO:teuthology.orchestra.run.smithi104.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 74 kB/s | 11 kB 00:00 2023-07-21T22:04:28.033 INFO:teuthology.orchestra.run.smithi104.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.3 MB/s | 384 kB 00:00 2023-07-21T22:04:28.059 INFO:teuthology.orchestra.run.smithi104.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 286 kB/s | 81 kB 00:00 2023-07-21T22:04:28.100 INFO:teuthology.orchestra.run.smithi104.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 117 kB/s | 18 kB 00:00 2023-07-21T22:04:28.103 INFO:teuthology.orchestra.run.smithi066.stdout:(14/131): libcephsqlite-18.0.0-4735.gbffe049d.e 687 kB/s | 137 kB 00:00 2023-07-21T22:04:28.242 INFO:teuthology.orchestra.run.smithi104.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 265 kB/s | 49 kB 00:00 2023-07-21T22:04:28.259 INFO:teuthology.orchestra.run.smithi104.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 370 kB/s | 59 kB 00:00 2023-07-21T22:04:28.390 INFO:teuthology.orchestra.run.smithi066.stdout:(15/131): librados-devel-18.0.0-4735.gbffe049d. 485 kB/s | 139 kB 00:00 2023-07-21T22:04:28.468 INFO:teuthology.orchestra.run.smithi104.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 267 kB/s | 60 kB 00:00 2023-07-21T22:04:28.501 INFO:teuthology.orchestra.run.smithi104.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-07-21T22:04:28.535 INFO:teuthology.orchestra.run.smithi104.stdout:(56/131): python3-kubernetes-11.0.0-6.el8.noarc 2.3 MB/s | 1.2 MB 00:00 2023-07-21T22:04:28.554 INFO:teuthology.orchestra.run.smithi066.stdout:(16/131): libradosstriper1-18.0.0-4735.gbffe049 2.8 MB/s | 473 kB 00:00 2023-07-21T22:04:28.585 INFO:teuthology.orchestra.run.smithi104.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-07-21T22:04:28.644 INFO:teuthology.orchestra.run.smithi104.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 236 kB/s | 34 kB 00:00 2023-07-21T22:04:28.694 INFO:teuthology.orchestra.run.smithi104.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-07-21T22:04:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:(8/131): ceph-osd-18.0.0-4735.gbffe049d.el8.x86 3.1 MB/s | 18 MB 00:05 2023-07-21T22:04:28.786 INFO:teuthology.orchestra.run.smithi104.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 132 kB/s | 19 kB 00:00 2023-07-21T22:04:28.803 INFO:teuthology.orchestra.run.smithi104.stdout:(61/131): python3-rsa-4.8-1.el8.noarch.rpm 283 kB/s | 61 kB 00:00 2023-07-21T22:04:28.828 INFO:teuthology.orchestra.run.smithi104.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 179 kB/s | 24 kB 00:00 2023-07-21T22:04:28.915 INFO:teuthology.orchestra.run.smithi149.stdout:(9/131): ceph-selinux-18.0.0-4735.gbffe049d.el8 146 kB/s | 24 kB 00:00 2023-07-21T22:04:28.953 INFO:teuthology.orchestra.run.smithi104.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 175 kB/s | 29 kB 00:00 2023-07-21T22:04:28.987 INFO:teuthology.orchestra.run.smithi104.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-07-21T22:04:29.037 INFO:teuthology.orchestra.run.smithi104.stdout:(65/131): python3-trustme-0.6.0-4.el8.noarch.rp 117 kB/s | 27 kB 00:00 2023-07-21T22:04:29.096 INFO:teuthology.orchestra.run.smithi104.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-07-21T22:04:29.121 INFO:teuthology.orchestra.run.smithi104.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-07-21T22:04:29.129 INFO:teuthology.orchestra.run.smithi066.stdout:(17/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 3.6 MB/s | 13 MB 00:03 2023-07-21T22:04:29.238 INFO:teuthology.orchestra.run.smithi104.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 162 kB/s | 23 kB 00:00 2023-07-21T22:04:29.263 INFO:teuthology.orchestra.run.smithi104.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 378 kB/s | 85 kB 00:00 2023-07-21T22:04:29.280 INFO:teuthology.orchestra.run.smithi104.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-07-21T22:04:29.296 INFO:teuthology.orchestra.run.smithi066.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 269 kB/s | 45 kB 00:00 2023-07-21T22:04:29.322 INFO:teuthology.orchestra.run.smithi104.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 800 kB/s | 47 kB 00:00 2023-07-21T22:04:29.372 INFO:teuthology.orchestra.run.smithi104.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.3 MB/s | 200 kB 00:00 2023-07-21T22:04:29.397 INFO:teuthology.orchestra.run.smithi104.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 567 kB/s | 43 kB 00:00 2023-07-21T22:04:29.431 INFO:teuthology.orchestra.run.smithi104.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-07-21T22:04:29.457 INFO:teuthology.orchestra.run.smithi104.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 817 kB/s | 48 kB 00:00 2023-07-21T22:04:29.482 INFO:teuthology.orchestra.run.smithi104.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-21T22:04:29.507 INFO:teuthology.orchestra.run.smithi104.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 851 kB/s | 43 kB 00:00 2023-07-21T22:04:29.532 INFO:teuthology.orchestra.run.smithi104.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-21T22:04:29.574 INFO:teuthology.orchestra.run.smithi104.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.4 MB/s | 229 kB 00:00 2023-07-21T22:04:29.608 INFO:teuthology.orchestra.run.smithi104.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-07-21T22:04:29.633 INFO:teuthology.orchestra.run.smithi104.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-07-21T22:04:29.675 INFO:teuthology.orchestra.run.smithi104.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 3.1 MB/s | 1.7 MB 00:00 2023-07-21T22:04:29.700 INFO:teuthology.orchestra.run.smithi104.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.6 MB/s | 155 kB 00:00 2023-07-21T22:04:29.743 INFO:teuthology.orchestra.run.smithi104.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2023-07-21T22:04:29.768 INFO:teuthology.orchestra.run.smithi104.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 194 kB/s | 18 kB 00:00 2023-07-21T22:04:29.793 INFO:teuthology.orchestra.run.smithi104.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-07-21T22:04:29.827 INFO:teuthology.orchestra.run.smithi104.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.9 MB/s | 605 kB 00:00 2023-07-21T22:04:29.852 INFO:teuthology.orchestra.run.smithi104.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2023-07-21T22:04:29.869 INFO:teuthology.orchestra.run.smithi104.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-21T22:04:29.894 INFO:teuthology.orchestra.run.smithi104.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-07-21T22:04:29.919 INFO:teuthology.orchestra.run.smithi104.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-21T22:04:29.944 INFO:teuthology.orchestra.run.smithi104.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T22:04:29.970 INFO:teuthology.orchestra.run.smithi104.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 770 kB/s | 39 kB 00:00 2023-07-21T22:04:30.003 INFO:teuthology.orchestra.run.smithi104.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-21T22:04:30.037 INFO:teuthology.orchestra.run.smithi104.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-21T22:04:30.062 INFO:teuthology.orchestra.run.smithi104.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-07-21T22:04:30.121 INFO:teuthology.orchestra.run.smithi104.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.6 MB/s | 2.3 MB 00:00 2023-07-21T22:04:30.146 INFO:teuthology.orchestra.run.smithi104.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-07-21T22:04:30.171 INFO:teuthology.orchestra.run.smithi104.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-21T22:04:30.196 INFO:teuthology.orchestra.run.smithi104.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-21T22:04:30.230 INFO:teuthology.orchestra.run.smithi104.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-21T22:04:30.255 INFO:teuthology.orchestra.run.smithi104.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 663 kB/s | 39 kB 00:00 2023-07-21T22:04:30.288 INFO:teuthology.orchestra.run.smithi066.stdout:(19/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 3.9 MB/s | 6.7 MB 00:01 2023-07-21T22:04:30.297 INFO:teuthology.orchestra.run.smithi104.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-21T22:04:30.331 INFO:teuthology.orchestra.run.smithi104.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-07-21T22:04:30.346 INFO:teuthology.orchestra.run.smithi066.stdout:(20/131): python3-ceph-common-18.0.0-4735.gbffe 113 kB/s | 119 kB 00:01 2023-07-21T22:04:30.357 INFO:teuthology.orchestra.run.smithi104.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-21T22:04:30.382 INFO:teuthology.orchestra.run.smithi104.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-21T22:04:30.424 INFO:teuthology.orchestra.run.smithi104.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-21T22:04:30.439 INFO:teuthology.orchestra.run.smithi066.stdout:(21/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.3 MB/s | 197 kB 00:00 2023-07-21T22:04:30.465 INFO:teuthology.orchestra.run.smithi104.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 639 kB/s | 27 kB 00:00 2023-07-21T22:04:30.480 INFO:teuthology.orchestra.run.smithi066.stdout:(22/131): python3-rados-18.0.0-4735.gbffe049d.e 2.7 MB/s | 370 kB 00:00 2023-07-21T22:04:30.507 INFO:teuthology.orchestra.run.smithi104.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T22:04:30.541 INFO:teuthology.orchestra.run.smithi104.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-07-21T22:04:30.593 INFO:teuthology.orchestra.run.smithi066.stdout:(23/131): python3-rbd-18.0.0-4735.gbffe049d.el8 2.3 MB/s | 364 kB 00:00 2023-07-21T22:04:30.623 INFO:teuthology.orchestra.run.smithi066.stdout:(24/131): python3-rgw-18.0.0-4735.gbffe049d.el8 752 kB/s | 107 kB 00:00 2023-07-21T22:04:30.624 INFO:teuthology.orchestra.run.smithi104.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-07-21T22:04:30.666 INFO:teuthology.orchestra.run.smithi104.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-07-21T22:04:30.709 INFO:teuthology.orchestra.run.smithi104.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-21T22:04:30.715 INFO:teuthology.orchestra.run.smithi066.stdout:(25/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 778 kB/s | 94 kB 00:00 2023-07-21T22:04:30.776 INFO:teuthology.orchestra.run.smithi104.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2023-07-21T22:04:30.801 INFO:teuthology.orchestra.run.smithi104.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-07-21T22:04:30.818 INFO:teuthology.orchestra.run.smithi104.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.9 MB/s | 126 kB 00:00 2023-07-21T22:04:30.835 INFO:teuthology.orchestra.run.smithi104.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 595 kB/s | 20 kB 00:00 2023-07-21T22:04:30.848 INFO:teuthology.orchestra.run.smithi066.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.3 MB/s | 179 kB 00:00 2023-07-21T22:04:30.860 INFO:teuthology.orchestra.run.smithi104.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-07-21T22:04:30.893 INFO:teuthology.orchestra.run.smithi104.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-21T22:04:30.936 INFO:teuthology.orchestra.run.smithi104.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T22:04:30.982 INFO:teuthology.orchestra.run.smithi066.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 173 kB/s | 23 kB 00:00 2023-07-21T22:04:31.069 INFO:teuthology.orchestra.run.smithi104.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-07-21T22:04:31.111 INFO:teuthology.orchestra.run.smithi104.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T22:04:31.153 INFO:teuthology.orchestra.run.smithi104.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-21T22:04:31.228 INFO:teuthology.orchestra.run.smithi104.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-07-21T22:04:31.328 INFO:teuthology.orchestra.run.smithi104.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-07-21T22:04:31.341 INFO:teuthology.orchestra.run.smithi066.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 360 kB/s | 129 kB 00:00 2023-07-21T22:04:31.370 INFO:teuthology.orchestra.run.smithi104.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-07-21T22:04:31.421 INFO:teuthology.orchestra.run.smithi104.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-07-21T22:04:31.496 INFO:teuthology.orchestra.run.smithi104.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-07-21T22:04:31.546 INFO:teuthology.orchestra.run.smithi104.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-07-21T22:04:31.691 INFO:teuthology.orchestra.run.smithi066.stdout:(29/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 3.2 MB/s | 3.4 MB 00:01 2023-07-21T22:04:31.975 INFO:teuthology.orchestra.run.smithi066.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 2.7 MB/s | 1.7 MB 00:00 2023-07-21T22:04:32.133 INFO:teuthology.orchestra.run.smithi066.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.5 MB/s | 240 kB 00:00 2023-07-21T22:04:32.300 INFO:teuthology.orchestra.run.smithi066.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 284 kB/s | 47 kB 00:00 2023-07-21T22:04:32.476 INFO:teuthology.orchestra.run.smithi066.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 83 kB/s | 15 kB 00:00 2023-07-21T22:04:32.546 INFO:teuthology.orchestra.run.smithi104.stdout:(130/131): librados2-18.0.0-4735.gbffe049d.el8. 3.1 MB/s | 3.7 MB 00:01 2023-07-21T22:04:32.626 INFO:teuthology.orchestra.run.smithi066.stdout:(34/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.7 MB/s | 258 kB 00:00 2023-07-21T22:04:32.666 INFO:teuthology.orchestra.run.smithi149.stdout:(10/131): ceph-common-18.0.0-4735.gbffe049d.el8 1.2 MB/s | 22 MB 00:18 2023-07-21T22:04:32.810 INFO:teuthology.orchestra.run.smithi066.stdout:(35/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 894 kB/s | 164 kB 00:00 2023-07-21T22:04:32.947 INFO:teuthology.orchestra.run.smithi149.stdout:(11/131): cephfs-mirror-18.0.0-4735.gbffe049d.e 919 kB/s | 258 kB 00:00 2023-07-21T22:04:32.960 INFO:teuthology.orchestra.run.smithi066.stdout:(36/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 120 kB/s | 18 kB 00:00 2023-07-21T22:04:33.071 INFO:teuthology.orchestra.run.smithi104.stdout:(131/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 2.2 MB/s | 3.6 MB 00:01 2023-07-21T22:04:33.074 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:04:33.074 INFO:teuthology.orchestra.run.smithi104.stdout:Total 10 MB/s | 205 MB 00:20 2023-07-21T22:04:33.124 INFO:teuthology.orchestra.run.smithi149.stdout:(12/131): libcephfs-devel-18.0.0-4735.gbffe049d 175 kB/s | 31 kB 00:00 2023-07-21T22:04:33.616 INFO:teuthology.orchestra.run.smithi149.stdout:(13/131): ceph-test-18.0.0-4735.gbffe049d.el8.x 10 MB/s | 48 MB 00:04 2023-07-21T22:04:33.716 INFO:teuthology.orchestra.run.smithi149.stdout:(14/131): libcephfs2-18.0.0-4735.gbffe049d.el8. 1.3 MB/s | 771 kB 00:00 2023-07-21T22:04:33.735 INFO:teuthology.orchestra.run.smithi066.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 395 kB/s | 306 kB 00:00 2023-07-21T22:04:33.758 INFO:teuthology.orchestra.run.smithi149.stdout:(15/131): libcephsqlite-18.0.0-4735.gbffe049d.e 970 kB/s | 137 kB 00:00 2023-07-21T22:04:33.867 INFO:teuthology.orchestra.run.smithi149.stdout:(16/131): librados-devel-18.0.0-4735.gbffe049d. 925 kB/s | 139 kB 00:00 2023-07-21T22:04:33.894 INFO:teuthology.orchestra.run.smithi066.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 3.4 MB/s | 7.4 MB 00:02 2023-07-21T22:04:33.917 INFO:teuthology.orchestra.run.smithi149.stdout:(17/131): libradosstriper1-18.0.0-4735.gbffe049 2.9 MB/s | 473 kB 00:00 2023-07-21T22:04:33.919 INFO:teuthology.orchestra.run.smithi066.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 321 kB/s | 59 kB 00:00 2023-07-21T22:04:33.949 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2023-07-21T22:04:34.051 INFO:teuthology.orchestra.run.smithi149.stdout:(18/131): python3-ceph-argparse-18.0.0-4735.gbf 335 kB/s | 45 kB 00:00 2023-07-21T22:04:34.068 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2023-07-21T22:04:34.069 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2023-07-21T22:04:34.193 INFO:teuthology.orchestra.run.smithi149.stdout:(19/131): python3-ceph-common-18.0.0-4735.gbffe 839 kB/s | 119 kB 00:00 2023-07-21T22:04:34.211 INFO:teuthology.orchestra.run.smithi066.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.6 MB/s | 476 kB 00:00 2023-07-21T22:04:34.343 INFO:teuthology.orchestra.run.smithi149.stdout:(20/131): python3-cephfs-18.0.0-4735.gbffe049d. 1.3 MB/s | 197 kB 00:00 2023-07-21T22:04:34.361 INFO:teuthology.orchestra.run.smithi066.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 290 kB/s | 44 kB 00:00 2023-07-21T22:04:34.437 INFO:teuthology.orchestra.run.smithi066.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 138 kB/s | 75 kB 00:00 2023-07-21T22:04:34.493 INFO:teuthology.orchestra.run.smithi149.stdout:(21/131): python3-rados-18.0.0-4735.gbffe049d.e 2.4 MB/s | 370 kB 00:00 2023-07-21T22:04:34.551 INFO:teuthology.orchestra.run.smithi066.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 980 kB/s | 185 kB 00:00 2023-07-21T22:04:34.612 INFO:teuthology.orchestra.run.smithi066.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 188 kB/s | 33 kB 00:00 2023-07-21T22:04:34.652 INFO:teuthology.orchestra.run.smithi149.stdout:(22/131): python3-rbd-18.0.0-4735.gbffe049d.el8 2.2 MB/s | 364 kB 00:00 2023-07-21T22:04:34.689 INFO:teuthology.orchestra.run.smithi066.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 119 kB/s | 16 kB 00:00 2023-07-21T22:04:34.794 INFO:teuthology.orchestra.run.smithi149.stdout:(23/131): python3-rgw-18.0.0-4735.gbffe049d.el8 754 kB/s | 107 kB 00:00 2023-07-21T22:04:34.863 INFO:teuthology.orchestra.run.smithi066.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-07-21T22:04:34.944 INFO:teuthology.orchestra.run.smithi149.stdout:(24/131): rbd-fuse-18.0.0-4735.gbffe049d.el8.x8 629 kB/s | 94 kB 00:00 2023-07-21T22:04:34.946 INFO:teuthology.orchestra.run.smithi066.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 517 kB/s | 173 kB 00:00 2023-07-21T22:04:35.005 INFO:teuthology.orchestra.run.smithi066.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 572 kB/s | 81 kB 00:00 2023-07-21T22:04:35.080 INFO:teuthology.orchestra.run.smithi066.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-21T22:04:35.131 INFO:teuthology.orchestra.run.smithi066.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-07-21T22:04:35.180 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2023-07-21T22:04:35.184 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2023-07-21T22:04:35.273 INFO:teuthology.orchestra.run.smithi066.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 344 kB/s | 49 kB 00:00 2023-07-21T22:04:35.428 INFO:teuthology.orchestra.run.smithi149.stdout:(25/131): rbd-mirror-18.0.0-4735.gbffe049d.el8. 7.1 MB/s | 3.4 MB 00:00 2023-07-21T22:04:35.431 INFO:teuthology.orchestra.run.smithi066.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 371 kB/s | 59 kB 00:00 2023-07-21T22:04:35.557 INFO:teuthology.orchestra.run.smithi066.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 2.4 MB/s | 1.2 MB 00:00 2023-07-21T22:04:35.574 INFO:teuthology.orchestra.run.smithi066.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 424 kB/s | 60 kB 00:00 2023-07-21T22:04:35.587 INFO:teuthology.orchestra.run.smithi149.stdout:(26/131): rbd-nbd-18.0.0-4735.gbffe049d.el8.x86 1.1 MB/s | 179 kB 00:00 2023-07-21T22:04:35.716 INFO:teuthology.orchestra.run.smithi066.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-07-21T22:04:35.733 INFO:teuthology.orchestra.run.smithi066.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 98 kB/s | 16 kB 00:00 2023-07-21T22:04:35.737 INFO:teuthology.orchestra.run.smithi149.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4735.g 154 kB/s | 23 kB 00:00 2023-07-21T22:04:35.875 INFO:teuthology.orchestra.run.smithi066.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 212 kB/s | 34 kB 00:00 2023-07-21T22:04:35.879 INFO:teuthology.orchestra.run.smithi149.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4735.gbffe049 910 kB/s | 129 kB 00:00 2023-07-21T22:04:35.900 INFO:teuthology.orchestra.run.smithi066.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.2 MB/s | 196 kB 00:00 2023-07-21T22:04:36.034 INFO:teuthology.orchestra.run.smithi066.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 389 kB/s | 61 kB 00:00 2023-07-21T22:04:36.059 INFO:teuthology.orchestra.run.smithi066.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 118 kB/s | 19 kB 00:00 2023-07-21T22:04:36.104 INFO:teuthology.orchestra.run.smithi149.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4735.gbffe0 7.6 MB/s | 1.7 MB 00:00 2023-07-21T22:04:36.408 INFO:teuthology.orchestra.run.smithi066.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 64 kB/s | 24 kB 00:00 2023-07-21T22:04:36.435 INFO:teuthology.orchestra.run.smithi066.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 78 kB/s | 29 kB 00:00 2023-07-21T22:04:36.560 INFO:teuthology.orchestra.run.smithi066.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 182 kB/s | 27 kB 00:00 2023-07-21T22:04:36.593 INFO:teuthology.orchestra.run.smithi066.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.6 MB/s | 255 kB 00:00 2023-07-21T22:04:36.727 INFO:teuthology.orchestra.run.smithi066.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2023-07-21T22:04:36.752 INFO:teuthology.orchestra.run.smithi066.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 386 kB/s | 61 kB 00:00 2023-07-21T22:04:36.878 INFO:teuthology.orchestra.run.smithi066.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 568 kB/s | 85 kB 00:00 2023-07-21T22:04:37.038 INFO:teuthology.orchestra.run.smithi149.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 8.6 MB/s | 7.4 MB 00:00 2023-07-21T22:04:37.039 INFO:teuthology.orchestra.run.smithi066.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 144 kB/s | 23 kB 00:00 2023-07-21T22:04:37.039 INFO:teuthology.orchestra.run.smithi066.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-21T22:04:37.040 INFO:teuthology.orchestra.run.smithi066.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 808 kB/s | 47 kB 00:00 2023-07-21T22:04:37.040 INFO:teuthology.orchestra.run.smithi066.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-21T22:04:37.081 INFO:teuthology.orchestra.run.smithi066.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-21T22:04:37.122 INFO:teuthology.orchestra.run.smithi066.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-21T22:04:37.130 INFO:teuthology.orchestra.run.smithi149.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4735.gbf 1.5 MB/s | 240 kB 00:00 2023-07-21T22:04:37.172 INFO:teuthology.orchestra.run.smithi066.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 960 kB/s | 48 kB 00:00 2023-07-21T22:04:37.206 INFO:teuthology.orchestra.run.smithi066.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.6 MB/s | 87 kB 00:00 2023-07-21T22:04:37.239 INFO:teuthology.orchestra.run.smithi066.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.3 MB/s | 43 kB 00:00 2023-07-21T22:04:37.280 INFO:teuthology.orchestra.run.smithi149.stdout:(32/131): ceph-mgr-rook-18.0.0-4735.gbffe049d.e 316 kB/s | 47 kB 00:00 2023-07-21T22:04:37.306 INFO:teuthology.orchestra.run.smithi066.stdout:(77/131): thrift-0.13.0-2.el8.x86_64.rpm 4.1 MB/s | 1.7 MB 00:00 2023-07-21T22:04:37.340 INFO:teuthology.orchestra.run.smithi066.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 1.1 MB/s | 109 kB 00:00 2023-07-21T22:04:37.390 INFO:teuthology.orchestra.run.smithi066.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-07-21T22:04:37.422 INFO:teuthology.orchestra.run.smithi149.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4735.gb 103 kB/s | 15 kB 00:00 2023-07-21T22:04:37.423 INFO:teuthology.orchestra.run.smithi066.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-07-21T22:04:37.449 INFO:teuthology.orchestra.run.smithi066.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-07-21T22:04:37.474 INFO:teuthology.orchestra.run.smithi066.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-21T22:04:37.516 INFO:teuthology.orchestra.run.smithi066.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2023-07-21T22:04:37.541 INFO:teuthology.orchestra.run.smithi066.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-07-21T22:04:37.572 INFO:teuthology.orchestra.run.smithi149.stdout:(34/131): ceph-volume-18.0.0-4735.gbffe049d.el8 1.7 MB/s | 258 kB 00:00 2023-07-21T22:04:37.584 INFO:teuthology.orchestra.run.smithi066.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-07-21T22:04:37.609 INFO:teuthology.orchestra.run.smithi066.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-07-21T22:04:37.651 INFO:teuthology.orchestra.run.smithi066.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-21T22:04:37.718 INFO:teuthology.orchestra.run.smithi066.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-07-21T22:04:37.731 INFO:teuthology.orchestra.run.smithi149.stdout:(35/131): cephadm-18.0.0-4735.gbffe049d.el8.noa 1.0 MB/s | 164 kB 00:00 2023-07-21T22:04:37.753 INFO:teuthology.orchestra.run.smithi066.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.8 MB/s | 60 kB 00:00 2023-07-21T22:04:37.802 INFO:teuthology.orchestra.run.smithi066.stdout:(90/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 11 MB/s | 2.3 MB 00:00 2023-07-21T22:04:37.827 INFO:teuthology.orchestra.run.smithi066.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.3 MB/s | 98 kB 00:00 2023-07-21T22:04:37.844 INFO:teuthology.orchestra.run.smithi066.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-07-21T22:04:37.861 INFO:teuthology.orchestra.run.smithi066.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-07-21T22:04:37.889 INFO:teuthology.orchestra.run.smithi149.stdout:(36/131): cephfs-top-18.0.0-4735.gbffe049d.el8. 113 kB/s | 18 kB 00:00 2023-07-21T22:04:37.903 INFO:teuthology.orchestra.run.smithi066.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-21T22:04:37.928 INFO:teuthology.orchestra.run.smithi066.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-21T22:04:37.953 INFO:teuthology.orchestra.run.smithi066.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-21T22:04:37.995 INFO:teuthology.orchestra.run.smithi066.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T22:04:38.029 INFO:teuthology.orchestra.run.smithi066.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-21T22:04:38.071 INFO:teuthology.orchestra.run.smithi066.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-21T22:04:38.113 INFO:teuthology.orchestra.run.smithi066.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-07-21T22:04:38.180 INFO:teuthology.orchestra.run.smithi066.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 585 kB/s | 39 kB 00:00 2023-07-21T22:04:38.239 INFO:teuthology.orchestra.run.smithi066.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-07-21T22:04:38.297 INFO:teuthology.orchestra.run.smithi066.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 9.0 MB/s | 538 kB 00:00 2023-07-21T22:04:38.340 INFO:teuthology.orchestra.run.smithi066.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-21T22:04:38.381 INFO:teuthology.orchestra.run.smithi066.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-21T22:04:38.432 INFO:teuthology.orchestra.run.smithi066.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-21T22:04:38.674 INFO:teuthology.orchestra.run.smithi066.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 18 MB/s | 14 MB 00:00 2023-07-21T22:04:38.681 INFO:teuthology.orchestra.run.smithi149.stdout:(37/131): librgw2-18.0.0-4735.gbffe049d.el8.x86 1.4 MB/s | 6.7 MB 00:04 2023-07-21T22:04:38.716 INFO:teuthology.orchestra.run.smithi066.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 643 kB/s | 27 kB 00:00 2023-07-21T22:04:38.757 INFO:teuthology.orchestra.run.smithi066.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T22:04:38.833 INFO:teuthology.orchestra.run.smithi066.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-07-21T22:04:38.858 INFO:teuthology.orchestra.run.smithi066.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 438 kB/s | 44 kB 00:00 2023-07-21T22:04:38.909 INFO:teuthology.orchestra.run.smithi066.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-07-21T22:04:38.950 INFO:teuthology.orchestra.run.smithi066.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-21T22:04:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-07-21T22:04:39.059 INFO:teuthology.orchestra.run.smithi066.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-07-21T22:04:39.110 INFO:teuthology.orchestra.run.smithi066.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.0 MB/s | 126 kB 00:00 2023-07-21T22:04:39.135 INFO:teuthology.orchestra.run.smithi066.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 265 kB/s | 20 kB 00:00 2023-07-21T22:04:39.160 INFO:teuthology.orchestra.run.smithi066.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-21T22:04:39.202 INFO:teuthology.orchestra.run.smithi066.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-21T22:04:39.244 INFO:teuthology.orchestra.run.smithi066.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T22:04:39.386 INFO:teuthology.orchestra.run.smithi066.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 22 MB/s | 4.9 MB 00:00 2023-07-21T22:04:39.428 INFO:teuthology.orchestra.run.smithi066.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T22:04:39.531 INFO:teuthology.orchestra.run.smithi066.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-07-21T22:04:39.935 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2023-07-21T22:04:39.985 INFO:teuthology.orchestra.run.smithi066.stdout:(124/131): ceph-test-18.0.0-4735.gbffe049d.el8. 3.7 MB/s | 48 MB 00:12 2023-07-21T22:04:40.021 INFO:teuthology.orchestra.run.smithi066.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 597 kB/s | 354 kB 00:00 2023-07-21T22:04:40.104 INFO:teuthology.orchestra.run.smithi066.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.6 MB/s | 808 kB 00:00 2023-07-21T22:04:40.130 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:04:40.154 INFO:teuthology.orchestra.run.smithi066.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-21T22:04:40.272 INFO:teuthology.orchestra.run.smithi066.stdout:(128/131): postgresql-server-10.21-2.module+el8 6.8 MB/s | 5.1 MB 00:00 2023-07-21T22:04:40.321 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:04:40.330 INFO:teuthology.orchestra.run.smithi066.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-21T22:04:40.481 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:04:40.572 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:04:40.806 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:04:41.225 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:04:41.414 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:04:41.781 INFO:teuthology.orchestra.run.smithi149.stdout:(38/131): ceph-radosgw-18.0.0-4735.gbffe049d.el 837 kB/s | 13 MB 00:16 2023-07-21T22:04:41.816 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:04:41.981 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:04:42.118 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:04:42.330 INFO:teuthology.orchestra.run.smithi066.stdout:(130/131): librados2-18.0.0-4735.gbffe049d.el8. 1.6 MB/s | 3.7 MB 00:02 2023-07-21T22:04:42.748 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:04:42.868 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:04:43.020 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:04:43.175 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T22:04:43.408 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:04:43.619 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:04:44.007 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:04:44.235 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:04:44.355 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:04:44.408 INFO:teuthology.orchestra.run.smithi066.stdout:(131/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 857 kB/s | 3.6 MB 00:04 2023-07-21T22:04:44.411 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:04:44.411 INFO:teuthology.orchestra.run.smithi066.stdout:Total 8.3 MB/s | 205 MB 00:24 2023-07-21T22:04:44.447 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:04:44.564 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:04:44.704 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:04:44.900 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T22:04:45.078 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T22:04:45.236 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T22:04:45.363 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:04:45.487 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:04:45.614 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:04:46.138 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2023-07-21T22:04:46.229 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:04:46.283 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2023-07-21T22:04:46.283 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2023-07-21T22:04:46.389 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:04:46.986 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:04:47.146 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:04:47.841 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:04:47.891 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2023-07-21T22:04:47.896 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2023-07-21T22:04:48.078 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T22:04:48.269 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T22:04:48.514 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:04:48.720 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:04:48.848 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:04:49.067 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:04:49.254 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:04:51.020 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:04:52.032 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T22:04:52.342 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T22:04:52.615 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T22:04:52.823 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T22:04:52.974 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2023-07-21T22:04:53.040 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:04:53.144 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:04:53.231 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:04:53.357 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:04:53.410 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:04:53.536 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:04:53.568 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:04:53.711 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T22:04:53.727 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:04:53.846 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T22:04:53.953 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:04:54.020 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:04:54.095 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:04:54.305 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T22:04:54.439 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:04:54.523 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T22:04:54.687 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:04:54.704 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:04:54.748 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:04:54.940 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T22:04:55.107 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T22:04:55.189 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:04:55.268 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T22:04:55.386 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:04:55.430 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T22:04:55.542 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:04:55.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T22:04:55.720 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:04:55.764 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T22:04:55.884 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:04:55.941 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T22:04:56.051 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:04:56.147 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T22:04:56.235 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T22:04:56.506 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:04:56.526 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T22:04:56.768 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:04:56.796 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T22:04:57.006 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T22:04:57.210 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:04:57.222 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T22:04:57.440 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T22:04:57.458 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:04:57.644 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:04:57.734 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T22:04:57.775 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:04:57.911 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T22:04:57.911 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:04:58.092 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T22:04:58.134 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:04:58.335 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T22:04:58.340 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T22:04:58.506 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T22:04:58.546 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T22:04:58.672 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T22:04:58.742 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T22:04:58.911 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:04:58.950 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T22:04:59.061 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:04:59.197 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T22:04:59.203 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:04:59.424 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T22:04:59.645 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T22:04:59.810 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:04:59.946 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:04:59.987 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:05:00.111 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:05:00.294 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:05:00.509 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:05:00.656 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:05:00.804 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:05:00.865 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:05:01.019 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:05:01.044 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:05:01.221 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:05:01.403 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:05:01.597 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:05:01.733 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:05:01.933 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:05:01.960 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:05:02.174 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:05:02.243 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T22:05:02.366 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:05:02.483 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T22:05:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:05:02.750 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:05:02.817 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:05:02.952 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:05:03.000 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:05:03.108 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:05:03.219 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:05:03.385 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:05:03.577 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:05:04.322 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:05:04.597 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:05:04.859 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:05:05.142 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:05:05.419 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:05:05.636 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:05:05.775 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:05:06.404 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:05:06.551 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:06.627 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:06.785 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:05:06.981 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T22:05:07.391 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T22:05:07.672 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T22:05:07.961 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T22:05:08.208 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:05:08.426 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:05:08.652 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:05:08.852 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:05:09.053 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T22:05:09.221 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T22:05:09.443 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:05:09.528 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:05:09.834 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T22:05:10.092 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T22:05:10.251 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:05:10.365 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:10.366 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:05:10.576 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:10.604 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T22:05:10.824 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T22:05:10.996 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T22:05:11.158 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:11.207 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T22:05:11.371 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T22:05:11.402 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:11.486 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:11.543 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T22:05:11.633 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:11.778 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T22:05:12.004 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T22:05:12.480 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T22:05:12.827 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T22:05:13.085 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T22:05:13.344 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T22:05:13.613 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T22:05:13.970 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T22:05:14.186 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T22:05:14.426 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T22:05:14.648 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T22:05:14.877 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T22:05:15.050 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T22:05:15.444 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T22:05:15.736 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T22:05:16.020 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T22:05:16.288 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T22:05:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:05:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:05:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:05:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:05:17.456 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:05:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:05:17.844 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:05:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:05:18.292 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:05:18.488 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:05:18.650 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:05:18.923 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:05:19.150 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:05:19.383 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:05:19.675 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:05:19.902 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:05:20.129 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:05:20.372 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:05:21.817 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:05:22.132 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:05:22.459 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:05:22.842 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:05:23.140 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:05:23.393 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:05:24.327 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:05:24.533 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:24.628 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:05:24.808 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:05:30.377 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:30.553 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:30.553 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /dev 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /run 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2023-07-21T22:05:30.554 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2023-07-21T22:05:30.555 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /dev 2023-07-21T22:05:30.555 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:05:30.608 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:30.871 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:05:31.350 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:05:31.649 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:31.810 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:05:31.957 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:32.443 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:05:32.614 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:05:32.833 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:05:33.029 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:05:33.487 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:05:33.646 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:05:33.663 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:05:35.296 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:05:35.458 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:05:35.585 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:05:35.793 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:05:35.961 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:05:36.175 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:05:36.446 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:05:37.404 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:05:37.561 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:05:37.651 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:05:38.016 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:05:38.209 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:05:38.245 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:05:38.386 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:05:38.398 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:05:39.710 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:05:39.884 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:05:39.917 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:05:40.061 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:05:40.103 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:05:40.245 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:05:40.591 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:05:40.728 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:05:45.240 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:05:46.098 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:05:46.434 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T22:05:46.612 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 117/134 2023-07-21T22:05:46.885 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T22:05:47.051 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:05:47.240 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:05:47.455 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T22:05:47.614 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:05:47.769 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:05:49.866 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:05:50.126 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:05:50.144 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:05:50.326 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:05:50.617 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:05:50.894 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:05:51.257 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:05:51.417 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:05:51.501 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:05:51.700 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:05:51.713 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:05:51.884 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:05:51.892 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:05:52.027 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T22:05:52.027 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:05:52.236 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:05:52.237 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:05:52.456 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:05:52.649 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:05:53.154 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:05:53.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:05:53.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:05:53.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:05:53.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:05:53.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:05:53.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:05:53.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:05:53.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:05:53.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:05:53.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:05:53.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:05:53.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:05:53.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:05:53.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:05:53.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:05:53.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:05:53.163 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T22:05:53.164 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T22:05:53.165 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T22:05:53.166 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T22:05:53.167 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T22:05:53.168 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T22:05:53.169 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T22:05:53.170 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:05:53.171 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:05:53.172 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:05:53.172 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:05:53.173 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:05:53.173 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /sys 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /proc 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /dev 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /run 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /mnt 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /var/tmp 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /home 2023-07-21T22:05:53.174 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /tmp 2023-07-21T22:05:53.175 INFO:teuthology.orchestra.run.smithi066.stdout:skipping the directory /dev 2023-07-21T22:05:53.175 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:05:53.589 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:05:55.461 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:05:55.605 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:05:55.606 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:54 ago on Fri 21 Jul 2023 10:04:01 PM UTC. 2023-07-21T22:05:55.671 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:05:55.958 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:05:56.130 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:05:56.704 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:05:56.938 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:05:56.983 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:05:58.155 INFO:teuthology.orchestra.run.smithi104.stdout:Installed products updated. 2023-07-21T22:05:58.366 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:05:58.366 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:05:58.367 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.368 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.369 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.370 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:05:58.371 INFO:teuthology.orchestra.run.smithi104.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:05:58.372 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.373 INFO:teuthology.orchestra.run.smithi104.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:05:58.374 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:05:58.375 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:05:58.376 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:05:58.376 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:05:58.376 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:05:58.376 INFO:teuthology.orchestra.run.smithi104.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:05:58.376 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:05:58.377 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:05:58.378 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:05:58.379 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:05:58.380 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.381 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:05:58.382 INFO:teuthology.orchestra.run.smithi104.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.383 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.384 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:05:58.384 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:05:58.384 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:05:58.384 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:05:58.384 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:05:58.384 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:05:58.385 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:05:58.385 INFO:teuthology.orchestra.run.smithi104.stdout: 2023-07-21T22:05:58.385 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2023-07-21T22:05:58.570 DEBUG:teuthology.parallel:result is None 2023-07-21T22:06:00.050 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:06:00.223 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:06:00.379 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:06:00.585 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:06:00.820 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:06:01.096 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:06:01.303 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:06:02.496 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:06:02.662 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:06:02.798 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:06:03.222 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:06:03.583 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:06:03.653 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:06:03.837 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:06:03.867 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:06:05.422 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:06:05.783 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:06:05.863 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:06:06.028 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:06:06.103 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:06:06.295 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:06:06.732 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:06:06.921 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:06:12.294 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:06:13.620 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:06:13.979 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T22:06:14.184 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 117/134 2023-07-21T22:06:14.475 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T22:06:14.636 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:06:14.904 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:06:15.125 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T22:06:15.341 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:06:15.495 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:06:19.037 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:19.328 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:19.368 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:06:19.714 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:06:19.980 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:06:20.333 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:06:20.693 INFO:teuthology.orchestra.run.smithi066.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:06:20.860 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:06:20.972 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:06:21.210 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:06:21.240 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:06:21.443 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:06:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:06:21.664 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T22:06:21.664 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:21.900 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:21.901 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:06:22.167 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:06:22.379 INFO:teuthology.orchestra.run.smithi066.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:22.647 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:22.648 INFO:teuthology.orchestra.run.smithi066.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-21T22:06:22.648 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:06:23.350 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:06:23.351 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:06:23.352 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:06:23.353 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:06:23.354 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:06:23.354 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:06:23.354 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:06:23.360 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:06:23.361 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:06:23.361 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:06:23.361 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:06:23.361 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:06:23.361 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:06:23.361 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:06:23.362 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:06:23.363 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:06:23.364 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:06:23.365 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T22:06:23.366 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T22:06:23.367 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T22:06:23.368 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T22:06:23.369 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T22:06:23.370 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T22:06:23.371 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T22:06:23.372 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T22:06:23.373 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T22:06:23.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:06:23.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:06:23.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:06:23.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:06:23.374 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:06:26.191 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:06:26.191 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:02:20 ago on Fri 21 Jul 2023 10:04:06 PM UTC. 2023-07-21T22:06:28.950 INFO:teuthology.orchestra.run.smithi066.stdout:Installed products updated. 2023-07-21T22:06:29.191 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:06:29.192 INFO:teuthology.orchestra.run.smithi066.stdout:Upgraded: 2023-07-21T22:06:29.192 INFO:teuthology.orchestra.run.smithi066.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:06:29.193 INFO:teuthology.orchestra.run.smithi066.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.193 INFO:teuthology.orchestra.run.smithi066.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.193 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2023-07-21T22:06:29.193 INFO:teuthology.orchestra.run.smithi066.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.194 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.195 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.196 INFO:teuthology.orchestra.run.smithi066.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:06:29.197 INFO:teuthology.orchestra.run.smithi066.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:06:29.198 INFO:teuthology.orchestra.run.smithi066.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.199 INFO:teuthology.orchestra.run.smithi066.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:06:29.200 INFO:teuthology.orchestra.run.smithi066.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:06:29.201 INFO:teuthology.orchestra.run.smithi066.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:06:29.202 INFO:teuthology.orchestra.run.smithi066.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:06:29.203 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:06:29.204 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:06:29.205 INFO:teuthology.orchestra.run.smithi066.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:06:29.206 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:06:29.207 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:06:29.208 INFO:teuthology.orchestra.run.smithi066.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:06:29.209 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:06:29.210 INFO:teuthology.orchestra.run.smithi066.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:06:29.211 INFO:teuthology.orchestra.run.smithi066.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:06:29.211 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:06:29.211 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2023-07-21T22:06:29.556 DEBUG:teuthology.parallel:result is None 2023-07-21T22:06:46.287 INFO:teuthology.orchestra.run.smithi149.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:06:46.899 INFO:teuthology.orchestra.run.smithi149.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.4 kB/s | 306 kB 02:09 2023-07-21T22:06:50.604 INFO:teuthology.orchestra.run.smithi149.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:06:50.605 INFO:teuthology.orchestra.run.smithi149.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:06:51.340 INFO:teuthology.orchestra.run.smithi149.stdout:(40/131): liboath-2.6.2-3.el8.x86_64.rpm 454 B/s | 59 kB 02:12 2023-07-21T22:06:51.365 INFO:teuthology.orchestra.run.smithi149.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 593 B/s | 75 kB 02:09 2023-07-21T22:06:51.532 INFO:teuthology.orchestra.run.smithi149.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 227 kB/s | 44 kB 00:00 2023-07-21T22:06:51.699 INFO:teuthology.orchestra.run.smithi149.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 198 kB/s | 33 kB 00:00 2023-07-21T22:06:51.724 INFO:teuthology.orchestra.run.smithi149.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 516 kB/s | 185 kB 00:00 2023-07-21T22:06:51.841 INFO:teuthology.orchestra.run.smithi149.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 115 kB/s | 16 kB 00:00 2023-07-21T22:06:51.967 INFO:teuthology.orchestra.run.smithi149.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 713 kB/s | 173 kB 00:00 2023-07-21T22:06:52.142 INFO:teuthology.orchestra.run.smithi149.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 464 kB/s | 81 kB 00:00 2023-07-21T22:06:52.292 INFO:teuthology.orchestra.run.smithi149.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 74 kB/s | 11 kB 00:00 2023-07-21T22:06:52.384 INFO:teuthology.orchestra.run.smithi149.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 709 kB/s | 384 kB 00:00 2023-07-21T22:06:52.443 INFO:teuthology.orchestra.run.smithi149.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 117 kB/s | 18 kB 00:00 2023-07-21T22:06:52.609 INFO:teuthology.orchestra.run.smithi149.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 292 kB/s | 49 kB 00:00 2023-07-21T22:06:52.726 INFO:teuthology.orchestra.run.smithi149.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2023-07-21T22:06:52.760 INFO:teuthology.orchestra.run.smithi149.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.1 MB/s | 1.2 MB 00:00 2023-07-21T22:06:52.863 INFO:teuthology.orchestra.run.smithi149.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 451 kB/s | 60 kB 00:00 2023-07-21T22:06:52.902 INFO:teuthology.orchestra.run.smithi149.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-07-21T22:06:52.953 INFO:teuthology.orchestra.run.smithi149.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 169 kB/s | 16 kB 00:00 2023-07-21T22:06:53.003 INFO:teuthology.orchestra.run.smithi149.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 335 kB/s | 34 kB 00:00 2023-07-21T22:06:53.112 INFO:teuthology.orchestra.run.smithi149.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 567 kB/s | 61 kB 00:00 2023-07-21T22:06:53.137 INFO:teuthology.orchestra.run.smithi149.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-07-21T22:06:53.220 INFO:teuthology.orchestra.run.smithi149.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 173 kB/s | 19 kB 00:00 2023-07-21T22:06:53.239 INFO:teuthology.orchestra.run.smithi149.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-07-21T22:06:53.354 INFO:teuthology.orchestra.run.smithi149.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 219 kB/s | 29 kB 00:00 2023-07-21T22:06:53.371 INFO:teuthology.orchestra.run.smithi149.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 206 kB/s | 27 kB 00:00 2023-07-21T22:06:53.491 INFO:teuthology.orchestra.run.smithi149.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.9 MB/s | 255 kB 00:00 2023-07-21T22:06:53.555 INFO:teuthology.orchestra.run.smithi149.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-07-21T22:06:53.597 INFO:teuthology.orchestra.run.smithi149.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-07-21T22:06:53.664 INFO:teuthology.orchestra.run.smithi149.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 786 kB/s | 85 kB 00:00 2023-07-21T22:06:53.714 INFO:teuthology.orchestra.run.smithi149.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-07-21T22:06:53.731 INFO:teuthology.orchestra.run.smithi149.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.0 MB/s | 70 kB 00:00 2023-07-21T22:06:53.782 INFO:teuthology.orchestra.run.smithi149.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 937 kB/s | 47 kB 00:00 2023-07-21T22:06:53.824 INFO:teuthology.orchestra.run.smithi149.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-21T22:06:53.882 INFO:teuthology.orchestra.run.smithi149.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 731 kB/s | 43 kB 00:00 2023-07-21T22:06:53.924 INFO:teuthology.orchestra.run.smithi149.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-21T22:06:53.966 INFO:teuthology.orchestra.run.smithi149.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-07-21T22:06:54.000 INFO:teuthology.orchestra.run.smithi149.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.5 MB/s | 87 kB 00:00 2023-07-21T22:06:54.042 INFO:teuthology.orchestra.run.smithi149.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-21T22:06:54.075 INFO:teuthology.orchestra.run.smithi149.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 3.2 MB/s | 109 kB 00:00 2023-07-21T22:06:54.138 INFO:teuthology.orchestra.run.smithi149.stdout:(78/131): thrift-0.13.0-2.el8.x86_64.rpm 3.7 MB/s | 1.7 MB 00:00 2023-07-21T22:06:54.159 INFO:teuthology.orchestra.run.smithi149.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-07-21T22:06:54.201 INFO:teuthology.orchestra.run.smithi149.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 515 kB/s | 19 kB 00:00 2023-07-21T22:06:54.235 INFO:teuthology.orchestra.run.smithi149.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.7 MB/s | 560 kB 00:00 2023-07-21T22:06:54.260 INFO:teuthology.orchestra.run.smithi149.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-07-21T22:06:54.294 INFO:teuthology.orchestra.run.smithi149.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-07-21T22:06:54.319 INFO:teuthology.orchestra.run.smithi149.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-07-21T22:06:54.352 INFO:teuthology.orchestra.run.smithi149.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-21T22:06:54.378 INFO:teuthology.orchestra.run.smithi149.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-21T22:06:54.420 INFO:teuthology.orchestra.run.smithi149.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-21T22:06:54.478 INFO:teuthology.orchestra.run.smithi149.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 19 MB/s | 2.3 MB 00:00 2023-07-21T22:06:54.504 INFO:teuthology.orchestra.run.smithi149.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-07-21T22:06:54.529 INFO:teuthology.orchestra.run.smithi149.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-07-21T22:06:54.554 INFO:teuthology.orchestra.run.smithi149.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-21T22:06:54.579 INFO:teuthology.orchestra.run.smithi149.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T22:06:54.605 INFO:teuthology.orchestra.run.smithi149.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2023-07-21T22:06:54.638 INFO:teuthology.orchestra.run.smithi149.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-07-21T22:06:54.663 INFO:teuthology.orchestra.run.smithi149.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-07-21T22:06:54.689 INFO:teuthology.orchestra.run.smithi149.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-21T22:06:54.747 INFO:teuthology.orchestra.run.smithi149.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-07-21T22:06:54.789 INFO:teuthology.orchestra.run.smithi149.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-21T22:06:54.840 INFO:teuthology.orchestra.run.smithi149.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-07-21T22:06:54.907 INFO:teuthology.orchestra.run.smithi149.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-07-21T22:06:55.049 INFO:teuthology.orchestra.run.smithi149.stdout:(101/131): python3-scipy-1.0.0-21.module+el8.5. 36 MB/s | 14 MB 00:00 2023-07-21T22:06:55.065 INFO:teuthology.orchestra.run.smithi149.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 245 kB/s | 39 kB 00:00 2023-07-21T22:06:55.116 INFO:teuthology.orchestra.run.smithi149.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-07-21T22:06:55.149 INFO:teuthology.orchestra.run.smithi149.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-07-21T22:06:55.175 INFO:teuthology.orchestra.run.smithi149.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-21T22:06:55.200 INFO:teuthology.orchestra.run.smithi149.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-07-21T22:06:55.242 INFO:teuthology.orchestra.run.smithi149.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-21T22:06:55.292 INFO:teuthology.orchestra.run.smithi149.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 531 kB/s | 27 kB 00:00 2023-07-21T22:06:55.367 INFO:teuthology.orchestra.run.smithi149.stdout:(109/131): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-07-21T22:06:55.393 INFO:teuthology.orchestra.run.smithi149.stdout:(110/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-07-21T22:06:55.418 INFO:teuthology.orchestra.run.smithi149.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 877 kB/s | 44 kB 00:00 2023-07-21T22:06:55.477 INFO:teuthology.orchestra.run.smithi149.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-07-21T22:06:55.543 INFO:teuthology.orchestra.run.smithi149.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 25 MB/s | 3.7 MB 00:00 2023-07-21T22:06:55.569 INFO:teuthology.orchestra.run.smithi149.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.4 MB/s | 225 kB 00:00 2023-07-21T22:06:55.594 INFO:teuthology.orchestra.run.smithi149.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-07-21T22:06:55.619 INFO:teuthology.orchestra.run.smithi149.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-07-21T22:06:55.644 INFO:teuthology.orchestra.run.smithi149.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 400 kB/s | 20 kB 00:00 2023-07-21T22:06:55.670 INFO:teuthology.orchestra.run.smithi149.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-21T22:06:55.703 INFO:teuthology.orchestra.run.smithi149.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-21T22:06:55.753 INFO:teuthology.orchestra.run.smithi149.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-07-21T22:06:55.829 INFO:teuthology.orchestra.run.smithi149.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 31 MB/s | 4.9 MB 00:00 2023-07-21T22:06:55.879 INFO:teuthology.orchestra.run.smithi149.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-07-21T22:06:55.963 INFO:teuthology.orchestra.run.smithi149.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 23 MB/s | 4.7 MB 00:00 2023-07-21T22:06:55.988 INFO:teuthology.orchestra.run.smithi149.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-07-21T22:06:56.105 INFO:teuthology.orchestra.run.smithi149.stdout:(125/131): postgresql-server-10.21-2.module+el8 36 MB/s | 5.1 MB 00:00 2023-07-21T22:06:56.138 INFO:teuthology.orchestra.run.smithi149.stdout:(126/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-07-21T22:06:56.205 INFO:teuthology.orchestra.run.smithi149.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-07-21T22:07:00.213 INFO:teuthology.orchestra.run.smithi149.stdout:(128/131): librbd1-18.0.0-4735.gbffe049d.el8.x8 909 kB/s | 3.6 MB 00:04 2023-07-21T22:07:00.289 INFO:teuthology.orchestra.run.smithi149.stdout:(129/131): librados2-18.0.0-4735.gbffe049d.el8. 903 kB/s | 3.7 MB 00:04 2023-07-21T22:07:00.330 INFO:teuthology.orchestra.run.smithi149.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 6.7 MB/s | 798 kB 00:00 2023-07-21T22:08:57.361 INFO:teuthology.orchestra.run.smithi149.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T22:08:58.082 INFO:teuthology.orchestra.run.smithi149.stdout:(131/131): python3-asyncssh-2.7.0-2.el8.noarch. 3.6 kB/s | 476 kB 02:11 2023-07-21T22:08:58.083 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:08:58.083 INFO:teuthology.orchestra.run.smithi149.stdout:Total 737 kB/s | 205 MB 04:45 2023-07-21T22:08:58.860 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2023-07-21T22:08:58.980 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2023-07-21T22:08:58.980 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2023-07-21T22:09:00.094 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2023-07-21T22:09:00.098 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2023-07-21T22:09:04.874 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2023-07-21T22:09:05.043 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T22:09:05.226 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T22:09:05.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:09:05.536 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T22:09:05.729 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:09:06.110 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T22:09:06.312 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:09:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 5/134 2023-07-21T22:09:06.870 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:09:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:09:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:09:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:09:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:09:07.613 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T22:09:07.824 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T22:09:08.031 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T22:09:08.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T22:09:08.665 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T22:09:08.803 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:09:08.909 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 14/134 2023-07-21T22:09:09.044 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:09:09.186 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 15/134 2023-07-21T22:09:09.398 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T22:09:09.592 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T22:09:09.773 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T22:09:09.894 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:09:10.037 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T22:09:10.178 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:09:10.790 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T22:09:10.961 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:09:11.550 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T22:09:11.769 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:09:12.455 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T22:09:12.667 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T22:09:12.841 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T22:09:13.079 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T22:09:13.234 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 26/134 2023-07-21T22:09:13.363 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 27/134 2023-07-21T22:09:13.557 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 28/134 2023-07-21T22:09:13.786 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:09:15.549 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T22:09:16.589 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T22:09:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T22:09:17.171 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T22:09:17.389 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T22:09:17.596 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T22:09:17.778 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T22:09:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 36/134 2023-07-21T22:09:18.133 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 37/134 2023-07-21T22:09:18.275 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T22:09:18.411 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T22:09:18.577 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:09:18.651 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T22:09:18.856 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T22:09:19.071 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T22:09:19.218 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:09:19.296 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T22:09:19.488 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T22:09:19.647 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T22:09:19.808 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T22:09:19.995 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T22:09:20.145 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T22:09:20.305 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T22:09:20.482 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T22:09:20.672 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T22:09:21.012 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T22:09:21.320 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T22:09:21.538 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T22:09:21.739 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T22:09:21.981 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T22:09:22.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T22:09:22.492 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T22:09:22.677 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T22:09:22.889 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T22:09:23.063 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T22:09:23.229 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T22:09:23.514 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T22:09:23.756 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T22:09:24.014 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T22:09:24.235 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T22:09:24.383 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T22:09:24.553 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T22:09:24.810 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T22:09:25.057 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T22:09:25.205 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T22:09:25.353 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T22:09:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T22:09:25.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T22:09:25.939 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T22:09:26.137 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T22:09:26.265 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T22:09:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T22:09:26.697 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T22:09:26.906 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T22:09:27.157 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T22:09:27.374 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T22:09:27.541 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T22:09:27.751 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T22:09:28.856 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T22:09:29.120 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T22:09:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T22:09:29.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T22:09:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T22:09:30.132 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T22:09:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T22:09:31.050 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:09:31.121 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 92/134 2023-07-21T22:09:31.350 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 93/134 2023-07-21T22:09:34.991 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:09:35.175 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:09:35.774 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 94/134 2023-07-21T22:09:36.060 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:09:36.131 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 95/134 2023-07-21T22:09:36.274 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:09:55.133 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 96/134 2023-07-21T22:09:55.133 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2023-07-21T22:09:55.133 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /dev 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /run 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /dev 2023-07-21T22:09:55.134 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:09:55.472 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:09:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 97/134 2023-07-21T22:09:57.223 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:09:57.443 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 98/134 2023-07-21T22:09:57.604 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:09:58.070 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 99/134 2023-07-21T22:09:58.230 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:09:58.246 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 100/134 2023-07-21T22:09:59.903 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:10:00.067 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 101/134 2023-07-21T22:10:00.193 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 102/134 2023-07-21T22:10:00.377 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 103/134 2023-07-21T22:10:00.545 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:10:00.772 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 104/134 2023-07-21T22:10:01.081 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:10:02.044 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 105/134 2023-07-21T22:10:02.195 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:10:02.286 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 106/134 2023-07-21T22:10:02.651 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 107/134 2023-07-21T22:10:02.865 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:10:02.901 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 108/134 2023-07-21T22:10:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:10:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 109/134 2023-07-21T22:10:04.380 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 110/134 2023-07-21T22:10:04.550 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:10:04.583 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 111/134 2023-07-21T22:10:04.728 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:10:04.771 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 112/134 2023-07-21T22:10:04.904 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:10:05.250 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 113/134 2023-07-21T22:10:05.395 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:10:09.918 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 114/134 2023-07-21T22:10:10.791 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 115/134 2023-07-21T22:10:11.106 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T22:10:11.330 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 117/134 2023-07-21T22:10:11.586 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T22:10:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:10:11.936 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T22:10:12.173 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T22:10:12.331 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 121/134 2023-07-21T22:10:12.478 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T22:10:14.686 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:10:14.944 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:10:14.962 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T22:10:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 124/134 2023-07-21T22:10:15.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 125/134 2023-07-21T22:10:15.731 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 126/134 2023-07-21T22:10:16.047 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T22:10:16.218 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:10:16.302 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T22:10:16.493 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:10:16.505 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T22:10:16.643 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:10:16.651 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T22:10:16.778 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T22:10:16.779 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:10:16.971 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:10:16.972 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:10:17.207 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T22:10:17.425 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:10:17.954 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:10:17.954 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 1/134 2023-07-21T22:10:17.954 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2/134 2023-07-21T22:10:17.954 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 3/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 4/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4735.gbffe0 5/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 6/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 7/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 8/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 9/134 2023-07-21T22:10:17.955 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 10/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 11/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 12/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 13/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_ 14/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 15/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 16/134 2023-07-21T22:10:17.956 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_6 17/134 2023-07-21T22:10:17.957 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86 18/134 2023-07-21T22:10:17.957 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 19/134 2023-07-21T22:10:17.957 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el 20/134 2023-07-21T22:10:17.957 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:18.0.0-4735.gbffe049d.el8. 21/134 2023-07-21T22:10:17.957 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_6 22/134 2023-07-21T22:10:17.957 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 23/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 24/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 25/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 26/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 27/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 28/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d. 29/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noa 30/134 2023-07-21T22:10:17.958 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.n 31/134 2023-07-21T22:10:17.965 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4735.gbff 32/134 2023-07-21T22:10:17.965 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el 33/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 34/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.e 35/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 36/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 37/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 38/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T22:10:17.966 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T22:10:17.967 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T22:10:17.968 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T22:10:17.969 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T22:10:17.970 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T22:10:17.971 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T22:10:17.972 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T22:10:17.973 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T22:10:17.974 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T22:10:17.975 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T22:10:17.976 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T22:10:17.977 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 129/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 131/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T22:10:17.978 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T22:10:20.454 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T22:10:20.455 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:06:19 ago on Fri 21 Jul 2023 10:04:01 PM UTC. 2023-07-21T22:10:23.921 INFO:teuthology.orchestra.run.smithi149.stdout:Installed products updated. 2023-07-21T22:10:24.134 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:10:24.134 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2023-07-21T22:10:24.135 INFO:teuthology.orchestra.run.smithi149.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T22:10:24.135 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.135 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.135 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2023-07-21T22:10:24.135 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T22:10:24.136 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T22:10:24.136 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T22:10:24.136 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.136 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.136 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.136 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.137 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.138 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.139 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.139 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.139 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.139 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:18.0.0-4735.gbffe049d.el8.noarch 2023-07-21T22:10:24.139 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T22:10:24.140 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T22:10:24.140 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T22:10:24.140 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T22:10:24.140 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-3.el8.x86_64 2023-07-21T22:10:24.140 INFO:teuthology.orchestra.run.smithi149.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T22:10:24.140 INFO:teuthology.orchestra.run.smithi149.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:10:24.141 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.142 INFO:teuthology.orchestra.run.smithi149.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T22:10:24.143 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T22:10:24.144 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T22:10:24.144 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T22:10:24.144 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T22:10:24.144 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T22:10:24.144 INFO:teuthology.orchestra.run.smithi149.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T22:10:24.145 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:10:24.145 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:10:24.145 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T22:10:24.145 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T22:10:24.145 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T22:10:24.145 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T22:10:24.146 INFO:teuthology.orchestra.run.smithi149.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T22:10:24.146 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T22:10:24.146 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.146 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.146 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.146 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T22:10:24.147 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T22:10:24.148 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T22:10:24.149 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T22:10:24.149 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T22:10:24.149 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T22:10:24.149 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T22:10:24.149 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T22:10:24.149 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T22:10:24.150 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T22:10:24.150 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T22:10:24.150 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T22:10:24.150 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T22:10:24.150 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T22:10:24.151 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T22:10:24.151 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T22:10:24.151 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T22:10:24.151 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.151 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.151 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T22:10:24.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T22:10:24.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:18.0.0-4735.gbffe049d.el8.x86_64 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T22:10:24.154 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T22:10:24.155 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T22:10:24.155 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T22:10:24.155 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:10:24.155 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2023-07-21T22:10:24.330 DEBUG:teuthology.parallel:result is None 2023-07-21T22:10:24.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:10:29.756 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:10:29.819 INFO:teuthology.orchestra.run.smithi066.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:10:29.819 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:10:29.820 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:10:29.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:10:30.246 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:10:30.275 INFO:teuthology.orchestra.run.smithi104.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:10:30.275 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:10:30.275 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:10:30.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:10:35.778 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T22:10:35.806 INFO:teuthology.orchestra.run.smithi149.stdout:18.0.0-4735.gbffe049d.el8 2023-07-21T22:10:35.807 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4735.gbffe049d.el8 2023-07-21T22:10:35.807 INFO:teuthology.task.install:The correct ceph version 18.0.0-4735.gbffe049d is installed. 2023-07-21T22:10:35.808 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-21T22:10:35.808 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:10:35.809 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:10:35.909 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:10:35.910 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:10:35.958 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:10:35.958 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T22:10:36.006 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-21T22:10:36.007 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:10:36.007 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:10:36.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:10:36.191 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:10:36.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:10:36.224 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:10:36.294 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:10:36.294 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T22:10:36.356 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T22:10:36.428 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-21T22:10:36.429 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:10:36.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:10:36.500 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:10:36.610 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:10:36.610 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:10:36.641 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:10:36.712 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:10:36.712 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T22:10:36.745 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T22:10:36.814 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-21T22:10:36.815 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:10:36.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:10:36.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:10:36.995 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:10:36.996 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:10:37.027 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:10:37.097 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:10:37.097 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T22:10:37.129 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T22:10:37.200 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-21T22:10:37.292 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-07-21T22:10:37.293 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:10:37.303 INFO:tasks.cephadm:Cluster fsid is 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:10:37.304 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-21T22:10:37.304 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.66', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.149'} 2023-07-21T22:10:37.305 INFO:tasks.cephadm:First mon is mon.a on smithi066 2023-07-21T22:10:37.305 INFO:tasks.cephadm:First mgr is x 2023-07-21T22:10:37.305 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-21T22:10:37.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2023-07-21T22:10:37.379 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2023-07-21T22:10:37.412 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2023-07-21T22:10:37.445 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-21T22:10:37.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:10:38.122 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71611/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4735-gbffe049d', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4735.gbffe049d'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-21 20:32:06.130547', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-base-2', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-07-21T22:10:43.786 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-rishabh-2023Jul13-base-2, sha1 bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:10:43.786 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm 2023-07-21T22:10:43.787 DEBUG:teuthology.orchestra.run.smithi066:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:46.092 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:10 /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:46.093 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:50.080 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:10 /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:50.081 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-base-2/bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:53.821 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 22:10 /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:53.822 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:53.875 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:53.897 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T22:10:53.938 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea on all hosts... 2023-07-21T22:10:53.938 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:10:53.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:10:53.945 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea pull 2023-07-21T22:10:55.160 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:10:55.267 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:10:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:11:20.144 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-21T22:11:20.145 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:11:20.145 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:11:20.145 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2023-07-21T22:11:20.145 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:11:20.146 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2023-07-21T22:11:20.146 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-21T22:11:24.966 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-07-21T22:11:24.966 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:11:24.967 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:11:24.967 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2023-07-21T22:11:24.967 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:11:24.967 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2023-07-21T22:11:24.967 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-07-21T22:11:26.418 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:11:26.418 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:11:26.419 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c", 2023-07-21T22:11:26.419 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2023-07-21T22:11:26.419 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:e12d2e98545c41acbacae1458e8c994c4b4d7018583b915c7d66098e77144f43" 2023-07-21T22:11:26.419 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:11:26.419 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:11:26.444 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2023-07-21T22:11:26.519 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2023-07-21T22:11:26.557 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2023-07-21T22:11:26.594 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2023-07-21T22:11:26.664 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2023-07-21T22:11:26.696 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2023-07-21T22:11:26.728 INFO:tasks.cephadm:Writing seed config... 2023-07-21T22:11:26.729 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-21T22:11:26.729 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-21T22:11:26.730 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-21T22:11:26.730 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-21T22:11:26.730 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-21T22:11:26.731 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-07-21T22:11:26.731 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-21T22:11:26.731 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-21T22:11:26.731 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-21T22:11:26.731 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-21T22:11:26.732 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-21T22:11:26.732 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-21T22:11:26.732 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-21T22:11:26.732 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-21T22:11:26.732 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-21T22:11:26.733 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-21T22:11:26.733 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-21T22:11:26.733 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-21T22:11:26.733 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-21T22:11:26.733 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-21T22:11:26.734 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-21T22:11:26.734 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-21T22:11:26.734 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-21T22:11:26.734 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-21T22:11:26.734 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-21T22:11:26.734 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-21T22:11:26.735 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-21T22:11:26.735 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-21T22:11:26.735 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-21T22:11:26.735 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-21T22:11:26.736 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-21T22:11:26.736 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-21T22:11:26.737 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-21T22:11:26.737 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-21T22:11:26.738 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-21T22:11:26.738 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-21T22:11:26.739 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-21T22:11:26.739 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-21T22:11:26.739 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-21T22:11:26.740 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-21T22:11:26.740 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-21T22:11:26.741 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-21T22:11:26.741 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-21T22:11:26.742 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-07-21T22:11:26.744 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:11:26.744 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-21T22:11:26.788 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 6bcb348c-2813-11ee-9b34-001a4aab830c ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-21T22:11:26.788 DEBUG:teuthology.orchestra.run.smithi066:mon.a> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a.service 2023-07-21T22:11:26.832 DEBUG:teuthology.orchestra.run.smithi066:mgr.x> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-21T22:11:26.878 INFO:tasks.cephadm:Bootstrapping... 2023-07-21T22:11:26.879 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea -v bootstrap --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.66 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:11:26.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:11:26.948 INFO:journalctl@ceph.mgr.x.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:11:27.110 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2023-07-21T22:11:27.111 INFO:teuthology.orchestra.run.smithi066.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea', '-v', 'bootstrap', '--fsid', '6bcb348c-2813-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.66', '--skip-admin-label'] 2023-07-21T22:11:27.286 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:11:27.287 INFO:teuthology.orchestra.run.smithi066.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-21T22:11:27.288 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying podman|docker is present... 2023-07-21T22:11:27.460 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:11:27.461 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying lvm2 is present... 2023-07-21T22:11:27.461 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying time synchronization is in place... 2023-07-21T22:11:27.475 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T22:11:27.475 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T22:11:27.488 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T22:11:27.488 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-07-21T22:11:27.502 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2023-07-21T22:11:27.516 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2023-07-21T22:11:27.516 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2023-07-21T22:11:27.516 INFO:teuthology.orchestra.run.smithi066.stdout:Repeating the final host check... 2023-07-21T22:11:27.688 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4.1.1 2023-07-21T22:11:27.688 INFO:teuthology.orchestra.run.smithi066.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-21T22:11:27.689 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl is present 2023-07-21T22:11:27.689 INFO:teuthology.orchestra.run.smithi066.stdout:lvcreate is present 2023-07-21T22:11:27.702 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T22:11:27.703 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T22:11:27.715 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T22:11:27.715 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-07-21T22:11:27.729 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2023-07-21T22:11:27.742 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2023-07-21T22:11:27.742 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2023-07-21T22:11:27.742 INFO:teuthology.orchestra.run.smithi066.stdout:Host looks OK 2023-07-21T22:11:27.743 INFO:teuthology.orchestra.run.smithi066.stdout:Cluster fsid: 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:27.743 INFO:teuthology.orchestra.run.smithi066.stdout:Acquiring lock 139872635857384 on /run/cephadm/6bcb348c-2813-11ee-9b34-001a4aab830c.lock 2023-07-21T22:11:27.743 INFO:teuthology.orchestra.run.smithi066.stdout:Lock 139872635857384 acquired on /run/cephadm/6bcb348c-2813-11ee-9b34-001a4aab830c.lock 2023-07-21T22:11:27.744 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 3300 ... 2023-07-21T22:11:27.744 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 6789 ... 2023-07-21T22:11:27.744 INFO:teuthology.orchestra.run.smithi066.stdout:Base mon IP(s) is [172.21.15.66:3300, 172.21.15.66:6789], mon addrv is [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2023-07-21T22:11:27.752 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-07-21T22:11:27.753 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.66 metric 100 2023-07-21T22:11:27.759 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-21T22:11:27.759 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-07-21T22:11:27.759 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-07-21T22:11:27.766 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-21T22:11:27.766 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-21T22:11:27.766 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T22:11:27.766 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-07-21T22:11:27.766 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 fe80::1007:d553:b784:5869/64 scope link noprefixroute 2023-07-21T22:11:27.767 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T22:11:27.767 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2023-07-21T22:11:27.768 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2023-07-21T22:11:27.768 INFO:teuthology.orchestra.run.smithi066.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-21T22:11:27.768 INFO:teuthology.orchestra.run.smithi066.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-21T22:11:27.769 INFO:teuthology.orchestra.run.smithi066.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:11:28.246 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c 2023-07-21T22:11:28.246 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea... 2023-07-21T22:11:28.246 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Getting image source signatures 2023-07-21T22:11:28.247 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:99ee8d5f86f7949582f207d2e584c4024ff3caa1cfdcb812f3da7de6a1ca5e74 2023-07-21T22:11:28.247 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-21T22:11:28.247 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying config sha256:fa7da5dee9716ef7a4d987f3d91e84057b58a2169f3bd7522f9d782cbf91e96c 2023-07-21T22:11:28.247 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-21T22:11:28.247 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Storing signatures 2023-07-21T22:11:29.228 INFO:teuthology.orchestra.run.smithi066.stdout:ceph: stdout ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev) 2023-07-21T22:11:29.228 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph version: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev) 2023-07-21T22:11:29.228 INFO:teuthology.orchestra.run.smithi066.stdout:Extracting ceph user uid/gid from container image... 2023-07-21T22:11:29.906 INFO:teuthology.orchestra.run.smithi066.stdout:stat: stdout 167 167 2023-07-21T22:11:29.906 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial keys... 2023-07-21T22:11:30.681 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQCSArtkoJTrEBAAaAKaXU7LdxNsfOk9bfezSw== 2023-07-21T22:11:31.463 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQCTArtkXZoMAxAAJY+6p9feeKUClkg88y+wmg== 2023-07-21T22:11:32.137 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQCTArtkDzy0LhAAnzAFyU525zpBU+9ZW8GSHQ== 2023-07-21T22:11:32.138 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial monmap... 2023-07-21T22:11:32.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T22:11:32.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-21T22:11:32.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:32.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T22:11:32.916 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool for a [v2:172.21.15.66:3300,v1:172.21.15.66:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T22:11:32.916 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = pacific 2023-07-21T22:11:32.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: set fsid to 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:32.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T22:11:32.917 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:11:32.917 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mon... 2023-07-21T22:11:33.694 INFO:teuthology.orchestra.run.smithi066.stdout:create mon.a on 2023-07-21T22:11:33.945 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-21T22:11:34.164 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-21T22:11:35.120 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c.target. 2023-07-21T22:11:35.120 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c.target. 2023-07-21T22:11:35.482 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a 2023-07-21T22:11:35.483 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-21T22:11:35.694 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c.target.wants/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@.service. 2023-07-21T22:11:35.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:35 smithi066 systemd[1]: Starting Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:11:36.477 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:36 smithi066 podman[125839]: 2023-07-21T22:11:36.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:36 smithi066 bash[125839]: 56af9c9954cf780d079456b8965e424830a6a68d0df6f93788be831683c78197 2023-07-21T22:11:36.523 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:11:36.524 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-07-21T22:11:36.540 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:11:36.540 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-07-21T22:11:36.540 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-07-21T22:11:36.541 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T22:11:36.541 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon to start... 2023-07-21T22:11:36.541 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon... 2023-07-21T22:11:36.812 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:36 smithi066 ceph-mon[125879]: mkfs 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:36 smithi066 ceph-mon[125879]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:11:36.813 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:36 smithi066 systemd[1]: Started Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:11:37.629 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout cluster: 2023-07-21T22:11:37.630 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout id: 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:37.630 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-21T22:11:37.630 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:37.630 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout services: 2023-07-21T22:11:37.630 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.689634s) 2023-07-21T22:11:37.630 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout data: 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-21T22:11:37.631 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pgs: 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.034+0000 7f92dfa19700 1 Processor -- start 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.035+0000 7f92dfa19700 1 -- start start 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.035+0000 7f92dfa19700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.035+0000 7f92dfa19700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d8108460 con 0x7f92d8079cc0 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.036+0000 7f92dd7b5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:37.632 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.036+0000 7f92dd7b5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45768/0 (socket says 172.21.15.66:45768) 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.036+0000 7f92dd7b5700 1 -- 172.21.15.66:0/515766293 learned_addr learned my addr 172.21.15.66:0/515766293 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.036+0000 7f92dd7b5700 1 -- 172.21.15.66:0/515766293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d81085a0 con 0x7f92d8079cc0 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.037+0000 7f92dd7b5700 1 --2- 172.21.15.66:0/515766293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8107f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f92d40098a0 tx=0x7f92d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5393eaa8db7ab2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.037+0000 7f92cffff700 1 -- 172.21.15.66:0/515766293 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d4004160 con 0x7f92d8079cc0 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.037+0000 7f92cffff700 1 -- 172.21.15.66:0/515766293 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d40042c0 con 0x7f92d8079cc0 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.037+0000 7f92cffff700 1 -- 172.21.15.66:0/515766293 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d40045a0 con 0x7f92d8079cc0 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.037+0000 7f92dfa19700 1 -- 172.21.15.66:0/515766293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 msgr2=0x7f92d8107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:37.633 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.037+0000 7f92dfa19700 1 --2- 172.21.15.66:0/515766293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8107f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f92d40098a0 tx=0x7f92d4009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.038+0000 7f92dfa19700 1 -- 172.21.15.66:0/515766293 shutdown_connections 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.038+0000 7f92dfa19700 1 --2- 172.21.15.66:0/515766293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8107f20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.038+0000 7f92dfa19700 1 -- 172.21.15.66:0/515766293 >> 172.21.15.66:0/515766293 conn(0x7f92d80ff890 msgr2=0x7f92d8101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.038+0000 7f92dfa19700 1 -- 172.21.15.66:0/515766293 shutdown_connections 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.038+0000 7f92dfa19700 1 -- 172.21.15.66:0/515766293 wait complete. 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.039+0000 7f92dfa19700 1 Processor -- start 2023-07-21T22:11:37.634 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.039+0000 7f92dfa19700 1 -- start start 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.039+0000 7f92dfa19700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8105870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.039+0000 7f92dfa19700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d8105db0 con 0x7f92d8079cc0 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.040+0000 7f92dd7b5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.040+0000 7f92dd7b5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8105870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45772/0 (socket says 172.21.15.66:45772) 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.040+0000 7f92dd7b5700 1 -- 172.21.15.66:0/3099999357 learned_addr learned my addr 172.21.15.66:0/3099999357 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.040+0000 7f92dd7b5700 1 -- 172.21.15.66:0/3099999357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4009550 con 0x7f92d8079cc0 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.040+0000 7f92dd7b5700 1 --2- 172.21.15.66:0/3099999357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8105870 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f92d4004000 tx=0x7f92d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:37.635 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d4011630 con 0x7f92d8079cc0 2023-07-21T22:11:37.636 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4011790 con 0x7f92d8079cc0 2023-07-21T22:11:37.636 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92d4011a30 con 0x7f92d8079cc0 2023-07-21T22:11:37.636 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d8103ec0 con 0x7f92d8079cc0 2023-07-21T22:11:37.636 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d81042e0 con 0x7f92d8079cc0 2023-07-21T22:11:37.636 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f92d4022070 con 0x7f92d8079cc0 2023-07-21T22:11:37.636 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.041+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f92d401c450 con 0x7f92d8079cc0 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.042+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d8104cf0 con 0x7f92d8079cc0 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.045+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f92d401b450 con 0x7f92d8079cc0 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.120+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f92d8192290 con 0x7f92d8079cc0 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.121+0000 7f92ce7fc700 1 -- 172.21.15.66:0/3099999357 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f92d401b5f0 con 0x7f92d8079cc0 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.121+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 msgr2=0x7f92d8105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.122+0000 7f92dfa19700 1 --2- 172.21.15.66:0/3099999357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8105870 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f92d4004000 tx=0x7f92d4004750 comp rx=0 tx=0).stop 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.122+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 shutdown_connections 2023-07-21T22:11:37.637 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.122+0000 7f92dfa19700 1 --2- 172.21.15.66:0/3099999357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92d8079cc0 0x7f92d8105870 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:37.638 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.122+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 >> 172.21.15.66:0/3099999357 conn(0x7f92d80ff890 msgr2=0x7f92d8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:37.638 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.122+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 shutdown_connections 2023-07-21T22:11:37.638 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:37.122+0000 7f92dfa19700 1 -- 172.21.15.66:0/3099999357 wait complete. 2023-07-21T22:11:37.638 INFO:teuthology.orchestra.run.smithi066.stdout:mon is available 2023-07-21T22:11:37.638 INFO:teuthology.orchestra.run.smithi066.stdout:Assimilating anything we can from ceph.conf... 2023-07-21T22:11:37.849 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:37 smithi066 ceph-mon[125879]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:11:37.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:37 smithi066 ceph-mon[125879]: monmap e1: 1 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0]} removed_ranks: {} 2023-07-21T22:11:37.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:37 smithi066 ceph-mon[125879]: fsmap 2023-07-21T22:11:37.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:37 smithi066 ceph-mon[125879]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T22:11:37.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:37 smithi066 ceph-mon[125879]: mgrmap e1: no daemons active 2023-07-21T22:11:37.850 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:37 smithi066 ceph-mon[125879]: from='client.? 172.21.15.66:0/3099999357' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-21T22:11:38.610 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:38.610 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T22:11:38.610 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:38.611 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T22:11:38.612 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.074+0000 7fc83a20b700 1 Processor -- start 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.074+0000 7fc83a20b700 1 -- start start 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.075+0000 7fc83a20b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.075+0000 7fc83a20b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83407a880 con 0x7fc83407be40 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.075+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.075+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45776/0 (socket says 172.21.15.66:45776) 2023-07-21T22:11:38.613 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.075+0000 7fc8337fe700 1 -- 172.21.15.66:0/1390137566 learned_addr learned my addr 172.21.15.66:0/1390137566 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.076+0000 7fc8337fe700 1 -- 172.21.15.66:0/1390137566 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83407a9c0 con 0x7fc83407be40 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.076+0000 7fc8337fe700 1 --2- 172.21.15.66:0/1390137566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83407a340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc8240098a0 tx=0x7fc824009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73961ef53b6a6dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.076+0000 7fc8327fc700 1 -- 172.21.15.66:0/1390137566 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc824004160 con 0x7fc83407be40 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.076+0000 7fc8327fc700 1 -- 172.21.15.66:0/1390137566 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8240042c0 con 0x7fc83407be40 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.076+0000 7fc8327fc700 1 -- 172.21.15.66:0/1390137566 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8240045a0 con 0x7fc83407be40 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 -- 172.21.15.66:0/1390137566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 msgr2=0x7fc83407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 --2- 172.21.15.66:0/1390137566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83407a340 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc8240098a0 tx=0x7fc824009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:38.614 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 -- 172.21.15.66:0/1390137566 shutdown_connections 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 --2- 172.21.15.66:0/1390137566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83407a340 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 -- 172.21.15.66:0/1390137566 >> 172.21.15.66:0/1390137566 conn(0x7fc834100970 msgr2=0x7fc834102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 -- 172.21.15.66:0/1390137566 shutdown_connections 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.077+0000 7fc83a20b700 1 -- 172.21.15.66:0/1390137566 wait complete. 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.078+0000 7fc83a20b700 1 Processor -- start 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.078+0000 7fc83a20b700 1 -- start start 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.078+0000 7fc83a20b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:38.615 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.078+0000 7fc83a20b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419afd0 con 0x7fc83407be40 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.079+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.079+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45784/0 (socket says 172.21.15.66:45784) 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.079+0000 7fc8337fe700 1 -- 172.21.15.66:0/1925854850 learned_addr learned my addr 172.21.15.66:0/1925854850 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.079+0000 7fc8337fe700 1 -- 172.21.15.66:0/1925854850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc824009550 con 0x7fc83407be40 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.079+0000 7fc8337fe700 1 --2- 172.21.15.66:0/1925854850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83419aa90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc824004000 tx=0x7fc824004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc824011630 con 0x7fc83407be40 2023-07-21T22:11:38.616 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824011790 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83419b1d0 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc824011a30 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83419b5f0 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc824022070 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.080+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc82401c450 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.081+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc834194870 con 0x7fc83407be40 2023-07-21T22:11:38.617 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.084+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc82401b450 con 0x7fc83407be40 2023-07-21T22:11:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.156+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fc83402cad0 con 0x7fc83407be40 2023-07-21T22:11:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.160+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc82401b5f0 con 0x7fc83407be40 2023-07-21T22:11:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.160+0000 7fc830ff9700 1 -- 172.21.15.66:0/1925854850 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fc8240375f0 con 0x7fc83407be40 2023-07-21T22:11:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.161+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 msgr2=0x7fc83419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.161+0000 7fc83a20b700 1 --2- 172.21.15.66:0/1925854850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83419aa90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc824004000 tx=0x7fc824004750 comp rx=0 tx=0).stop 2023-07-21T22:11:38.618 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.161+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 shutdown_connections 2023-07-21T22:11:38.619 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.161+0000 7fc83a20b700 1 --2- 172.21.15.66:0/1925854850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc83407be40 0x7fc83419aa90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:38.619 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.161+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 >> 172.21.15.66:0/1925854850 conn(0x7fc834100970 msgr2=0x7fc8341022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:38.619 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.162+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 shutdown_connections 2023-07-21T22:11:38.619 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:38.162+0000 7fc83a20b700 1 -- 172.21.15.66:0/1925854850 wait complete. 2023-07-21T22:11:38.619 INFO:teuthology.orchestra.run.smithi066.stdout:Generating new minimal ceph.conf... 2023-07-21T22:11:38.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:38 smithi066 ceph-mon[125879]: from='client.? 172.21.15.66:0/1925854850' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T22:11:38.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:38 smithi066 ceph-mon[125879]: from='client.? 172.21.15.66:0/1925854850' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-21T22:11:39.692 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:39 smithi066 ceph-mon[125879]: from='client.? 172.21.15.66:0/2570971796' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:11:40.695 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.116+0000 7fe2c13f6700 1 Processor -- start 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.117+0000 7fe2c13f6700 1 -- start start 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.117+0000 7fe2c13f6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.117+0000 7fe2c13f6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc07a840 con 0x7fe2bc07be00 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.117+0000 7fe2baffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.117+0000 7fe2baffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45786/0 (socket says 172.21.15.66:45786) 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.117+0000 7fe2baffd700 1 -- 172.21.15.66:0/760846983 learned_addr learned my addr 172.21.15.66:0/760846983 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:40.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.118+0000 7fe2baffd700 1 -- 172.21.15.66:0/760846983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc07a980 con 0x7fe2bc07be00 2023-07-21T22:11:40.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.118+0000 7fe2baffd700 1 --2- 172.21.15.66:0/760846983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc07a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0098a0 tx=0x7fe2ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5c4049a79341cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:40.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.118+0000 7fe2b9ffb700 1 -- 172.21.15.66:0/760846983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac004160 con 0x7fe2bc07be00 2023-07-21T22:11:40.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.118+0000 7fe2b9ffb700 1 -- 172.21.15.66:0/760846983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2ac0042c0 con 0x7fe2bc07be00 2023-07-21T22:11:40.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.118+0000 7fe2b9ffb700 1 -- 172.21.15.66:0/760846983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac011a00 con 0x7fe2bc07be00 2023-07-21T22:11:40.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 -- 172.21.15.66:0/760846983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 msgr2=0x7fe2bc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:40.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 --2- 172.21.15.66:0/760846983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc07a300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0098a0 tx=0x7fe2ac009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:40.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 -- 172.21.15.66:0/760846983 shutdown_connections 2023-07-21T22:11:40.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 --2- 172.21.15.66:0/760846983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc07a300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:40.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 -- 172.21.15.66:0/760846983 >> 172.21.15.66:0/760846983 conn(0x7fe2bc100970 msgr2=0x7fe2bc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:40.700 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 -- 172.21.15.66:0/760846983 shutdown_connections 2023-07-21T22:11:40.700 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.119+0000 7fe2c13f6700 1 -- 172.21.15.66:0/760846983 wait complete. 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.120+0000 7fe2c13f6700 1 Processor -- start 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.120+0000 7fe2c13f6700 1 -- start start 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.120+0000 7fe2c13f6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.121+0000 7fe2c13f6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc19b880 con 0x7fe2bc07be00 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.121+0000 7fe2baffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.121+0000 7fe2baffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45796/0 (socket says 172.21.15.66:45796) 2023-07-21T22:11:40.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.121+0000 7fe2baffd700 1 -- 172.21.15.66:0/2570971796 learned_addr learned my addr 172.21.15.66:0/2570971796 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.121+0000 7fe2baffd700 1 -- 172.21.15.66:0/2570971796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac009550 con 0x7fe2bc07be00 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.121+0000 7fe2baffd700 1 --2- 172.21.15.66:0/2570971796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc19b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009130 tx=0x7fe2ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac003ab0 con 0x7fe2bc07be00 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc19ba80 con 0x7fe2bc07be00 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe2ac003c10 con 0x7fe2bc07be00 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe2ac0247d0 con 0x7fe2bc07be00 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc19bea0 con 0x7fe2bc07be00 2023-07-21T22:11:40.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe2ac024d50 con 0x7fe2bc07be00 2023-07-21T22:11:40.703 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.122+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe2ac0194a0 con 0x7fe2bc07be00 2023-07-21T22:11:40.703 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.123+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc1952f0 con 0x7fe2bc07be00 2023-07-21T22:11:40.703 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.126+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe2ac01abb0 con 0x7fe2bc07be00 2023-07-21T22:11:40.703 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.200+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fe2bc02c850 con 0x7fe2bc07be00 2023-07-21T22:11:40.703 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.200+0000 7fe2a3fff700 1 -- 172.21.15.66:0/2570971796 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fe2ac035090 con 0x7fe2bc07be00 2023-07-21T22:11:40.703 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 msgr2=0x7fe2bc19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 --2- 172.21.15.66:0/2570971796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc19b340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac009130 tx=0x7fe2ac011ad0 comp rx=0 tx=0).stop 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 shutdown_connections 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 --2- 172.21.15.66:0/2570971796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe2bc07be00 0x7fe2bc19b340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 >> 172.21.15.66:0/2570971796 conn(0x7fe2bc100970 msgr2=0x7fe2bc1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 shutdown_connections 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:39.201+0000 7fe2c13f6700 1 -- 172.21.15.66:0/2570971796 wait complete. 2023-07-21T22:11:40.704 INFO:teuthology.orchestra.run.smithi066.stdout:Restarting the monitor... 2023-07-21T22:11:40.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:40 smithi066 systemd[1]: Stopping Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:11:41.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:41 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a[125855]: 2023-07-21T22:11:41.132+0000 7f3136531700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:11:41.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:41 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a[125855]: 2023-07-21T22:11:41.132+0000 7f3136531700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-21T22:11:41.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:41 smithi066 bash[126218]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a 2023-07-21T22:11:42.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:41 smithi066 systemd[1]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T22:11:42.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:41 smithi066 systemd[1]: Stopped Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:11:42.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:41 smithi066 systemd[1]: Starting Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:11:42.629 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 podman[126339]: 2023-07-21T22:11:42.630 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 bash[126339]: 084af29f85fd2d06b82b8c90996f5dbef34f1927b5af7ecd84b3cb86435deb8d 2023-07-21T22:11:42.630 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:11:42.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:11:42.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: pidfile_write: ignore empty --pid-file 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: load: jerasure load: lrc 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Git sha 0 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: DB SUMMARY 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: DB Session ID: E4DRCQ8QOX9PH3EBV8HV 2023-07-21T22:11:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98745 ; 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:11:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.env: 0x55dbb4ee1720 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.info_log: 0x55dbb6843260 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:11:42.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.statistics: (nil) 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.use_fsync: 0 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:11:42.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.db_log_dir: 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.wal_dir: 2023-07-21T22:11:42.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:11:42.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.write_buffer_manager: 0x55dbb67643c0 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:11:42.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.unordered_write: 0 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.row_cache: None 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.wal_filter: None 2023-07-21T22:11:42.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.wal_compression: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:11:42.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:11:42.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:11:42.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_open_files: -1 2023-07-21T22:11:42.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Compression algorithms supported: 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kZSTD supported: 0 2023-07-21T22:11:42.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:11:42.708 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.merge_operator: 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_filter: None 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:11:42.709 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55dbb6843360) 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_type: 0 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_index_type: 0 2023-07-21T22:11:42.710 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_shortening: 1 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: checksum: 4 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: no_block_cache: 0 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache: 0x55dbb68031f0 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_options: 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: capacity : 536870912 2023-07-21T22:11:42.711 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_shard_bits : 4 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: strict_capacity_limit : 0 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_compressed: (nil) 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: persistent_cache: (nil) 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size: 4096 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size_deviation: 10 2023-07-21T22:11:42.712 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_restart_interval: 16 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_block_restart_interval: 1 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: metadata_block_size: 4096 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: partition_filters: 0 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: use_delta_encoding: 1 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: filter_policy: bloomfilter 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: whole_key_filtering: 1 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: verify_compression: 0 2023-07-21T22:11:42.713 INFO:journalctl@ceph.mon.a.smithi066.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: format_version: 5 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_index_compression: 1 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_align: 0 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: max_auto_readahead_size: 262144 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: prepopulate_block_cache: 0 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:11:42.714 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression: NoCompression 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.num_levels: 7 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:11:42.715 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:11:42.716 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:11:42.717 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:11:42.718 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:11:42.719 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:11:42.720 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:11:42.721 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:11:42.722 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:11:42.723 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:11:42.723 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:11:42.723 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:11:42.723 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:11:42.723 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:11:42.723 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.ttl: 2592000 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:11:42.724 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:11:42.725 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 54a19289-29cc-4dbe-ade3-5886df3d875c 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977502597803, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977502599692, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93986, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91991, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13640, "raw_average_key_size": 49, "raw_value_size": 84556, "raw_average_value_size": 309, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689977502, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "54a19289-29cc-4dbe-ade3-5886df3d875c", "db_session_id": "E4DRCQ8QOX9PH3EBV8HV", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:11:42.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977502599826, "job": 1, "event": "recovery_finished"} 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55dbb689c000 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: DB pointer 0x55dbb6888000 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** DB Stats ** 2023-07-21T22:11:42.727 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-07-21T22:11:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: L0 2/0 93.62 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: Sum 2/0 93.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-07-21T22:11:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:11:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative compaction: 0.00 GB write, 13.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval compaction: 0.00 GB write, 13.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:11:42.731 INFO:journalctl@ceph.mon.a.smithi066.stdout: Block cache BinnedLRUCache@0x55dbb68031f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: starting mon.a rank 0 at public addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] at bind addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???) e1 preinit fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).mds e1 new map 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).mds e1 print_map 2023-07-21T22:11:42.732 INFO:journalctl@ceph.mon.a.smithi066.stdout: e1 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout: legacy client fscid: -1 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout: No filesystems configured 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:11:42.733 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expand map: {default=false} 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expand map: {default=info} 2023-07-21T22:11:42.734 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:11:42.735 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:11:42.736 INFO:teuthology.orchestra.run.smithi066.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-21T22:11:42.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 systemd[1]: Started Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:11:42.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T22:11:42.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: monmap e1: 1 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0]} removed_ranks: {} 2023-07-21T22:11:42.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: fsmap 2023-07-21T22:11:42.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T22:11:42.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:42 smithi066 ceph-mon[126379]: mgrmap e1: no daemons active 2023-07-21T22:11:43.690 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.187+0000 7f3c39e1d700 1 Processor -- start 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.187+0000 7f3c39e1d700 1 -- start start 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.187+0000 7f3c39e1d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.187+0000 7f3c39e1d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3407a880 con 0x7f3c3407be40 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.188+0000 7f3c337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.188+0000 7f3c337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59540/0 (socket says 172.21.15.66:59540) 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.188+0000 7f3c337fe700 1 -- 172.21.15.66:0/175412729 learned_addr learned my addr 172.21.15.66:0/175412729 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:43.691 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.188+0000 7f3c337fe700 1 -- 172.21.15.66:0/175412729 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3407a9c0 con 0x7f3c3407be40 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.189+0000 7f3c337fe700 1 --2- 172.21.15.66:0/175412729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3407a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3c240098a0 tx=0x7f3c24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b493efdfe69e5b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.189+0000 7f3c327fc700 1 -- 172.21.15.66:0/175412729 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c24004160 con 0x7f3c3407be40 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.189+0000 7f3c327fc700 1 -- 172.21.15.66:0/175412729 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c240042c0 con 0x7f3c3407be40 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.189+0000 7f3c327fc700 1 -- 172.21.15.66:0/175412729 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c24011a00 con 0x7f3c3407be40 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 -- 172.21.15.66:0/175412729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 msgr2=0x7f3c3407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 --2- 172.21.15.66:0/175412729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3407a340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3c240098a0 tx=0x7f3c24009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:43.692 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 -- 172.21.15.66:0/175412729 shutdown_connections 2023-07-21T22:11:43.695 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 --2- 172.21.15.66:0/175412729 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3407a340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:43.695 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 -- 172.21.15.66:0/175412729 >> 172.21.15.66:0/175412729 conn(0x7f3c34100970 msgr2=0x7f3c34102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:43.695 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 -- 172.21.15.66:0/175412729 shutdown_connections 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.190+0000 7f3c39e1d700 1 -- 172.21.15.66:0/175412729 wait complete. 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.191+0000 7f3c39e1d700 1 Processor -- start 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.191+0000 7f3c39e1d700 1 -- start start 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c39e1d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c39e1d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3419b860 con 0x7f3c3407be40 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:43.696 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59546/0 (socket says 172.21.15.66:59546) 2023-07-21T22:11:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c337fe700 1 -- 172.21.15.66:0/1793424669 learned_addr learned my addr 172.21.15.66:0/1793424669 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c337fe700 1 -- 172.21.15.66:0/1793424669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24009550 con 0x7f3c3407be40 2023-07-21T22:11:43.697 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.192+0000 7f3c337fe700 1 --2- 172.21.15.66:0/1793424669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3419b320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009fb0 tx=0x7f3c24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:43.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c24003a40 con 0x7f3c3407be40 2023-07-21T22:11:43.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3c24003ba0 con 0x7f3c3407be40 2023-07-21T22:11:43.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c2401a750 con 0x7f3c3407be40 2023-07-21T22:11:43.698 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3419ba60 con 0x7f3c3407be40 2023-07-21T22:11:43.699 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3419be80 con 0x7f3c3407be40 2023-07-21T22:11:43.699 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3c2401ac80 con 0x7f3c3407be40 2023-07-21T22:11:43.699 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.193+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3c24024430 con 0x7f3c3407be40 2023-07-21T22:11:43.699 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.194+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c34195300 con 0x7f3c3407be40 2023-07-21T22:11:43.700 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.197+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3c24024b60 con 0x7f3c3407be40 2023-07-21T22:11:43.700 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.269+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f3c3402c850 con 0x7f3c3407be40 2023-07-21T22:11:43.700 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.272+0000 7f3c30ff9700 1 -- 172.21.15.66:0/1793424669 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3c24024d00 con 0x7f3c3407be40 2023-07-21T22:11:43.700 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 msgr2=0x7f3c3419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:43.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 --2- 172.21.15.66:0/1793424669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3419b320 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009fb0 tx=0x7f3c24011ad0 comp rx=0 tx=0).stop 2023-07-21T22:11:43.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 shutdown_connections 2023-07-21T22:11:43.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 --2- 172.21.15.66:0/1793424669 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c3407be40 0x7f3c3419b320 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:43.701 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 >> 172.21.15.66:0/1793424669 conn(0x7f3c34100970 msgr2=0x7f3c34102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:43.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 shutdown_connections 2023-07-21T22:11:43.702 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:43.273+0000 7f3c39e1d700 1 -- 172.21.15.66:0/1793424669 wait complete. 2023-07-21T22:11:43.702 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-21T22:11:43.725 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:11:43.726 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mgr... 2023-07-21T22:11:43.726 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 9283 ... 2023-07-21T22:11:43.726 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 8765 ... 2023-07-21T22:11:44.103 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T22:11:44.103 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-21T22:11:44.314 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c.target.wants/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@.service. 2023-07-21T22:11:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:44 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1793424669' entity='client.admin' 2023-07-21T22:11:44.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:44 smithi066 systemd[1]: Starting Ceph mgr.x for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:11:45.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:44 smithi066 podman[126610]: 2023-07-21T22:11:45.121 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:44 smithi066 bash[126610]: cb1b8fddfef59791a8f4d87a844985a71eaea537a472c4df79fb0164a7646a92 2023-07-21T22:11:45.144 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:11:45.145 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-07-21T22:11:45.162 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:11:45.162 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-07-21T22:11:45.162 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-07-21T22:11:45.162 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T22:11:45.183 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:11:45.183 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-07-21T22:11:45.202 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:11:45.202 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-07-21T22:11:45.202 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-07-21T22:11:45.203 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-21T22:11:45.203 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr to start... 2023-07-21T22:11:45.203 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr... 2023-07-21T22:11:45.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:45 smithi066 systemd[1]: Started Ceph mgr.x for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:11:45.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:45.193+0000 7f1605071200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:11:45.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:45.314+0000 7f1605071200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:11:45.861 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:45.435+0000 7f1605071200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:11:45.862 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:45.580+0000 7f1605071200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:11:46.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:45 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4085906462' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:11:46.193 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:45.820+0000 7f1605071200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:11:46.401 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:46.402 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:11:46.402 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:11:46.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:11:46.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:11:46.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:11:46.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:11:46.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:11:46.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:11:46.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:11:46.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:46.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:11:46.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:11:46.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:11:46.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:46.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:11:46.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:11:46.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:11:46.411 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:11:46.412 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:11:46.412 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:11:46.412 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:11:46.412 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:11:46.412 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:11:46.412 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.413 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:11:46.414 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:11:36.433872+0000", 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:46.415 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.761+0000 7fccfe9b6700 1 Processor -- start 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.762+0000 7fccfe9b6700 1 -- start start 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.762+0000 7fccfe9b6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf8105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.762+0000 7fccfe9b6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8105c70 con 0x7fccf8105320 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.763+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf8105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.763+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf8105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59562/0 (socket says 172.21.15.66:59562) 2023-07-21T22:11:46.416 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.763+0000 7fccf7fff700 1 -- 172.21.15.66:0/3604363499 learned_addr learned my addr 172.21.15.66:0/3604363499 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.763+0000 7fccf7fff700 1 -- 172.21.15.66:0/3604363499 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf8105db0 con 0x7fccf8105320 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.763+0000 7fccf7fff700 1 --2- 172.21.15.66:0/3604363499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf8105730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcce80098a0 tx=0x7fcce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9121b252332689 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccf6ffd700 1 -- 172.21.15.66:0/3604363499 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce8004160 con 0x7fccf8105320 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccf6ffd700 1 -- 172.21.15.66:0/3604363499 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcce80042c0 con 0x7fccf8105320 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccf6ffd700 1 -- 172.21.15.66:0/3604363499 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce8011a00 con 0x7fccf8105320 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 -- 172.21.15.66:0/3604363499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 msgr2=0x7fccf8105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 --2- 172.21.15.66:0/3604363499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf8105730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcce80098a0 tx=0x7fcce8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:46.417 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 -- 172.21.15.66:0/3604363499 shutdown_connections 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 --2- 172.21.15.66:0/3604363499 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf8105730 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 -- 172.21.15.66:0/3604363499 >> 172.21.15.66:0/3604363499 conn(0x7fccf8100970 msgr2=0x7fccf8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 -- 172.21.15.66:0/3604363499 shutdown_connections 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.764+0000 7fccfe9b6700 1 -- 172.21.15.66:0/3604363499 wait complete. 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.765+0000 7fccfe9b6700 1 Processor -- start 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccfe9b6700 1 -- start start 2023-07-21T22:11:46.418 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccfe9b6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:46.420 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccfe9b6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf8199600 con 0x7fccf8105320 2023-07-21T22:11:46.420 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:46.421 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccf7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf81990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59566/0 (socket says 172.21.15.66:59566) 2023-07-21T22:11:46.421 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccf7fff700 1 -- 172.21.15.66:0/4085906462 learned_addr learned my addr 172.21.15.66:0/4085906462 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:46.421 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.766+0000 7fccf7fff700 1 -- 172.21.15.66:0/4085906462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce8009550 con 0x7fccf8105320 2023-07-21T22:11:46.421 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.767+0000 7fccf7fff700 1 --2- 172.21.15.66:0/4085906462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf81990c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009130 tx=0x7fcce8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:46.421 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.767+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce8003b70 con 0x7fccf8105320 2023-07-21T22:11:46.421 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.767+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fcce8003cd0 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.767+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce80197d0 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.767+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf8199800 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.767+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf8199c20 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.768+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcce80199c0 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.768+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcce800b720 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.768+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf8061b40 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.771+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fcce802a060 con 0x7fccf8105320 2023-07-21T22:11:46.422 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.849+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fccf819cc30 con 0x7fccf8105320 2023-07-21T22:11:46.423 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.849+0000 7fccf57fa700 1 -- 172.21.15.66:0/4085906462 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fcce8019e00 con 0x7fccf8105320 2023-07-21T22:11:46.423 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.850+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 msgr2=0x7fccf81990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:46.423 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.850+0000 7fccfe9b6700 1 --2- 172.21.15.66:0/4085906462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf81990c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009130 tx=0x7fcce8011ad0 comp rx=0 tx=0).stop 2023-07-21T22:11:46.423 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.851+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 shutdown_connections 2023-07-21T22:11:46.423 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.851+0000 7fccfe9b6700 1 --2- 172.21.15.66:0/4085906462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fccf8105320 0x7fccf81990c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:46.424 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.851+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 >> 172.21.15.66:0/4085906462 conn(0x7fccf8100970 msgr2=0x7fccf81020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:46.424 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.851+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 shutdown_connections 2023-07-21T22:11:46.424 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:45.851+0000 7fccfe9b6700 1 -- 172.21.15.66:0/4085906462 wait complete. 2023-07-21T22:11:46.424 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (1/15)... 2023-07-21T22:11:46.592 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:46.252+0000 7f1605071200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:11:46.592 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:46.382+0000 7f1605071200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:11:46.592 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:46.550+0000 7f1605071200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:11:46.942 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:46.679+0000 7f1605071200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:11:47.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:47 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:47.248+0000 7f1605071200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:11:48.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:48 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:48.078+0000 7f1605071200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:11:48.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:48 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:48.293+0000 7f1605071200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:11:49.374 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:49 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/846672249' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:11:49.503 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:49.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:11:49.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:11:49.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:11:49.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:11:49.504 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:49.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:11:49.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.507 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:11:49.507 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:49.507 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:11:49.507 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:11:49.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:11:49.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:11:49.513 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:11:49.514 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:49.515 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:11:49.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:11:49.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:11:49.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T22:11:49.516 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:11:49.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:11:49.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:11:49.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:11:49.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:11:49.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:49.517 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:11:36.433872+0000", 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:11:49.518 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.882+0000 7f48060af700 1 Processor -- start 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.882+0000 7f48060af700 1 -- start start 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.882+0000 7f48060af700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.882+0000 7f48060af700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480007a840 con 0x7f480007be00 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.883+0000 7f47ff7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.883+0000 7f47ff7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59574/0 (socket says 172.21.15.66:59574) 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.883+0000 7f47ff7fe700 1 -- 172.21.15.66:0/2479181631 learned_addr learned my addr 172.21.15.66:0/2479181631 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:49.519 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.883+0000 7f47ff7fe700 1 -- 172.21.15.66:0/2479181631 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480007a980 con 0x7f480007be00 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.883+0000 7f47ff7fe700 1 --2- 172.21.15.66:0/2479181631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480007a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47f00098a0 tx=0x7f47f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9f9d93b3c70a44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f47fe7fc700 1 -- 172.21.15.66:0/2479181631 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47f0004160 con 0x7f480007be00 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f47fe7fc700 1 -- 172.21.15.66:0/2479181631 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47f00042c0 con 0x7f480007be00 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f47fe7fc700 1 -- 172.21.15.66:0/2479181631 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47f0011a00 con 0x7f480007be00 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f48060af700 1 -- 172.21.15.66:0/2479181631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 msgr2=0x7f480007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f48060af700 1 --2- 172.21.15.66:0/2479181631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480007a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f47f00098a0 tx=0x7f47f0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:49.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f48060af700 1 -- 172.21.15.66:0/2479181631 shutdown_connections 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f48060af700 1 --2- 172.21.15.66:0/2479181631 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480007a300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.884+0000 7f48060af700 1 -- 172.21.15.66:0/2479181631 >> 172.21.15.66:0/2479181631 conn(0x7f4800100970 msgr2=0x7f4800102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.885+0000 7f48060af700 1 -- 172.21.15.66:0/2479181631 shutdown_connections 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.885+0000 7f48060af700 1 -- 172.21.15.66:0/2479181631 wait complete. 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.886+0000 7f48060af700 1 Processor -- start 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.886+0000 7f48060af700 1 -- start start 2023-07-21T22:11:49.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.886+0000 7f48060af700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.886+0000 7f48060af700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f480019b880 con 0x7f480007be00 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47ff7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47ff7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59590/0 (socket says 172.21.15.66:59590) 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47ff7fe700 1 -- 172.21.15.66:0/846672249 learned_addr learned my addr 172.21.15.66:0/846672249 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47ff7fe700 1 -- 172.21.15.66:0/846672249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0009550 con 0x7f480007be00 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47ff7fe700 1 --2- 172.21.15.66:0/846672249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480019b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009130 tx=0x7f47f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:49.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47f0003ab0 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f47f0003c10 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47f00247d0 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.887+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480019ba80 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.888+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480019bea0 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.888+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f47f0024d00 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.888+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f47f0019450 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.889+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48001952f0 con 0x7f480007be00 2023-07-21T22:11:49.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.891+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f47f001abb0 con 0x7f480007be00 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.968+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f480002c850 con 0x7f480007be00 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.969+0000 7f47fcff9700 1 -- 172.21.15.66:0/846672249 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f47f001ad50 con 0x7f480007be00 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 msgr2=0x7f480019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 --2- 172.21.15.66:0/846672249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480019b340 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009130 tx=0x7f47f0011ad0 comp rx=0 tx=0).stop 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 shutdown_connections 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 --2- 172.21.15.66:0/846672249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f480007be00 0x7f480019b340 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:49.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 >> 172.21.15.66:0/846672249 conn(0x7f4800100970 msgr2=0x7f48001020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:49.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 shutdown_connections 2023-07-21T22:11:49.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:48.970+0000 7f48060af700 1 -- 172.21.15.66:0/846672249 wait complete. 2023-07-21T22:11:49.525 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (2/15)... 2023-07-21T22:11:49.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:49 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:49.332+0000 7f1605071200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:11:49.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:49 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:49.460+0000 7f1605071200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:11:50.111 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:49 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:49.692+0000 7f1605071200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:11:50.403 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:50.069+0000 7f1605071200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:11:50.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:50.362+0000 7f1605071200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:11:50.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:50.565+0000 7f1605071200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:11:51.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:50.928+0000 7f1605071200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:11:52.409 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:51.928+0000 7f1605071200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:11:52.409 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:52.113+0000 7f1605071200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:11:52.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:52.367+0000 7f1605071200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:11:52.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:52.525+0000 7f1605071200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:11:53.192 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:52.819+0000 7f1605071200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:11:54.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:11:53 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:11:53.969+0000 7f1605071200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: Activating manager daemon x 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: mgrmap e2: x(active, starting, since 0.00463265s) 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:11:54.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:11:54.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: Manager daemon x is now available 2023-07-21T22:11:54.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:11:54.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:11:54.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' 2023-07-21T22:11:54.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:54 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' 2023-07-21T22:11:55.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:55 smithi066 ceph-mon[126379]: from='mgr.14100 172.21.15.66:0/2888744859' entity='mgr.x' 2023-07-21T22:11:55.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:55 smithi066 ceph-mon[126379]: mgrmap e3: x(active, since 1.01055s) 2023-07-21T22:11:57.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:57 smithi066 ceph-mon[126379]: mgrmap e4: x(active, since 2s) 2023-07-21T22:11:58.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:11:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:11:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T22:11:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T22:11:58.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T22:11:58.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T22:11:58.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T22:11:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T22:11:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T22:11:58.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T22:11:58.724 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T22:11:58.724 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T22:11:58.724 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T22:11:58.724 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.724 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T22:11:58.725 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T22:11:58.725 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T22:11:58.725 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T22:11:58.725 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T22:11:58.725 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T22:11:58.726 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T22:11:58.726 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T22:11:58.726 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T22:11:58.726 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.726 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T22:11:58.726 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T22:11:58.727 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T22:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T22:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T22:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2023-07-21T22:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.728 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T22:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T22:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T22:11:36.433872+0000", 2023-07-21T22:11:58.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T22:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2023-07-21T22:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T22:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:11:58.730 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.948+0000 7f0654e75700 1 Processor -- start 2023-07-21T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.948+0000 7f0654e75700 1 -- start start 2023-07-21T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.948+0000 7f0654e75700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.948+0000 7f0654e75700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065007a860 con 0x7f065007be20 2023-07-21T22:11:58.731 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.949+0000 7f064f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.949+0000 7f064f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58512/0 (socket says 172.21.15.66:58512) 2023-07-21T22:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.949+0000 7f064f7fe700 1 -- 172.21.15.66:0/1599595724 learned_addr learned my addr 172.21.15.66:0/1599595724 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:58.732 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.949+0000 7f064f7fe700 1 -- 172.21.15.66:0/1599595724 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065007a9a0 con 0x7f065007be20 2023-07-21T22:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.949+0000 7f064f7fe700 1 --2- 172.21.15.66:0/1599595724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065007a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f06400098a0 tx=0x7f0640009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45446e7e0a53b812 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.949+0000 7f064e7fc700 1 -- 172.21.15.66:0/1599595724 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0640004160 con 0x7f065007be20 2023-07-21T22:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f064e7fc700 1 -- 172.21.15.66:0/1599595724 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f06400042c0 con 0x7f065007be20 2023-07-21T22:11:58.733 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f064e7fc700 1 -- 172.21.15.66:0/1599595724 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0640011a00 con 0x7f065007be20 2023-07-21T22:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 -- 172.21.15.66:0/1599595724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 msgr2=0x7f065007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 --2- 172.21.15.66:0/1599595724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065007a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f06400098a0 tx=0x7f0640009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 -- 172.21.15.66:0/1599595724 shutdown_connections 2023-07-21T22:11:58.734 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 --2- 172.21.15.66:0/1599595724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065007a320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 -- 172.21.15.66:0/1599595724 >> 172.21.15.66:0/1599595724 conn(0x7f06501007f0 msgr2=0x7f0650102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 -- 172.21.15.66:0/1599595724 shutdown_connections 2023-07-21T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.950+0000 7f0654e75700 1 -- 172.21.15.66:0/1599595724 wait complete. 2023-07-21T22:11:58.735 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.951+0000 7f0654e75700 1 Processor -- start 2023-07-21T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.951+0000 7f0654e75700 1 -- start start 2023-07-21T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.952+0000 7f0654e75700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.952+0000 7f0654e75700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065019b640 con 0x7f065007be20 2023-07-21T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.952+0000 7f064f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:58.736 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.952+0000 7f064f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58520/0 (socket says 172.21.15.66:58520) 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.952+0000 7f064f7fe700 1 -- 172.21.15.66:0/3991665779 learned_addr learned my addr 172.21.15.66:0/3991665779 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.952+0000 7f064f7fe700 1 -- 172.21.15.66:0/3991665779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0640009550 con 0x7f065007be20 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.953+0000 7f064f7fe700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065019b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0640009fb0 tx=0x7f0640011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.953+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0640003b70 con 0x7f065007be20 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.953+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0640003cd0 con 0x7f065007be20 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.953+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f065019b840 con 0x7f065007be20 2023-07-21T22:11:58.737 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.953+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06400247a0 con 0x7f065007be20 2023-07-21T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.953+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065019bc60 con 0x7f065007be20 2023-07-21T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.954+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0650061b40 con 0x7f065007be20 2023-07-21T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.954+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f064001a430 con 0x7f065007be20 2023-07-21T22:11:58.738 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.955+0000 7f064cff9700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f0638038710 0x7f063803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.955+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f064004c4d0 con 0x7f065007be20 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.955+0000 7f064effd700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f0638038710 0x7f063803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.956+0000 7f064effd700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f0638038710 0x7f063803abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0644006fd0 tx=0x7f0644006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:57.961+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f064001ab40 con 0x7f065007be20 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.246+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f065019ed70 con 0x7f065007be20 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.247+0000 7f064cff9700 1 -- 172.21.15.66:0/3991665779 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0640018350 con 0x7f065007be20 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f0638038710 msgr2=0x7f063803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:58.739 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f0638038710 0x7f063803abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0644006fd0 tx=0x7f0644006e40 comp rx=0 tx=0).stop 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 msgr2=0x7f065019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065019b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0640009fb0 tx=0x7f0640011ad0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 shutdown_connections 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f0638038710 0x7f063803abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 --2- 172.21.15.66:0/3991665779 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f065007be20 0x7f065019b100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.249+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 >> 172.21.15.66:0/3991665779 conn(0x7f06501007f0 msgr2=0x7f0650101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:58.740 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.250+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 shutdown_connections 2023-07-21T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:58.250+0000 7f0654e75700 1 -- 172.21.15.66:0/3991665779 wait complete. 2023-07-21T22:11:58.741 INFO:teuthology.orchestra.run.smithi066.stdout:mgr is available 2023-07-21T22:11:58.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:58 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3991665779' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T22:11:59.909 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:59.909 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T22:11:59.909 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:11:59.910 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T22:11:59.910 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T22:11:59.910 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T22:11:59.910 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T22:11:59.910 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:59.910 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T22:11:59.911 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T22:11:59.912 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.255+0000 7fd055702700 1 Processor -- start 2023-07-21T22:11:59.912 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.255+0000 7fd055702700 1 -- start start 2023-07-21T22:11:59.912 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.255+0000 7fd055702700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.255+0000 7fd055702700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05007a840 con 0x7fd05007be00 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.256+0000 7fd04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.256+0000 7fd04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58522/0 (socket says 172.21.15.66:58522) 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.256+0000 7fd04effd700 1 -- 172.21.15.66:0/4094972427 learned_addr learned my addr 172.21.15.66:0/4094972427 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.256+0000 7fd04effd700 1 -- 172.21.15.66:0/4094972427 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05007a980 con 0x7fd05007be00 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.256+0000 7fd04effd700 1 --2- 172.21.15.66:0/4094972427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05007a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098a0 tx=0x7fd040009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6aeeb96f135384 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:59.913 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd04dffb700 1 -- 172.21.15.66:0/4094972427 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd040004160 con 0x7fd05007be00 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd04dffb700 1 -- 172.21.15.66:0/4094972427 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd0400042c0 con 0x7fd05007be00 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd04dffb700 1 -- 172.21.15.66:0/4094972427 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd040011a00 con 0x7fd05007be00 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 -- 172.21.15.66:0/4094972427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 msgr2=0x7fd05007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 --2- 172.21.15.66:0/4094972427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05007a300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098a0 tx=0x7fd040009bb0 comp rx=0 tx=0).stop 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 -- 172.21.15.66:0/4094972427 shutdown_connections 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 --2- 172.21.15.66:0/4094972427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05007a300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 -- 172.21.15.66:0/4094972427 >> 172.21.15.66:0/4094972427 conn(0x7fd050100970 msgr2=0x7fd050102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:59.914 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 -- 172.21.15.66:0/4094972427 shutdown_connections 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.257+0000 7fd055702700 1 -- 172.21.15.66:0/4094972427 wait complete. 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.259+0000 7fd055702700 1 Processor -- start 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.259+0000 7fd055702700 1 -- start start 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd055702700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd055702700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05019b880 con 0x7fd05007be00 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:59.915 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd04effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58524/0 (socket says 172.21.15.66:58524) 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd04effd700 1 -- 172.21.15.66:0/4280851948 learned_addr learned my addr 172.21.15.66:0/4280851948 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd04effd700 1 -- 172.21.15.66:0/4280851948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd040009550 con 0x7fd05007be00 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.260+0000 7fd04effd700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05019b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd040009130 tx=0x7fd040011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.261+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd040003ab0 con 0x7fd05007be00 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.261+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd040003c10 con 0x7fd05007be00 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.261+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0400247d0 con 0x7fd05007be00 2023-07-21T22:11:59.916 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.261+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05019ba80 con 0x7fd05007be00 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.261+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05019bea0 con 0x7fd05007be00 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.262+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fd04001abb0 con 0x7fd05007be00 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.262+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0501952f0 con 0x7fd05007be00 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.263+0000 7fd037fff700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fd038038710 0x7fd03803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.263+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd04004c430 con 0x7fd05007be00 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.263+0000 7fd04e7fc700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fd038038710 0x7fd03803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.264+0000 7fd04e7fc700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fd038038710 0x7fd03803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd044006fd0 tx=0x7fd044006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:11:59.917 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.268+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd040018710 con 0x7fd05007be00 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.473+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd050061b40 con 0x7fd05007be00 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.474+0000 7fd037fff700 1 -- 172.21.15.66:0/4280851948 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd040018350 con 0x7fd05007be00 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fd038038710 msgr2=0x7fd03803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fd038038710 0x7fd03803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd044006fd0 tx=0x7fd044006e40 comp rx=0 tx=0).stop 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 msgr2=0x7fd05019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05019b340 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd040009130 tx=0x7fd040011ad0 comp rx=0 tx=0).stop 2023-07-21T22:11:59.918 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 shutdown_connections 2023-07-21T22:11:59.919 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fd038038710 0x7fd03803abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:59.919 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 --2- 172.21.15.66:0/4280851948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd05007be00 0x7fd05019b340 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:11:59.919 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.476+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 >> 172.21.15.66:0/4280851948 conn(0x7fd050100970 msgr2=0x7fd0501020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:11:59.919 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.477+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 shutdown_connections 2023-07-21T22:11:59.919 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:11:59.477+0000 7fd055702700 1 -- 172.21.15.66:0/4280851948 wait complete. 2023-07-21T22:11:59.919 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling cephadm module... 2023-07-21T22:11:59.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:11:59 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4280851948' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T22:12:01.050 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/208935758' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-21T22:12:01.050 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setuser ceph since I am not root 2023-07-21T22:12:01.050 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setgroup ceph since I am not root 2023-07-21T22:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.406+0000 7f1f71220700 1 Processor -- start 2023-07-21T22:12:01.189 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.406+0000 7f1f71220700 1 -- start start 2023-07-21T22:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.407+0000 7f1f71220700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.407+0000 7f1f71220700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c07a840 con 0x7f1f6c07be00 2023-07-21T22:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.407+0000 7f1f6ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:01.190 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.407+0000 7f1f6ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58530/0 (socket says 172.21.15.66:58530) 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.407+0000 7f1f6ad9d700 1 -- 172.21.15.66:0/3106405083 learned_addr learned my addr 172.21.15.66:0/3106405083 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.407+0000 7f1f6ad9d700 1 -- 172.21.15.66:0/3106405083 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c07a980 con 0x7f1f6c07be00 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.408+0000 7f1f6ad9d700 1 --2- 172.21.15.66:0/3106405083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c07a300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098a0 tx=0x7f1f5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e9974d87f4bad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.408+0000 7f1f69d9b700 1 -- 172.21.15.66:0/3106405083 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c004160 con 0x7f1f6c07be00 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.408+0000 7f1f69d9b700 1 -- 172.21.15.66:0/3106405083 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f5c0042c0 con 0x7f1f6c07be00 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.408+0000 7f1f69d9b700 1 -- 172.21.15.66:0/3106405083 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c011a00 con 0x7f1f6c07be00 2023-07-21T22:12:01.191 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.408+0000 7f1f71220700 1 -- 172.21.15.66:0/3106405083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 msgr2=0x7f1f6c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.408+0000 7f1f71220700 1 --2- 172.21.15.66:0/3106405083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c07a300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0098a0 tx=0x7f1f5c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.409+0000 7f1f71220700 1 -- 172.21.15.66:0/3106405083 shutdown_connections 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.409+0000 7f1f71220700 1 --2- 172.21.15.66:0/3106405083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c07a300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.409+0000 7f1f71220700 1 -- 172.21.15.66:0/3106405083 >> 172.21.15.66:0/3106405083 conn(0x7f1f6c100970 msgr2=0x7f1f6c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.409+0000 7f1f71220700 1 -- 172.21.15.66:0/3106405083 shutdown_connections 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.409+0000 7f1f71220700 1 -- 172.21.15.66:0/3106405083 wait complete. 2023-07-21T22:12:01.192 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.410+0000 7f1f71220700 1 Processor -- start 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.410+0000 7f1f71220700 1 -- start start 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.411+0000 7f1f71220700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.411+0000 7f1f71220700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c19b880 con 0x7f1f6c07be00 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.411+0000 7f1f6ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.411+0000 7f1f6ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58538/0 (socket says 172.21.15.66:58538) 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.411+0000 7f1f6ad9d700 1 -- 172.21.15.66:0/208935758 learned_addr learned my addr 172.21.15.66:0/208935758 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.411+0000 7f1f6ad9d700 1 -- 172.21.15.66:0/208935758 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009550 con 0x7f1f6c07be00 2023-07-21T22:12:01.193 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.412+0000 7f1f6ad9d700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c19b340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009130 tx=0x7f1f5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.412+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c003ab0 con 0x7f1f6c07be00 2023-07-21T22:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.412+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1f5c003c10 con 0x7f1f6c07be00 2023-07-21T22:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.412+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f5c0247d0 con 0x7f1f6c07be00 2023-07-21T22:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.412+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c19ba80 con 0x7f1f6c07be00 2023-07-21T22:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.412+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c19bea0 con 0x7f1f6c07be00 2023-07-21T22:12:01.194 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.413+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f1f5c01abb0 con 0x7f1f6c07be00 2023-07-21T22:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.413+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c1952f0 con 0x7f1f6c07be00 2023-07-21T22:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.414+0000 7f1f53fff700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f1f54038710 0x7f1f5403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.414+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1f5c04c430 con 0x7f1f6c07be00 2023-07-21T22:12:01.195 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.414+0000 7f1f6a59c700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f1f54038710 0x7f1f5403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.415+0000 7f1f6a59c700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f1f54038710 0x7f1f5403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f60006fd0 tx=0x7f1f60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.419+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1f5c018710 con 0x7f1f6c07be00 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.686+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f1f6c02c850 con 0x7f1f6c07be00 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.740+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f1f5c02d770 con 0x7f1f6c07be00 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.741+0000 7f1f53fff700 1 -- 172.21.15.66:0/208935758 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f1f5c04c190 con 0x7f1f6c07be00 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f1f54038710 msgr2=0x7f1f5403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:01.196 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f1f54038710 0x7f1f5403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f60006fd0 tx=0x7f1f60006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 msgr2=0x7f1f6c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c19b340 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009130 tx=0x7f1f5c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 shutdown_connections 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f1f54038710 0x7f1f5403abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 --2- 172.21.15.66:0/208935758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f6c07be00 0x7f1f6c19b340 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.744+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 >> 172.21.15.66:0/208935758 conn(0x7f1f6c100970 msgr2=0x7f1f6c1020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.745+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 shutdown_connections 2023-07-21T22:12:01.197 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:00.745+0000 7f1f71220700 1 -- 172.21.15.66:0/208935758 wait complete. 2023-07-21T22:12:01.416 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:01.008+0000 7fe6d908a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:12:01.416 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:01.118+0000 7fe6d908a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:12:01.417 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:01.249+0000 7fe6d908a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:12:01.417 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:01.374+0000 7fe6d908a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:12:01.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/208935758' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-21T22:12:01.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-mon[126379]: mgrmap e5: x(active, since 6s) 2023-07-21T22:12:01.943 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:01.629+0000 7fe6d908a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:12:02.400 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:02.057+0000 7fe6d908a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:12:02.400 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:02.188+0000 7fe6d908a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:12:02.401 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:02.358+0000 7fe6d908a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:12:02.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:12:02.473 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-21T22:12:02.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:12:02.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T22:12:02.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T22:12:02.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:12:02.474 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.708+0000 7fe036e94700 1 Processor -- start 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.708+0000 7fe036e94700 1 -- start start 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.708+0000 7fe036e94700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03007a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.708+0000 7fe036e94700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03007a840 con 0x7fe03007be00 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.709+0000 7fe034c30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03007a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.709+0000 7fe034c30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03007a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58586/0 (socket says 172.21.15.66:58586) 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.709+0000 7fe034c30700 1 -- 172.21.15.66:0/3137902570 learned_addr learned my addr 172.21.15.66:0/3137902570 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:02.475 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.709+0000 7fe034c30700 1 -- 172.21.15.66:0/3137902570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03007a980 con 0x7fe03007be00 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.709+0000 7fe034c30700 1 --2- 172.21.15.66:0/3137902570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03007a300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe0240098a0 tx=0x7fe024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddff42f1cb0ea428 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.710+0000 7fe02f7fe700 1 -- 172.21.15.66:0/3137902570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe024004160 con 0x7fe03007be00 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.710+0000 7fe02f7fe700 1 -- 172.21.15.66:0/3137902570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0240042c0 con 0x7fe03007be00 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.710+0000 7fe02f7fe700 1 -- 172.21.15.66:0/3137902570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe024011a00 con 0x7fe03007be00 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.710+0000 7fe036e94700 1 -- 172.21.15.66:0/3137902570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 msgr2=0x7fe03007a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.710+0000 7fe036e94700 1 --2- 172.21.15.66:0/3137902570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03007a300 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe0240098a0 tx=0x7fe024009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:02.476 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.710+0000 7fe036e94700 1 -- 172.21.15.66:0/3137902570 shutdown_connections 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.711+0000 7fe036e94700 1 --2- 172.21.15.66:0/3137902570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03007a300 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.711+0000 7fe036e94700 1 -- 172.21.15.66:0/3137902570 >> 172.21.15.66:0/3137902570 conn(0x7fe030100970 msgr2=0x7fe030102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.711+0000 7fe036e94700 1 -- 172.21.15.66:0/3137902570 shutdown_connections 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.711+0000 7fe036e94700 1 -- 172.21.15.66:0/3137902570 wait complete. 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.712+0000 7fe036e94700 1 Processor -- start 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.712+0000 7fe036e94700 1 -- start start 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.712+0000 7fe036e94700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:02.477 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.712+0000 7fe036e94700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03019b880 con 0x7fe03007be00 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.713+0000 7fe034c30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.713+0000 7fe034c30700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03019b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:58594/0 (socket says 172.21.15.66:58594) 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.713+0000 7fe034c30700 1 -- 172.21.15.66:0/235824343 learned_addr learned my addr 172.21.15.66:0/235824343 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.713+0000 7fe034c30700 1 -- 172.21.15.66:0/235824343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024009550 con 0x7fe03007be00 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.713+0000 7fe034c30700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03019b340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe024009fb0 tx=0x7fe024011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.713+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe024003a40 con 0x7fe03007be00 2023-07-21T22:12:02.478 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.714+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03019ba80 con 0x7fe03007be00 2023-07-21T22:12:02.479 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.714+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe024003ba0 con 0x7fe03007be00 2023-07-21T22:12:02.479 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.714+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe02401a750 con 0x7fe03007be00 2023-07-21T22:12:02.479 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.714+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03019bea0 con 0x7fe03007be00 2023-07-21T22:12:02.479 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.715+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe02401a9d0 con 0x7fe03007be00 2023-07-21T22:12:02.479 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.715+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0301952f0 con 0x7fe03007be00 2023-07-21T22:12:02.479 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.715+0000 7fe02dffb700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 0x7fe02003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:02.480 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.715+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe02404c3f0 con 0x7fe03007be00 2023-07-21T22:12:02.480 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.715+0000 7fe02ffff700 1 -- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 msgr2=0x7fe02003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:02.480 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.715+0000 7fe02ffff700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 0x7fe02003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:12:02.480 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.721+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe024018d20 con 0x7fe03007be00 2023-07-21T22:12:02.480 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.916+0000 7fe02ffff700 1 -- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 msgr2=0x7fe02003ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:01.916+0000 7fe02ffff700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 0x7fe02003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.000+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fe030061b40 con 0x7fe03007be00 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.000+0000 7fe02dffb700 1 -- 172.21.15.66:0/235824343 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fe024018b40 con 0x7fe03007be00 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 msgr2=0x7fe02003ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 0x7fe02003ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 msgr2=0x7fe03019b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:02.481 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03019b340 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe024009fb0 tx=0x7fe024011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 shutdown_connections 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7fe0200386c0 0x7fe02003ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 --2- 172.21.15.66:0/235824343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe03007be00 0x7fe03019b340 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 >> 172.21.15.66:0/235824343 conn(0x7fe030100970 msgr2=0x7fe0301020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 shutdown_connections 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:02.003+0000 7fe036e94700 1 -- 172.21.15.66:0/235824343 wait complete. 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2023-07-21T22:12:02.482 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 5... 2023-07-21T22:12:02.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:02.497+0000 7fe6d908a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:12:03.129 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/235824343' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T22:12:03.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:03 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:03.088+0000 7fe6d908a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:12:04.192 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:03 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:03.870+0000 7fe6d908a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:12:04.193 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:04 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:04.085+0000 7fe6d908a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:12:05.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:05.076+0000 7fe6d908a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:12:05.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:05.204+0000 7fe6d908a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:12:05.860 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:05.437+0000 7fe6d908a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:12:05.860 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:05.817+0000 7fe6d908a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:12:06.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:06.103+0000 7fe6d908a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:12:06.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:06.306+0000 7fe6d908a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:12:07.192 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:06.670+0000 7fe6d908a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:12:08.150 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:07 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:07.669+0000 7fe6d908a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:12:08.150 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:07 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:07.850+0000 7fe6d908a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:12:08.150 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:08.107+0000 7fe6d908a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:12:08.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:08.265+0000 7fe6d908a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:12:08.942 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:08.554+0000 7fe6d908a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:12:10.192 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:09.671+0000 7fe6d908a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:12:10.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: Active manager daemon x restarted 2023-07-21T22:12:10.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: Activating manager daemon x 2023-07-21T22:12:10.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: osdmap e2: 0 total, 0 up, 0 in 2023-07-21T22:12:10.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: mgrmap e6: x(active, starting, since 0.00402956s) 2023-07-21T22:12:10.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:12:10.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: Manager daemon x is now available 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:10.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:10.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:09 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:11.076 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:12:11.076 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-21T22:12:11.076 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T22:12:11.076 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:12:11.076 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.000+0000 7f7e508e6700 1 Processor -- start 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.000+0000 7f7e508e6700 1 -- start start 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.000+0000 7f7e508e6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c108210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.000+0000 7f7e508e6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c079da0 con 0x7f7e4c105e30 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.001+0000 7f7e4a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c108210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.001+0000 7f7e4a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c108210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55570/0 (socket says 172.21.15.66:55570) 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.001+0000 7f7e4a59c700 1 -- 172.21.15.66:0/2233386700 learned_addr learned my addr 172.21.15.66:0/2233386700 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:11.077 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.001+0000 7f7e4a59c700 1 -- 172.21.15.66:0/2233386700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c079ee0 con 0x7f7e4c105e30 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.001+0000 7f7e4a59c700 1 --2- 172.21.15.66:0/2233386700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c108210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0098a0 tx=0x7f7e3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45400ef94c2f0c9b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e4959a700 1 -- 172.21.15.66:0/2233386700 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e3c004160 con 0x7f7e4c105e30 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e4959a700 1 -- 172.21.15.66:0/2233386700 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e3c0042c0 con 0x7f7e4c105e30 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e4959a700 1 -- 172.21.15.66:0/2233386700 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e3c011a00 con 0x7f7e4c105e30 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e508e6700 1 -- 172.21.15.66:0/2233386700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 msgr2=0x7f7e4c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e508e6700 1 --2- 172.21.15.66:0/2233386700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c108210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0098a0 tx=0x7f7e3c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.078 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e508e6700 1 -- 172.21.15.66:0/2233386700 shutdown_connections 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e508e6700 1 --2- 172.21.15.66:0/2233386700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c108210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e508e6700 1 -- 172.21.15.66:0/2233386700 >> 172.21.15.66:0/2233386700 conn(0x7f7e4c0ff890 msgr2=0x7f7e4c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.002+0000 7f7e508e6700 1 -- 172.21.15.66:0/2233386700 shutdown_connections 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.003+0000 7f7e508e6700 1 -- 172.21.15.66:0/2233386700 wait complete. 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.003+0000 7f7e508e6700 1 Processor -- start 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.004+0000 7f7e508e6700 1 -- start start 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.004+0000 7f7e508e6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:11.079 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.004+0000 7f7e508e6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c19b840 con 0x7f7e4c105e30 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.004+0000 7f7e4a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.004+0000 7f7e4a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55576/0 (socket says 172.21.15.66:55576) 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.004+0000 7f7e4a59c700 1 -- 172.21.15.66:0/1563246703 learned_addr learned my addr 172.21.15.66:0/1563246703 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.005+0000 7f7e4a59c700 1 -- 172.21.15.66:0/1563246703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c009550 con 0x7f7e4c105e30 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.005+0000 7f7e4a59c700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c19b300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009130 tx=0x7f7e3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.005+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e3c003ab0 con 0x7f7e4c105e30 2023-07-21T22:12:11.080 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.005+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7e3c003c10 con 0x7f7e4c105e30 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.005+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c19ba40 con 0x7f7e4c105e30 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.005+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e3c0247d0 con 0x7f7e4c105e30 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.006+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c19be60 con 0x7f7e4c105e30 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.007+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7e3c02d3f0 con 0x7f7e4c105e30 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.007+0000 7f7e3b7fe700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.007+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 --> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e3403b320 con 0x7f7e34038760 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.007+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7e3c04c670 con 0x7f7e4c105e30 2023-07-21T22:12:11.081 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.007+0000 7f7e49d9b700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.007+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.208+0000 7f7e49d9b700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.208+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.609+0000 7f7e49d9b700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:03.609+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:04.409+0000 7f7e49d9b700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:11.082 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:04.409+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:06.010+0000 7f7e49d9b700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:06.011+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:09.214+0000 7f7e49d9b700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/979491945 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:09.214+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:09.676+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f7e3c02d8d0 con 0x7f7e4c105e30 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:09.676+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 msgr2=0x7f7e3403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:09.676+0000 7f7e3b7fe700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.677+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f7e3c00fb40 con 0x7f7e4c105e30 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.677+0000 7f7e3b7fe700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f7e3403bfe0 0x7f7e3403e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.677+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e3403b320 con 0x7f7e3403bfe0 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.678+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f7e3403bfe0 0x7f7e3403e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.679+0000 7f7e49d9b700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f7e3403bfe0 0x7f7e3403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7e40003a10 tx=0x7f7e400092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.681+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f7e3403b320 con 0x7f7e3403bfe0 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.687+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f7e4c19c640 con 0x7f7e3403bfe0 2023-07-21T22:12:11.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e3b7fe700 1 -- 172.21.15.66:0/1563246703 <== mgr.14118 v2:172.21.15.66:6800/3065709937 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7e4c19c640 con 0x7f7e3403bfe0 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f7e3403bfe0 msgr2=0x7f7e3403e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f7e3403bfe0 0x7f7e3403e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7e40003a10 tx=0x7f7e400092b0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 msgr2=0x7f7e4c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c19b300 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009130 tx=0x7f7e3c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 shutdown_connections 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f7e3403bfe0 0x7f7e3403e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c105e30 0x7f7e4c19b300 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 --2- 172.21.15.66:0/1563246703 >> [v2:172.21.15.66:6800/979491945,v1:172.21.15.66:6801/979491945] conn(0x7f7e34038760 0x7f7e3403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:11.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 >> 172.21.15.66:0/1563246703 conn(0x7f7e4c0ff890 msgr2=0x7f7e4c100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:11.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 shutdown_connections 2023-07-21T22:12:11.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:10.688+0000 7f7e508e6700 1 -- 172.21.15.66:0/1563246703 wait complete. 2023-07-21T22:12:11.086 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 5 is available 2023-07-21T22:12:11.086 INFO:teuthology.orchestra.run.smithi066.stdout:Setting orchestrator backend to cephadm... 2023-07-21T22:12:11.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: Found migration_current of "None". Setting to last migration. 2023-07-21T22:12:11.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:11.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:12:11.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:12:11.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:11.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:11.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:10 smithi066 ceph-mon[126379]: mgrmap e7: x(active, since 1.00708s) 2023-07-21T22:12:12.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:11 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:10] ENGINE Bus STARTING 2023-07-21T22:12:12.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:11 smithi066 ceph-mon[126379]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T22:12:12.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:11 smithi066 ceph-mon[126379]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T22:12:12.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:11 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:10] ENGINE Serving on http://172.21.15.66:8765 2023-07-21T22:12:12.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:11 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:12.250 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.604+0000 7f2d90686700 1 Processor -- start 2023-07-21T22:12:12.254 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.604+0000 7f2d90686700 1 -- start start 2023-07-21T22:12:12.254 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.605+0000 7f2d90686700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8807a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:12.254 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.605+0000 7f2d90686700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8807a8a0 con 0x7f2d8807be60 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.605+0000 7f2d8e422700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8807a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.605+0000 7f2d8e422700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8807a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55660/0 (socket says 172.21.15.66:55660) 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.605+0000 7f2d8e422700 1 -- 172.21.15.66:0/4042560921 learned_addr learned my addr 172.21.15.66:0/4042560921 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.606+0000 7f2d8e422700 1 -- 172.21.15.66:0/4042560921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8807a9e0 con 0x7f2d8807be60 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.606+0000 7f2d8e422700 1 --2- 172.21.15.66:0/4042560921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8807a360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d840098a0 tx=0x7f2d84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58da41b05eb5b042 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.606+0000 7f2d8d420700 1 -- 172.21.15.66:0/4042560921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d84004160 con 0x7f2d8807be60 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.606+0000 7f2d8d420700 1 -- 172.21.15.66:0/4042560921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d840042c0 con 0x7f2d8807be60 2023-07-21T22:12:12.255 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.606+0000 7f2d8d420700 1 -- 172.21.15.66:0/4042560921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d84011a00 con 0x7f2d8807be60 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 -- 172.21.15.66:0/4042560921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 msgr2=0x7f2d8807a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 --2- 172.21.15.66:0/4042560921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8807a360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d840098a0 tx=0x7f2d84009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 -- 172.21.15.66:0/4042560921 shutdown_connections 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 --2- 172.21.15.66:0/4042560921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8807a360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 -- 172.21.15.66:0/4042560921 >> 172.21.15.66:0/4042560921 conn(0x7f2d88100970 msgr2=0x7f2d88102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 -- 172.21.15.66:0/4042560921 shutdown_connections 2023-07-21T22:12:12.256 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.607+0000 7f2d90686700 1 -- 172.21.15.66:0/4042560921 wait complete. 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.608+0000 7f2d90686700 1 Processor -- start 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.608+0000 7f2d90686700 1 -- start start 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.608+0000 7f2d90686700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.608+0000 7f2d90686700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8819fc50 con 0x7f2d8807be60 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d8e422700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d8e422700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55662/0 (socket says 172.21.15.66:55662) 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d8e422700 1 -- 172.21.15.66:0/248184921 learned_addr learned my addr 172.21.15.66:0/248184921 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:12.257 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d8e422700 1 -- 172.21.15.66:0/248184921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009550 con 0x7f2d8807be60 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d8e422700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8819f710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009130 tx=0x7f2d84011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d84003ab0 con 0x7f2d8807be60 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2d84003c10 con 0x7f2d8807be60 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d840247d0 con 0x7f2d8807be60 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.609+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8819fe50 con 0x7f2d8807be60 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.610+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d881a0270 con 0x7f2d8807be60 2023-07-21T22:12:12.258 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.611+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d88061b40 con 0x7f2d8807be60 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.611+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f2d8401abb0 con 0x7f2d8807be60 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.611+0000 7f2d7f7fe700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f2d74038630 0x7f2d7403aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.611+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2d8404c650 con 0x7f2d8807be60 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.611+0000 7f2d8dc21700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f2d74038630 0x7f2d7403aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.612+0000 7f2d8dc21700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f2d74038630 0x7f2d7403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.616+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2d84018530 con 0x7f2d8807be60 2023-07-21T22:12:12.259 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.734+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2d8402d770 con 0x7f2d8807be60 2023-07-21T22:12:12.260 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.845+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d88102630 con 0x7f2d74038630 2023-07-21T22:12:12.260 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.851+0000 7f2d7f7fe700 1 -- 172.21.15.66:0/248184921 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2d88102630 con 0x7f2d74038630 2023-07-21T22:12:12.260 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f2d74038630 msgr2=0x7f2d7403aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:12.260 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f2d74038630 0x7f2d7403aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:12.260 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 msgr2=0x7f2d8819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:12.260 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8819f710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009130 tx=0x7f2d84011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:12.261 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 shutdown_connections 2023-07-21T22:12:12.261 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f2d74038630 0x7f2d7403aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:12.261 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 --2- 172.21.15.66:0/248184921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d8807be60 0x7f2d8819f710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:12.261 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 >> 172.21.15.66:0/248184921 conn(0x7f2d88100970 msgr2=0x7f2d88106560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:12.261 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.854+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 shutdown_connections 2023-07-21T22:12:12.261 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:11.855+0000 7f2d90686700 1 -- 172.21.15.66:0/248184921 wait complete. 2023-07-21T22:12:12.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:12 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:10] ENGINE Serving on https://172.21.15.66:7150 2023-07-21T22:12:12.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:12 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:10] ENGINE Bus STARTED 2023-07-21T22:12:12.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:12 smithi066 ceph-mon[126379]: mgrmap e8: x(active, since 2s) 2023-07-21T22:12:12.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:12 smithi066 ceph-mon[126379]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:12.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:12 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:12.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:12 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:13.438 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.719+0000 7f93b22c9700 1 Processor -- start 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.720+0000 7f93b22c9700 1 -- start start 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.720+0000 7f93b22c9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.720+0000 7f93b22c9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac105c70 con 0x7f93ac105320 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.720+0000 7f93abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.720+0000 7f93abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34896/0 (socket says 172.21.15.66:34896) 2023-07-21T22:12:13.439 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.720+0000 7f93abfff700 1 -- 172.21.15.66:0/358939408 learned_addr learned my addr 172.21.15.66:0/358939408 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.721+0000 7f93abfff700 1 -- 172.21.15.66:0/358939408 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac105db0 con 0x7f93ac105320 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.721+0000 7f93abfff700 1 --2- 172.21.15.66:0/358939408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac105730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f939c0098a0 tx=0x7f939c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c69c11a148cad8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.721+0000 7f93aaffd700 1 -- 172.21.15.66:0/358939408 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939c004160 con 0x7f93ac105320 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.721+0000 7f93aaffd700 1 -- 172.21.15.66:0/358939408 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f939c0042c0 con 0x7f93ac105320 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.721+0000 7f93aaffd700 1 -- 172.21.15.66:0/358939408 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939c011a00 con 0x7f93ac105320 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 -- 172.21.15.66:0/358939408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 msgr2=0x7f93ac105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:13.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 --2- 172.21.15.66:0/358939408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac105730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f939c0098a0 tx=0x7f939c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 -- 172.21.15.66:0/358939408 shutdown_connections 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 --2- 172.21.15.66:0/358939408 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac105730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 -- 172.21.15.66:0/358939408 >> 172.21.15.66:0/358939408 conn(0x7f93ac100970 msgr2=0x7f93ac102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 -- 172.21.15.66:0/358939408 shutdown_connections 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.722+0000 7f93b22c9700 1 -- 172.21.15.66:0/358939408 wait complete. 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.723+0000 7f93b22c9700 1 Processor -- start 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.723+0000 7f93b22c9700 1 -- start start 2023-07-21T22:12:13.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.724+0000 7f93b22c9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.724+0000 7f93b22c9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ac19b820 con 0x7f93ac105320 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.724+0000 7f93abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.724+0000 7f93abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34904/0 (socket says 172.21.15.66:34904) 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.724+0000 7f93abfff700 1 -- 172.21.15.66:0/3262293626 learned_addr learned my addr 172.21.15.66:0/3262293626 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.724+0000 7f93abfff700 1 -- 172.21.15.66:0/3262293626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009550 con 0x7f93ac105320 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.725+0000 7f93abfff700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac19b2e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f939c009fb0 tx=0x7f939c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:13.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.725+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939c003a40 con 0x7f93ac105320 2023-07-21T22:12:13.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.725+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f939c003ba0 con 0x7f93ac105320 2023-07-21T22:12:13.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.725+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ac19ba20 con 0x7f93ac105320 2023-07-21T22:12:13.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.725+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f939c01a750 con 0x7f93ac105320 2023-07-21T22:12:13.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.725+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ac19be40 con 0x7f93ac105320 2023-07-21T22:12:13.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.726+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f939c024b60 con 0x7f93ac105320 2023-07-21T22:12:13.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.726+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ac1097b0 con 0x7f93ac105320 2023-07-21T22:12:13.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.727+0000 7f93a97fa700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9394038710 0x7f939403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:13.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.727+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f939c04c7b0 con 0x7f93ac105320 2023-07-21T22:12:13.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.727+0000 7f93ab7fe700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9394038710 0x7f939403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:13.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.727+0000 7f93ab7fe700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9394038710 0x7f939403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006fd0 tx=0x7f93a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:13.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.732+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f939c024430 con 0x7f93ac105320 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.938+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f93ac102ca0 con 0x7f9394038710 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.939+0000 7f93a97fa700 1 -- 172.21.15.66:0/3262293626 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f93ac102ca0 con 0x7f9394038710 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.941+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9394038710 msgr2=0x7f939403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.941+0000 7f93b22c9700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9394038710 0x7f939403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006fd0 tx=0x7f93a0006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 msgr2=0x7f93ac19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac19b2e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f939c009fb0 tx=0x7f939c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:13.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 shutdown_connections 2023-07-21T22:12:13.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9394038710 0x7f939403abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:13.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 --2- 172.21.15.66:0/3262293626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93ac105320 0x7f93ac19b2e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:13.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 >> 172.21.15.66:0/3262293626 conn(0x7f93ac100970 msgr2=0x7f93ac102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:13.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 shutdown_connections 2023-07-21T22:12:13.446 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:12.942+0000 7f93b22c9700 1 -- 172.21.15.66:0/3262293626 wait complete. 2023-07-21T22:12:13.446 INFO:teuthology.orchestra.run.smithi066.stdout:Generating ssh key... 2023-07-21T22:12:14.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:13 smithi066 ceph-mon[126379]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:14.693 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: Generating public/private rsa key pair. 2023-07-21T22:12:14.693 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: Your identification has been saved in /tmp/tmp3xyg9ww5/key. 2023-07-21T22:12:14.693 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: Your public key has been saved in /tmp/tmp3xyg9ww5/key.pub. 2023-07-21T22:12:14.693 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: The key fingerprint is: 2023-07-21T22:12:14.693 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: SHA256:9KLQVglq0OHSa2TV+WogIu/c1QBLnL/AfuTmVZ2kijE ceph-6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: The key's randomart image is: 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: +---[RSA 3072]----+ 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | .o.oo. . | 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | +*o .o. . | 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | .oB+ +. + . | 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: |. .==oEo .+ o | 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | o o+=oOS+. | 2023-07-21T22:12:14.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | ...oB.*. | 2023-07-21T22:12:14.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | o . =.o | 2023-07-21T22:12:14.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | o . . | 2023-07-21T22:12:14.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: | | 2023-07-21T22:12:14.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: +----[SHA256]-----+ 2023-07-21T22:12:14.822 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.954+0000 7f3068955700 1 Processor -- start 2023-07-21T22:12:14.822 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.954+0000 7f3068955700 1 -- start start 2023-07-21T22:12:14.823 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.955+0000 7f3068955700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306407a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:14.823 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.955+0000 7f3068955700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306407a820 con 0x7f306407bde0 2023-07-21T22:12:14.823 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.955+0000 7f306259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306407a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:14.823 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.955+0000 7f306259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306407a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34914/0 (socket says 172.21.15.66:34914) 2023-07-21T22:12:14.823 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.955+0000 7f306259c700 1 -- 172.21.15.66:0/43320834 learned_addr learned my addr 172.21.15.66:0/43320834 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:14.823 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.955+0000 7f306259c700 1 -- 172.21.15.66:0/43320834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306407a960 con 0x7f306407bde0 2023-07-21T22:12:14.824 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.956+0000 7f306259c700 1 --2- 172.21.15.66:0/43320834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306407a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f30540098a0 tx=0x7f3054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca9d53c9136ee39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:14.824 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.956+0000 7f306159a700 1 -- 172.21.15.66:0/43320834 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054004160 con 0x7f306407bde0 2023-07-21T22:12:14.824 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.956+0000 7f306159a700 1 -- 172.21.15.66:0/43320834 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f30540042c0 con 0x7f306407bde0 2023-07-21T22:12:14.825 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.956+0000 7f306159a700 1 -- 172.21.15.66:0/43320834 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054011a00 con 0x7f306407bde0 2023-07-21T22:12:14.825 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 -- 172.21.15.66:0/43320834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 msgr2=0x7f306407a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:14.825 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 --2- 172.21.15.66:0/43320834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306407a2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f30540098a0 tx=0x7f3054009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:14.825 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 -- 172.21.15.66:0/43320834 shutdown_connections 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 --2- 172.21.15.66:0/43320834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306407a2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 -- 172.21.15.66:0/43320834 >> 172.21.15.66:0/43320834 conn(0x7f3064100970 msgr2=0x7f3064102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 -- 172.21.15.66:0/43320834 shutdown_connections 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.957+0000 7f3068955700 1 -- 172.21.15.66:0/43320834 wait complete. 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.958+0000 7f3068955700 1 Processor -- start 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.959+0000 7f3068955700 1 -- start start 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.959+0000 7f3068955700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:14.826 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.959+0000 7f3068955700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306419b850 con 0x7f306407bde0 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.959+0000 7f306259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.959+0000 7f306259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34924/0 (socket says 172.21.15.66:34924) 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.959+0000 7f306259c700 1 -- 172.21.15.66:0/378546251 learned_addr learned my addr 172.21.15.66:0/378546251 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f306259c700 1 -- 172.21.15.66:0/378546251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009550 con 0x7f306407bde0 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f306259c700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306419b310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3054009fb0 tx=0x7f3054011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3054003a40 con 0x7f306407bde0 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3054003ba0 con 0x7f306407bde0 2023-07-21T22:12:14.827 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306419ba50 con 0x7f306407bde0 2023-07-21T22:12:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305401a750 con 0x7f306407bde0 2023-07-21T22:12:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.960+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306419be70 con 0x7f306407bde0 2023-07-21T22:12:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.962+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30641952f0 con 0x7f306407bde0 2023-07-21T22:12:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.962+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3054024b60 con 0x7f306407bde0 2023-07-21T22:12:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.962+0000 7f30537fe700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f304c038710 0x7f304c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.962+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f305404c750 con 0x7f306407bde0 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.963+0000 7f3061d9b700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f304c038710 0x7f304c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.963+0000 7f3061d9b700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f304c038710 0x7f304c03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3058006fd0 tx=0x7f3058006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:13.967+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3054024430 con 0x7f306407bde0 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.175+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3064060950 con 0x7f304c038710 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.351+0000 7f30537fe700 1 -- 172.21.15.66:0/378546251 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3064060950 con 0x7f304c038710 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.353+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f304c038710 msgr2=0x7f304c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.353+0000 7f3068955700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f304c038710 0x7f304c03abc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3058006fd0 tx=0x7f3058006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:14.829 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.353+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 msgr2=0x7f306419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.353+0000 7f3068955700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306419b310 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3054009fb0 tx=0x7f3054011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.354+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 shutdown_connections 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.354+0000 7f3068955700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f304c038710 0x7f304c03abc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.354+0000 7f3068955700 1 --2- 172.21.15.66:0/378546251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f306407bde0 0x7f306419b310 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.354+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 >> 172.21.15.66:0/378546251 conn(0x7f3064100970 msgr2=0x7f3064102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.354+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 shutdown_connections 2023-07-21T22:12:14.830 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:14.354+0000 7f3068955700 1 -- 172.21.15.66:0/378546251 wait complete. 2023-07-21T22:12:15.692 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:15 smithi066 ceph-mon[126379]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:15.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:15 smithi066 ceph-mon[126379]: Generating ssh key... 2023-07-21T22:12:15.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:15 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:15.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:15 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:16.001 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:12:16.002 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.318+0000 7f426282f700 1 Processor -- start 2023-07-21T22:12:16.002 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.318+0000 7f426282f700 1 -- start start 2023-07-21T22:12:16.002 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.318+0000 7f426282f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:16.002 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.318+0000 7f426282f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c07a820 con 0x7f425c07bde0 2023-07-21T22:12:16.002 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.318+0000 7f426182d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:16.007 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426182d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34926/0 (socket says 172.21.15.66:34926) 2023-07-21T22:12:16.007 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426182d700 1 -- 172.21.15.66:0/4195955847 learned_addr learned my addr 172.21.15.66:0/4195955847 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:16.008 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426182d700 1 -- 172.21.15.66:0/4195955847 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c07a960 con 0x7f425c07bde0 2023-07-21T22:12:16.008 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426182d700 1 --2- 172.21.15.66:0/4195955847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c07a2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42580098a0 tx=0x7f4258009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb74396288fee2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:16.008 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426082b700 1 -- 172.21.15.66:0/4195955847 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4258004160 con 0x7f425c07bde0 2023-07-21T22:12:16.008 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426082b700 1 -- 172.21.15.66:0/4195955847 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f42580042c0 con 0x7f425c07bde0 2023-07-21T22:12:16.008 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.319+0000 7f426082b700 1 -- 172.21.15.66:0/4195955847 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4258011a00 con 0x7f425c07bde0 2023-07-21T22:12:16.009 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 -- 172.21.15.66:0/4195955847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 msgr2=0x7f425c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:16.009 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 --2- 172.21.15.66:0/4195955847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c07a2e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f42580098a0 tx=0x7f4258009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:16.009 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 -- 172.21.15.66:0/4195955847 shutdown_connections 2023-07-21T22:12:16.009 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 --2- 172.21.15.66:0/4195955847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c07a2e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:16.010 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 -- 172.21.15.66:0/4195955847 >> 172.21.15.66:0/4195955847 conn(0x7f425c1007f0 msgr2=0x7f425c102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:16.010 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 -- 172.21.15.66:0/4195955847 shutdown_connections 2023-07-21T22:12:16.010 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.320+0000 7f426282f700 1 -- 172.21.15.66:0/4195955847 wait complete. 2023-07-21T22:12:16.010 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.321+0000 7f426282f700 1 Processor -- start 2023-07-21T22:12:16.010 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.321+0000 7f426282f700 1 -- start start 2023-07-21T22:12:16.010 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.322+0000 7f426282f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.322+0000 7f426282f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c19b630 con 0x7f425c07bde0 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.322+0000 7f426182d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.322+0000 7f426182d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34942/0 (socket says 172.21.15.66:34942) 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.322+0000 7f426182d700 1 -- 172.21.15.66:0/3053190417 learned_addr learned my addr 172.21.15.66:0/3053190417 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.322+0000 7f426182d700 1 -- 172.21.15.66:0/3053190417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009550 con 0x7f425c07bde0 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.323+0000 7f426182d700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c19b0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4258006af0 tx=0x7f4258011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:16.011 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.323+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4258003a40 con 0x7f425c07bde0 2023-07-21T22:12:16.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.323+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4258003ba0 con 0x7f425c07bde0 2023-07-21T22:12:16.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.323+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c19b830 con 0x7f425c07bde0 2023-07-21T22:12:16.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.323+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f425801a700 con 0x7f425c07bde0 2023-07-21T22:12:16.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.323+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c19bc50 con 0x7f425c07bde0 2023-07-21T22:12:16.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.324+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4258024ad0 con 0x7f425c07bde0 2023-07-21T22:12:16.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.324+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c195110 con 0x7f425c07bde0 2023-07-21T22:12:16.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.325+0000 7f4252ffd700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f4248038760 0x7f424803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:16.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.325+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f425804c780 con 0x7f425c07bde0 2023-07-21T22:12:16.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.325+0000 7f426102c700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f4248038760 0x7f424803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:16.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.326+0000 7f426102c700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f4248038760 0x7f424803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f424c006fd0 tx=0x7f424c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:16.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.330+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4258018b80 con 0x7f425c07bde0 2023-07-21T22:12:16.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.536+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f425c060950 con 0x7f4248038760 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.537+0000 7f4252ffd700 1 -- 172.21.15.66:0/3053190417 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f425c060950 con 0x7f4248038760 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f4248038760 msgr2=0x7f424803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f4248038760 0x7f424803ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f424c006fd0 tx=0x7f424c006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 msgr2=0x7f425c19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c19b0f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4258006af0 tx=0x7f4258011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 shutdown_connections 2023-07-21T22:12:16.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f4248038760 0x7f424803ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 --2- 172.21.15.66:0/3053190417 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f425c07bde0 0x7f425c19b0f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 >> 172.21.15.66:0/3053190417 conn(0x7f425c1007f0 msgr2=0x7f425c101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 shutdown_connections 2023-07-21T22:12:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:15.540+0000 7f426282f700 1 -- 172.21.15.66:0/3053190417 wait complete. 2023-07-21T22:12:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:12:16.015 INFO:teuthology.orchestra.run.smithi066.stdout:Adding key to root@localhost authorized_keys... 2023-07-21T22:12:16.016 INFO:teuthology.orchestra.run.smithi066.stdout:Adding host smithi066... 2023-07-21T22:12:16.692 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:16 smithi066 ceph-mon[126379]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:17.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:17 smithi066 ceph-mon[126379]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:18.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:18 smithi066 ceph-mon[126379]: Deploying cephadm binary to smithi066 2023-07-21T22:12:20.402 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Added host 'smithi066' with addr '172.21.15.66' 2023-07-21T22:12:20.402 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.473+0000 7f9bd4151700 1 Processor -- start 2023-07-21T22:12:20.402 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.473+0000 7f9bd4151700 1 -- start start 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.473+0000 7f9bd4151700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc07a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.473+0000 7f9bd4151700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc07a880 con 0x7f9bcc07be40 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.474+0000 7f9bd1eed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc07a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.474+0000 7f9bd1eed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc07a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34946/0 (socket says 172.21.15.66:34946) 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.474+0000 7f9bd1eed700 1 -- 172.21.15.66:0/414608907 learned_addr learned my addr 172.21.15.66:0/414608907 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.474+0000 7f9bd1eed700 1 -- 172.21.15.66:0/414608907 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc07a9c0 con 0x7f9bcc07be40 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.474+0000 7f9bd1eed700 1 --2- 172.21.15.66:0/414608907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc07a340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80098a0 tx=0x7f9bc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec741c15c427b70 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:20.403 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd0eeb700 1 -- 172.21.15.66:0/414608907 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bc8004160 con 0x7f9bcc07be40 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd0eeb700 1 -- 172.21.15.66:0/414608907 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9bc80042c0 con 0x7f9bcc07be40 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd0eeb700 1 -- 172.21.15.66:0/414608907 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bc8011a00 con 0x7f9bcc07be40 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd4151700 1 -- 172.21.15.66:0/414608907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 msgr2=0x7f9bcc07a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd4151700 1 --2- 172.21.15.66:0/414608907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc07a340 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80098a0 tx=0x7f9bc8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd4151700 1 -- 172.21.15.66:0/414608907 shutdown_connections 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd4151700 1 --2- 172.21.15.66:0/414608907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc07a340 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.404 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.475+0000 7f9bd4151700 1 -- 172.21.15.66:0/414608907 >> 172.21.15.66:0/414608907 conn(0x7f9bcc100970 msgr2=0x7f9bcc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.476+0000 7f9bd4151700 1 -- 172.21.15.66:0/414608907 shutdown_connections 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.476+0000 7f9bd4151700 1 -- 172.21.15.66:0/414608907 wait complete. 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd4151700 1 Processor -- start 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd4151700 1 -- start start 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd4151700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd4151700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc19b860 con 0x7f9bcc07be40 2023-07-21T22:12:20.405 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd1eed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd1eed700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34960/0 (socket says 172.21.15.66:34960) 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.477+0000 7f9bd1eed700 1 -- 172.21.15.66:0/386957357 learned_addr learned my addr 172.21.15.66:0/386957357 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bd1eed700 1 -- 172.21.15.66:0/386957357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8009550 con 0x7f9bcc07be40 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bd1eed700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc19b320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009fb0 tx=0x7f9bc8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bc8003a40 con 0x7f9bcc07be40 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9bc8003ba0 con 0x7f9bcc07be40 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9bc801a750 con 0x7f9bcc07be40 2023-07-21T22:12:20.406 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc19ba60 con 0x7f9bcc07be40 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.478+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc19be80 con 0x7f9bcc07be40 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.479+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f9bc8024b60 con 0x7f9bcc07be40 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.479+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc195300 con 0x7f9bcc07be40 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.480+0000 7f9bc2ffd700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9bb8038710 0x7f9bb803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.480+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9bc804c750 con 0x7f9bcc07be40 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.480+0000 7f9bd16ec700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9bb8038710 0x7f9bb803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.481+0000 7f9bd16ec700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9bb8038710 0x7f9bb803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc006fd0 tx=0x7f9bbc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:20.407 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.485+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9bc8024430 con 0x7f9bcc07be40 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:16.698+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bcc060950 con 0x7f9bb8038710 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.959+0000 7f9bc2ffd700 1 -- 172.21.15.66:0/386957357 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9bcc060950 con 0x7f9bb8038710 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9bb8038710 msgr2=0x7f9bb803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9bb8038710 0x7f9bb803abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc006fd0 tx=0x7f9bbc006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 msgr2=0x7f9bcc19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc19b320 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009fb0 tx=0x7f9bc8011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.408 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 shutdown_connections 2023-07-21T22:12:20.409 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f9bb8038710 0x7f9bb803abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.409 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 --2- 172.21.15.66:0/386957357 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9bcc07be40 0x7f9bcc19b320 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:20.409 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.962+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 >> 172.21.15.66:0/386957357 conn(0x7f9bcc100970 msgr2=0x7f9bcc102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:20.409 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.963+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 shutdown_connections 2023-07-21T22:12:20.409 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:19.963+0000 7f9bd4151700 1 -- 172.21.15.66:0/386957357 wait complete. 2023-07-21T22:12:20.409 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mon service... 2023-07-21T22:12:21.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:20 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:21.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:20 smithi066 ceph-mon[126379]: Added host smithi066 2023-07-21T22:12:21.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:20 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:21.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-21T22:12:21.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.926+0000 7f61598a2700 1 Processor -- start 2023-07-21T22:12:21.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.926+0000 7f61598a2700 1 -- start start 2023-07-21T22:12:21.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.926+0000 7f61598a2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:21.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.926+0000 7f61598a2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615407a840 con 0x7f615407be00 2023-07-21T22:12:21.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.927+0000 7f6152ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.927+0000 7f6152ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34976/0 (socket says 172.21.15.66:34976) 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.927+0000 7f6152ffd700 1 -- 172.21.15.66:0/4182770535 learned_addr learned my addr 172.21.15.66:0/4182770535 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.927+0000 7f6152ffd700 1 -- 172.21.15.66:0/4182770535 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615407a980 con 0x7f615407be00 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.927+0000 7f6152ffd700 1 --2- 172.21.15.66:0/4182770535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615407a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61440098a0 tx=0x7f6144009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5122c99f413c406e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f6151ffb700 1 -- 172.21.15.66:0/4182770535 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6144004160 con 0x7f615407be00 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f6151ffb700 1 -- 172.21.15.66:0/4182770535 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f61440042c0 con 0x7f615407be00 2023-07-21T22:12:21.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f6151ffb700 1 -- 172.21.15.66:0/4182770535 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6144011a00 con 0x7f615407be00 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f61598a2700 1 -- 172.21.15.66:0/4182770535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 msgr2=0x7f615407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f61598a2700 1 --2- 172.21.15.66:0/4182770535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615407a300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61440098a0 tx=0x7f6144009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f61598a2700 1 -- 172.21.15.66:0/4182770535 shutdown_connections 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f61598a2700 1 --2- 172.21.15.66:0/4182770535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615407a300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f61598a2700 1 -- 172.21.15.66:0/4182770535 >> 172.21.15.66:0/4182770535 conn(0x7f6154100970 msgr2=0x7f6154102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.928+0000 7f61598a2700 1 -- 172.21.15.66:0/4182770535 shutdown_connections 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.929+0000 7f61598a2700 1 -- 172.21.15.66:0/4182770535 wait complete. 2023-07-21T22:12:21.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.929+0000 7f61598a2700 1 Processor -- start 2023-07-21T22:12:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.930+0000 7f61598a2700 1 -- start start 2023-07-21T22:12:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.930+0000 7f61598a2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.930+0000 7f61598a2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615419b880 con 0x7f615407be00 2023-07-21T22:12:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.930+0000 7f6152ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.930+0000 7f6152ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34988/0 (socket says 172.21.15.66:34988) 2023-07-21T22:12:21.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.930+0000 7f6152ffd700 1 -- 172.21.15.66:0/1501740992 learned_addr learned my addr 172.21.15.66:0/1501740992 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.931+0000 7f6152ffd700 1 -- 172.21.15.66:0/1501740992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144009550 con 0x7f615407be00 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.931+0000 7f6152ffd700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615419b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6144009130 tx=0x7f6144011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.931+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6144003ab0 con 0x7f615407be00 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.931+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6144003c10 con 0x7f615407be00 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.931+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615419ba80 con 0x7f615407be00 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.931+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61440247d0 con 0x7f615407be00 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.932+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615419bea0 con 0x7f615407be00 2023-07-21T22:12:21.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.933+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f614401abb0 con 0x7f615407be00 2023-07-21T22:12:21.590 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.933+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61541952f0 con 0x7f615407be00 2023-07-21T22:12:21.590 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.933+0000 7f61588a0700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f613c038760 0x7f613c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:21.590 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.933+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f614404c790 con 0x7f615407be00 2023-07-21T22:12:21.590 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.933+0000 7f61527fc700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f613c038760 0x7f613c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:21.590 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.934+0000 7f61527fc700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f613c038760 0x7f613c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6148006fd0 tx=0x7f6148006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:21.590 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:20.939+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6144018530 con 0x7f615407be00 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.167+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6154060950 con 0x7f613c038760 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.175+0000 7f61588a0700 1 -- 172.21.15.66:0/1501740992 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6154060950 con 0x7f613c038760 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.177+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f613c038760 msgr2=0x7f613c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.177+0000 7f61598a2700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f613c038760 0x7f613c03ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6148006fd0 tx=0x7f6148006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.177+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 msgr2=0x7f615419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.177+0000 7f61598a2700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615419b340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6144009130 tx=0x7f6144011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.178+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 shutdown_connections 2023-07-21T22:12:21.591 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.178+0000 7f61598a2700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f613c038760 0x7f613c03ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:21.592 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.178+0000 7f61598a2700 1 --2- 172.21.15.66:0/1501740992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f615407be00 0x7f615419b340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:21.592 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.178+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 >> 172.21.15.66:0/1501740992 conn(0x7f6154100970 msgr2=0x7f61541020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:21.592 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.178+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 shutdown_connections 2023-07-21T22:12:21.592 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:21.178+0000 7f61598a2700 1 -- 172.21.15.66:0/1501740992 wait complete. 2023-07-21T22:12:21.592 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mgr service... 2023-07-21T22:12:22.408 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:22 smithi066 ceph-mon[126379]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:22.408 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:22 smithi066 ceph-mon[126379]: Saving service mon spec with placement count:5 2023-07-21T22:12:22.408 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:22 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:23.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-21T22:12:23.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.261+0000 7fdf60f8f700 1 Processor -- start 2023-07-21T22:12:23.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.262+0000 7fdf60f8f700 1 -- start start 2023-07-21T22:12:23.012 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.262+0000 7fdf60f8f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.262+0000 7fdf60f8f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c105c70 con 0x7fdf5c105320 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.262+0000 7fdf5a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.262+0000 7fdf5a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43798/0 (socket says 172.21.15.66:43798) 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.262+0000 7fdf5a59c700 1 -- 172.21.15.66:0/1753439116 learned_addr learned my addr 172.21.15.66:0/1753439116 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.263+0000 7fdf5a59c700 1 -- 172.21.15.66:0/1753439116 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c105db0 con 0x7fdf5c105320 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.263+0000 7fdf5a59c700 1 --2- 172.21.15.66:0/1753439116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c105730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0098a0 tx=0x7fdf4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c4177556b052e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:23.013 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.263+0000 7fdf5959a700 1 -- 172.21.15.66:0/1753439116 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c004160 con 0x7fdf5c105320 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf5959a700 1 -- 172.21.15.66:0/1753439116 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf4c0042c0 con 0x7fdf5c105320 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf5959a700 1 -- 172.21.15.66:0/1753439116 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c011a00 con 0x7fdf5c105320 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf60f8f700 1 -- 172.21.15.66:0/1753439116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 msgr2=0x7fdf5c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf60f8f700 1 --2- 172.21.15.66:0/1753439116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c105730 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0098a0 tx=0x7fdf4c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf60f8f700 1 -- 172.21.15.66:0/1753439116 shutdown_connections 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf60f8f700 1 --2- 172.21.15.66:0/1753439116 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c105730 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:23.014 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.264+0000 7fdf60f8f700 1 -- 172.21.15.66:0/1753439116 >> 172.21.15.66:0/1753439116 conn(0x7fdf5c100970 msgr2=0x7fdf5c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.265+0000 7fdf60f8f700 1 -- 172.21.15.66:0/1753439116 shutdown_connections 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.265+0000 7fdf60f8f700 1 -- 172.21.15.66:0/1753439116 wait complete. 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.268+0000 7fdf60f8f700 1 Processor -- start 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.268+0000 7fdf60f8f700 1 -- start start 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.268+0000 7fdf60f8f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.268+0000 7fdf60f8f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c19b820 con 0x7fdf5c105320 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.269+0000 7fdf5a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:23.015 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.269+0000 7fdf5a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43808/0 (socket says 172.21.15.66:43808) 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.269+0000 7fdf5a59c700 1 -- 172.21.15.66:0/3995338383 learned_addr learned my addr 172.21.15.66:0/3995338383 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.269+0000 7fdf5a59c700 1 -- 172.21.15.66:0/3995338383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009550 con 0x7fdf5c105320 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.269+0000 7fdf5a59c700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c19b2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009130 tx=0x7fdf4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.270+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c003b70 con 0x7fdf5c105320 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.270+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdf4c003cd0 con 0x7fdf5c105320 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.270+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c19ba20 con 0x7fdf5c105320 2023-07-21T22:12:23.016 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.270+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf4c0197d0 con 0x7fdf5c105320 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.270+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c19be40 con 0x7fdf5c105320 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.272+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c061b40 con 0x7fdf5c105320 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.273+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fdf4c01a8d0 con 0x7fdf5c105320 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.273+0000 7fdf4b7fe700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fdf44038760 0x7fdf4403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.273+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdf4c04eb20 con 0x7fdf5c105320 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.273+0000 7fdf59d9b700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fdf44038760 0x7fdf4403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:23.017 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.274+0000 7fdf59d9b700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fdf44038760 0x7fdf4403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.278+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdf4c010970 con 0x7fdf5c105320 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.500+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdf5c19eca0 con 0x7fdf44038760 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.507+0000 7fdf4b7fe700 1 -- 172.21.15.66:0/3995338383 <== mgr.14118 v2:172.21.15.66:6800/3065709937 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdf5c19eca0 con 0x7fdf44038760 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fdf44038760 msgr2=0x7fdf4403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fdf44038760 0x7fdf4403ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 msgr2=0x7fdf5c19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:23.018 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c19b2e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009130 tx=0x7fdf4c011ad0 comp rx=0 tx=0).stop 2023-07-21T22:12:23.019 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 shutdown_connections 2023-07-21T22:12:23.019 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fdf44038760 0x7fdf4403ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:23.019 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 --2- 172.21.15.66:0/3995338383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf5c105320 0x7fdf5c19b2e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:23.019 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 >> 172.21.15.66:0/3995338383 conn(0x7fdf5c100970 msgr2=0x7fdf5c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:23.019 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 shutdown_connections 2023-07-21T22:12:23.019 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:22.510+0000 7fdf60f8f700 1 -- 172.21.15.66:0/3995338383 wait complete. 2023-07-21T22:12:23.692 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:23 smithi066 ceph-mon[126379]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:23.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:23 smithi066 ceph-mon[126379]: Saving service mgr spec with placement count:2 2023-07-21T22:12:23.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:23 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:23.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:23 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:23.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:23 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:24.311 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.571+0000 7f0653471700 1 Processor -- start 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.571+0000 7f0653471700 1 -- start start 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.571+0000 7f0653471700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.571+0000 7f0653471700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c07a840 con 0x7f064c07be00 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.572+0000 7f065120d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.572+0000 7f065120d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43814/0 (socket says 172.21.15.66:43814) 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.572+0000 7f065120d700 1 -- 172.21.15.66:0/2817010039 learned_addr learned my addr 172.21.15.66:0/2817010039 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:24.312 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.572+0000 7f065120d700 1 -- 172.21.15.66:0/2817010039 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c07a980 con 0x7f064c07be00 2023-07-21T22:12:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.573+0000 7f065120d700 1 --2- 172.21.15.66:0/2817010039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c07a300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06480098a0 tx=0x7f0648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751b2d1541eec915 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.573+0000 7f0643fff700 1 -- 172.21.15.66:0/2817010039 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0648004160 con 0x7f064c07be00 2023-07-21T22:12:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.573+0000 7f0643fff700 1 -- 172.21.15.66:0/2817010039 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06480042c0 con 0x7f064c07be00 2023-07-21T22:12:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.573+0000 7f0643fff700 1 -- 172.21.15.66:0/2817010039 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06480119b0 con 0x7f064c07be00 2023-07-21T22:12:24.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.573+0000 7f0653471700 1 -- 172.21.15.66:0/2817010039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 msgr2=0x7f064c07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:24.316 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.573+0000 7f0653471700 1 --2- 172.21.15.66:0/2817010039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c07a300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06480098a0 tx=0x7f0648009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.574+0000 7f0653471700 1 -- 172.21.15.66:0/2817010039 shutdown_connections 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.574+0000 7f0653471700 1 --2- 172.21.15.66:0/2817010039 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c07a300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.574+0000 7f0653471700 1 -- 172.21.15.66:0/2817010039 >> 172.21.15.66:0/2817010039 conn(0x7f064c100970 msgr2=0x7f064c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.574+0000 7f0653471700 1 -- 172.21.15.66:0/2817010039 shutdown_connections 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.574+0000 7f0653471700 1 -- 172.21.15.66:0/2817010039 wait complete. 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.575+0000 7f0653471700 1 Processor -- start 2023-07-21T22:12:24.317 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.576+0000 7f0653471700 1 -- start start 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.576+0000 7f0653471700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.576+0000 7f0653471700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c19b880 con 0x7f064c07be00 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.576+0000 7f065120d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.576+0000 7f065120d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43820/0 (socket says 172.21.15.66:43820) 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.576+0000 7f065120d700 1 -- 172.21.15.66:0/3889432809 learned_addr learned my addr 172.21.15.66:0/3889432809 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.577+0000 7f065120d700 1 -- 172.21.15.66:0/3889432809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0648009550 con 0x7f064c07be00 2023-07-21T22:12:24.318 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.577+0000 7f065120d700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c19b340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0648003e60 tx=0x7f0648011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.577+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0648003a60 con 0x7f064c07be00 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.577+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0648003bc0 con 0x7f064c07be00 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.577+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06480196c0 con 0x7f064c07be00 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.577+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c19ba80 con 0x7f064c07be00 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.578+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c19bea0 con 0x7f064c07be00 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.579+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f064801ab60 con 0x7f064c07be00 2023-07-21T22:12:24.319 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.579+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c061b40 con 0x7f064c07be00 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.579+0000 7f06427fc700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f0638038740 0x7f063803abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.579+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f064804c9e0 con 0x7f064c07be00 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.580+0000 7f0650a0c700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f0638038740 0x7f063803abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.580+0000 7f0650a0c700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f0638038740 0x7f063803abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f063c006fd0 tx=0x7f063c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.585+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0648010790 con 0x7f064c07be00 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.801+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f064c19e8e0 con 0x7f064c07be00 2023-07-21T22:12:24.320 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.807+0000 7f06427fc700 1 -- 172.21.15.66:0/3889432809 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0648018350 con 0x7f064c07be00 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.809+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f0638038740 msgr2=0x7f063803abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.809+0000 7f0653471700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f0638038740 0x7f063803abf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f063c006fd0 tx=0x7f063c006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.809+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 msgr2=0x7f064c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.809+0000 7f0653471700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c19b340 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0648003e60 tx=0x7f0648011a30 comp rx=0 tx=0).stop 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.810+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 shutdown_connections 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.810+0000 7f0653471700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f0638038740 0x7f063803abf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.321 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.810+0000 7f0653471700 1 --2- 172.21.15.66:0/3889432809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f064c07be00 0x7f064c19b340 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:24.322 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.810+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 >> 172.21.15.66:0/3889432809 conn(0x7f064c100970 msgr2=0x7f064c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:24.322 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.810+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 shutdown_connections 2023-07-21T22:12:24.322 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:23.810+0000 7f0653471700 1 -- 172.21.15.66:0/3889432809 wait complete. 2023-07-21T22:12:24.942 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:24 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3889432809' entity='client.admin' 2023-07-21T22:12:25.711 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.886+0000 7f12d30b4700 1 Processor -- start 2023-07-21T22:12:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.887+0000 7f12d30b4700 1 -- start start 2023-07-21T22:12:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.887+0000 7f12d30b4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc07a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.887+0000 7f12d30b4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc07a840 con 0x7f12cc07be00 2023-07-21T22:12:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.887+0000 7f12d0e50700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc07a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.887+0000 7f12d0e50700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc07a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43836/0 (socket says 172.21.15.66:43836) 2023-07-21T22:12:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.887+0000 7f12d0e50700 1 -- 172.21.15.66:0/1634108003 learned_addr learned my addr 172.21.15.66:0/1634108003 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.888+0000 7f12d0e50700 1 -- 172.21.15.66:0/1634108003 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc07a980 con 0x7f12cc07be00 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.888+0000 7f12d0e50700 1 --2- 172.21.15.66:0/1634108003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc07a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f12c00098a0 tx=0x7f12c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c5a99d3ef1628b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.888+0000 7f12cb7fe700 1 -- 172.21.15.66:0/1634108003 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c0004160 con 0x7f12cc07be00 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.888+0000 7f12cb7fe700 1 -- 172.21.15.66:0/1634108003 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12c00042c0 con 0x7f12cc07be00 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.888+0000 7f12cb7fe700 1 -- 172.21.15.66:0/1634108003 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c00119b0 con 0x7f12cc07be00 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 -- 172.21.15.66:0/1634108003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 msgr2=0x7f12cc07a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 --2- 172.21.15.66:0/1634108003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc07a300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f12c00098a0 tx=0x7f12c0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:25.713 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 -- 172.21.15.66:0/1634108003 shutdown_connections 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 --2- 172.21.15.66:0/1634108003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc07a300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 -- 172.21.15.66:0/1634108003 >> 172.21.15.66:0/1634108003 conn(0x7f12cc100970 msgr2=0x7f12cc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 -- 172.21.15.66:0/1634108003 shutdown_connections 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.889+0000 7f12d30b4700 1 -- 172.21.15.66:0/1634108003 wait complete. 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.890+0000 7f12d30b4700 1 Processor -- start 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.890+0000 7f12d30b4700 1 -- start start 2023-07-21T22:12:25.714 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d30b4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d30b4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc19b880 con 0x7f12cc07be00 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d0e50700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d0e50700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43850/0 (socket says 172.21.15.66:43850) 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d0e50700 1 -- 172.21.15.66:0/3858252405 learned_addr learned my addr 172.21.15.66:0/3858252405 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d0e50700 1 -- 172.21.15.66:0/3858252405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0009550 con 0x7f12cc07be00 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.891+0000 7f12d0e50700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc19b340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f12c0003e60 tx=0x7f12c0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.892+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c0003a60 con 0x7f12cc07be00 2023-07-21T22:12:25.715 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.892+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12c0003bc0 con 0x7f12cc07be00 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.892+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc19ba80 con 0x7f12cc07be00 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.892+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12c00196c0 con 0x7f12cc07be00 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.892+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc19bea0 con 0x7f12cc07be00 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.893+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f12c001ab60 con 0x7f12cc07be00 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.893+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc061b40 con 0x7f12cc07be00 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.893+0000 7f12c9ffb700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f12bc038740 0x7f12bc03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:25.716 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.893+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f12c004c9d0 con 0x7f12cc07be00 2023-07-21T22:12:25.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.894+0000 7f12cbfff700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f12bc038740 0x7f12bc03abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:25.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.894+0000 7f12cbfff700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f12bc038740 0x7f12bc03abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006fd0 tx=0x7f12b8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:25.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:24.898+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f12c0010790 con 0x7f12cc07be00 2023-07-21T22:12:25.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.131+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f12cc19e8e0 con 0x7f12cc07be00 2023-07-21T22:12:25.717 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.137+0000 7f12c9ffb700 1 -- 172.21.15.66:0/3858252405 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f12c0018350 con 0x7f12cc07be00 2023-07-21T22:12:25.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.139+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f12bc038740 msgr2=0x7f12bc03abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:25.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.139+0000 7f12d30b4700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f12bc038740 0x7f12bc03abf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006fd0 tx=0x7f12b8006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:25.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 msgr2=0x7f12cc19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:25.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc19b340 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f12c0003e60 tx=0x7f12c0011a30 comp rx=0 tx=0).stop 2023-07-21T22:12:25.718 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 shutdown_connections 2023-07-21T22:12:25.719 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f12bc038740 0x7f12bc03abf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:25.719 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 --2- 172.21.15.66:0/3858252405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f12cc07be00 0x7f12cc19b340 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:25.719 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 >> 172.21.15.66:0/3858252405 conn(0x7f12cc100970 msgr2=0x7f12cc1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:25.719 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 shutdown_connections 2023-07-21T22:12:25.719 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:25.140+0000 7f12d30b4700 1 -- 172.21.15.66:0/3858252405 wait complete. 2023-07-21T22:12:25.719 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling the dashboard module... 2023-07-21T22:12:26.351 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:26 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3858252405' entity='client.admin' 2023-07-21T22:12:26.351 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:26 smithi066 ceph-mon[126379]: from='mgr.14118 172.21.15.66:0/3573265789' entity='mgr.x' 2023-07-21T22:12:27.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1539089396' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-21T22:12:27.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setuser ceph since I am not root 2023-07-21T22:12:27.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setgroup ceph since I am not root 2023-07-21T22:12:27.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.265+0000 7fc27baf8700 1 Processor -- start 2023-07-21T22:12:27.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.266+0000 7fc27baf8700 1 -- start start 2023-07-21T22:12:27.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.266+0000 7fc27baf8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:27.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.266+0000 7fc27baf8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27407a860 con 0x7fc27407be20 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.266+0000 7fc279894700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.266+0000 7fc279894700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43858/0 (socket says 172.21.15.66:43858) 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.266+0000 7fc279894700 1 -- 172.21.15.66:0/159779177 learned_addr learned my addr 172.21.15.66:0/159779177 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.267+0000 7fc279894700 1 -- 172.21.15.66:0/159779177 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27407a9a0 con 0x7fc27407be20 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.267+0000 7fc279894700 1 --2- 172.21.15.66:0/159779177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27407a320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc2700098a0 tx=0x7fc270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3531f5ac7129033a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.267+0000 7fc278892700 1 -- 172.21.15.66:0/159779177 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc270004160 con 0x7fc27407be20 2023-07-21T22:12:27.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.267+0000 7fc278892700 1 -- 172.21.15.66:0/159779177 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2700042c0 con 0x7fc27407be20 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.267+0000 7fc278892700 1 -- 172.21.15.66:0/159779177 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2700119b0 con 0x7fc27407be20 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 -- 172.21.15.66:0/159779177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 msgr2=0x7fc27407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 --2- 172.21.15.66:0/159779177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27407a320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc2700098a0 tx=0x7fc270009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 -- 172.21.15.66:0/159779177 shutdown_connections 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 --2- 172.21.15.66:0/159779177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27407a320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 -- 172.21.15.66:0/159779177 >> 172.21.15.66:0/159779177 conn(0x7fc274100970 msgr2=0x7fc274102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 -- 172.21.15.66:0/159779177 shutdown_connections 2023-07-21T22:12:27.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.268+0000 7fc27baf8700 1 -- 172.21.15.66:0/159779177 wait complete. 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.269+0000 7fc27baf8700 1 Processor -- start 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.269+0000 7fc27baf8700 1 -- start start 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc27baf8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc27baf8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27419b890 con 0x7fc27407be20 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc279894700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc279894700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27419b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43866/0 (socket says 172.21.15.66:43866) 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc279894700 1 -- 172.21.15.66:0/1539089396 learned_addr learned my addr 172.21.15.66:0/1539089396 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:27.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc279894700 1 -- 172.21.15.66:0/1539089396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009550 con 0x7fc27407be20 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.270+0000 7fc279894700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27419b350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc270000c00 tx=0x7fc270011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.271+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc270003aa0 con 0x7fc27407be20 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.271+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc270003c00 con 0x7fc27407be20 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.271+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27419ba90 con 0x7fc27407be20 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.271+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc270024800 con 0x7fc27407be20 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.271+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27419beb0 con 0x7fc27407be20 2023-07-21T22:12:27.599 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.272+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc27001ab60 con 0x7fc27407be20 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.272+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274195330 con 0x7fc27407be20 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.272+0000 7fc26affd700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fc260038760 0x7fc26003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.272+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc27004c800 con 0x7fc27407be20 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.273+0000 7fc279093700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fc260038760 0x7fc26003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.273+0000 7fc279093700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fc260038760 0x7fc26003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc264006fd0 tx=0x7fc264006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.277+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc270018530 con 0x7fc27407be20 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:26.547+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc27402cce0 con 0x7fc27407be20 2023-07-21T22:12:27.600 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.140+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc27002d720 con 0x7fc27407be20 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.142+0000 7fc26affd700 1 -- 172.21.15.66:0/1539089396 <== mon.0 v2:172.21.15.66:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc27004c150 con 0x7fc27407be20 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fc260038760 msgr2=0x7fc26003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fc260038760 0x7fc26003ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc264006fd0 tx=0x7fc264006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 msgr2=0x7fc27419b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27419b350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc270000c00 tx=0x7fc270011a30 comp rx=0 tx=0).stop 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 shutdown_connections 2023-07-21T22:12:27.601 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7fc260038760 0x7fc26003ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:27.602 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 --2- 172.21.15.66:0/1539089396 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc27407be20 0x7fc27419b350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:27.602 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 >> 172.21.15.66:0/1539089396 conn(0x7fc274100970 msgr2=0x7fc274102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:27.602 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 shutdown_connections 2023-07-21T22:12:27.602 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:27.144+0000 7fc27baf8700 1 -- 172.21.15.66:0/1539089396 wait complete. 2023-07-21T22:12:27.866 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:27.449+0000 7f7c665ad200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:12:27.866 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:27.562+0000 7f7c665ad200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:12:27.867 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:27.698+0000 7f7c665ad200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:12:28.182 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:27.827+0000 7f7c665ad200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:12:28.182 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:28.072+0000 7f7c665ad200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:12:28.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1539089396' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-21T22:12:28.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-mon[126379]: mgrmap e9: x(active, since 17s) 2023-07-21T22:12:28.847 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:28.492+0000 7f7c665ad200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:12:28.848 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:28.632+0000 7f7c665ad200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:12:28.877 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:12:28.878 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-21T22:12:28.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T22:12:28.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T22:12:28.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T22:12:28.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:12:28.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.097+0000 7f47e2c20700 1 Processor -- start 2023-07-21T22:12:28.879 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.097+0000 7f47e2c20700 1 -- start start 2023-07-21T22:12:28.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.097+0000 7f47e2c20700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:28.880 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.097+0000 7f47e2c20700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc105c70 con 0x7f47dc105320 2023-07-21T22:12:28.883 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.098+0000 7f47e09bc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:28.883 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.098+0000 7f47e09bc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43902/0 (socket says 172.21.15.66:43902) 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.098+0000 7f47e09bc700 1 -- 172.21.15.66:0/2248360465 learned_addr learned my addr 172.21.15.66:0/2248360465 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.098+0000 7f47e09bc700 1 -- 172.21.15.66:0/2248360465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc105db0 con 0x7f47dc105320 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.099+0000 7f47e09bc700 1 --2- 172.21.15.66:0/2248360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc105730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f47d00098a0 tx=0x7f47d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267bbfb1f2de2a3a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.099+0000 7f47db7fe700 1 -- 172.21.15.66:0/2248360465 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d0004160 con 0x7f47dc105320 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.099+0000 7f47db7fe700 1 -- 172.21.15.66:0/2248360465 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47d00042c0 con 0x7f47dc105320 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.099+0000 7f47db7fe700 1 -- 172.21.15.66:0/2248360465 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d00119b0 con 0x7f47dc105320 2023-07-21T22:12:28.884 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 -- 172.21.15.66:0/2248360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 msgr2=0x7f47dc105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 --2- 172.21.15.66:0/2248360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc105730 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f47d00098a0 tx=0x7f47d0009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 -- 172.21.15.66:0/2248360465 shutdown_connections 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 --2- 172.21.15.66:0/2248360465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc105730 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 -- 172.21.15.66:0/2248360465 >> 172.21.15.66:0/2248360465 conn(0x7f47dc100970 msgr2=0x7f47dc102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 -- 172.21.15.66:0/2248360465 shutdown_connections 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.100+0000 7f47e2c20700 1 -- 172.21.15.66:0/2248360465 wait complete. 2023-07-21T22:12:28.885 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.101+0000 7f47e2c20700 1 Processor -- start 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.101+0000 7f47e2c20700 1 -- start start 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.101+0000 7f47e2c20700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.101+0000 7f47e2c20700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc19b840 con 0x7f47dc105320 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.102+0000 7f47e09bc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.102+0000 7f47e09bc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43912/0 (socket says 172.21.15.66:43912) 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.102+0000 7f47e09bc700 1 -- 172.21.15.66:0/1243942173 learned_addr learned my addr 172.21.15.66:0/1243942173 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.102+0000 7f47e09bc700 1 -- 172.21.15.66:0/1243942173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d0009550 con 0x7f47dc105320 2023-07-21T22:12:28.886 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.102+0000 7f47e09bc700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc19b300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47d0003f00 tx=0x7f47d0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.102+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d0003b20 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.103+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc19ba40 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.103+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47d0003c80 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.103+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f47d0019770 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.103+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc19be60 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.104+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f47d001a430 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.104+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc1097b0 con 0x7f47dc105320 2023-07-21T22:12:28.887 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.104+0000 7f47d9ffb700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 0x7f47cc03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.104+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f47d004c9a0 con 0x7f47dc105320 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.104+0000 7f47dbfff700 1 -- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 msgr2=0x7f47cc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.104+0000 7f47dbfff700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 0x7f47cc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.110+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f47d002d9a0 con 0x7f47dc105320 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.305+0000 7f47dbfff700 1 -- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 msgr2=0x7f47cc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.305+0000 7f47dbfff700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 0x7f47cc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:12:28.888 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.377+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f47dc19eee0 con 0x7f47dc105320 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.377+0000 7f47d9ffb700 1 -- 172.21.15.66:0/1243942173 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f47d002d350 con 0x7f47dc105320 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 msgr2=0x7f47cc03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 0x7f47cc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 msgr2=0x7f47dc19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc19b300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f47d0003f00 tx=0x7f47d0011a30 comp rx=0 tx=0).stop 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 shutdown_connections 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f47cc038760 0x7f47cc03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:28.889 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 --2- 172.21.15.66:0/1243942173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47dc105320 0x7f47dc19b300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:28.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 >> 172.21.15.66:0/1243942173 conn(0x7f47dc100970 msgr2=0x7f47dc102040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:28.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 shutdown_connections 2023-07-21T22:12:28.890 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:28.380+0000 7f47e2c20700 1 -- 172.21.15.66:0/1243942173 wait complete. 2023-07-21T22:12:28.890 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2023-07-21T22:12:28.890 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 9... 2023-07-21T22:12:29.176 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:28.805+0000 7f7c665ad200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:12:29.176 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:28.943+0000 7f7c665ad200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:12:29.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:29 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1243942173' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T22:12:29.943 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:29 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:29.531+0000 7f7c665ad200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:12:30.692 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:30 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:30.317+0000 7f7c665ad200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:12:30.693 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:30 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:30.534+0000 7f7c665ad200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:12:31.935 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:31 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:31.531+0000 7f7c665ad200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:12:31.935 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:31 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:31.659+0000 7f7c665ad200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:12:32.192 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:31 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:31.892+0000 7f7c665ad200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:12:32.600 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:32 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:32.269+0000 7f7c665ad200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:12:32.942 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:32 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:32.558+0000 7f7c665ad200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:12:32.943 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:32 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:32.761+0000 7f7c665ad200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:12:33.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:33 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:33.120+0000 7f7c665ad200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:12:34.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:34.099+0000 7f7c665ad200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:12:34.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:34.282+0000 7f7c665ad200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:12:34.942 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:34.538+0000 7f7c665ad200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:12:34.943 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:34.692+0000 7f7c665ad200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:12:35.442 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:34.979+0000 7f7c665ad200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:12:36.443 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:12:36.095+0000 7f7c665ad200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:12:36.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: Active manager daemon x restarted 2023-07-21T22:12:36.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: Activating manager daemon x 2023-07-21T22:12:36.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: osdmap e3: 0 total, 0 up, 0 in 2023-07-21T22:12:36.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: mgrmap e10: x(active, starting, since 0.00641172s) 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: Manager daemon x is now available 2023-07-21T22:12:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:36.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:37.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:12:37.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:12:37.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:37 smithi066 ceph-mon[126379]: mgrmap e11: x(active, since 1.00761s) 2023-07-21T22:12:37.582 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2023-07-21T22:12:37.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-21T22:12:37.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T22:12:37.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2023-07-21T22:12:37.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.414+0000 7f5c570c5700 1 Processor -- start 2023-07-21T22:12:37.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.414+0000 7f5c570c5700 1 -- start start 2023-07-21T22:12:37.585 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.415+0000 7f5c570c5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.415+0000 7f5c570c5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5007a820 con 0x7f5c5007bde0 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.415+0000 7f5c54e61700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.415+0000 7f5c54e61700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43918/0 (socket says 172.21.15.66:43918) 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.415+0000 7f5c54e61700 1 -- 172.21.15.66:0/476312933 learned_addr learned my addr 172.21.15.66:0/476312933 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.416+0000 7f5c54e61700 1 -- 172.21.15.66:0/476312933 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5007a960 con 0x7f5c5007bde0 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.416+0000 7f5c54e61700 1 --2- 172.21.15.66:0/476312933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5007a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9f3b56eb66025e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:37.586 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.416+0000 7f5c4f7fe700 1 -- 172.21.15.66:0/476312933 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c44004160 con 0x7f5c5007bde0 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.416+0000 7f5c4f7fe700 1 -- 172.21.15.66:0/476312933 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c440042c0 con 0x7f5c5007bde0 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.416+0000 7f5c4f7fe700 1 -- 172.21.15.66:0/476312933 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c440119b0 con 0x7f5c5007bde0 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 -- 172.21.15.66:0/476312933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 msgr2=0x7f5c5007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 --2- 172.21.15.66:0/476312933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5007a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c44009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 -- 172.21.15.66:0/476312933 shutdown_connections 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 --2- 172.21.15.66:0/476312933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5007a2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 -- 172.21.15.66:0/476312933 >> 172.21.15.66:0/476312933 conn(0x7f5c50100970 msgr2=0x7f5c50102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:37.587 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 -- 172.21.15.66:0/476312933 shutdown_connections 2023-07-21T22:12:37.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.417+0000 7f5c570c5700 1 -- 172.21.15.66:0/476312933 wait complete. 2023-07-21T22:12:37.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.418+0000 7f5c570c5700 1 Processor -- start 2023-07-21T22:12:37.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.418+0000 7f5c570c5700 1 -- start start 2023-07-21T22:12:37.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c570c5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:37.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c570c5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5019b860 con 0x7f5c5007bde0 2023-07-21T22:12:37.588 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c54e61700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:37.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c54e61700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43922/0 (socket says 172.21.15.66:43922) 2023-07-21T22:12:37.589 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c54e61700 1 -- 172.21.15.66:0/4273164753 learned_addr learned my addr 172.21.15.66:0/4273164753 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:37.592 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c54e61700 1 -- 172.21.15.66:0/4273164753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009550 con 0x7f5c5007bde0 2023-07-21T22:12:37.592 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.419+0000 7f5c54e61700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5019b320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006af0 tx=0x7f5c44011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.420+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c44003a40 con 0x7f5c5007bde0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.420+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c44003ba0 con 0x7f5c5007bde0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.420+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5019ba60 con 0x7f5c5007bde0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.420+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c44019690 con 0x7f5c5007bde0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.420+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5019be80 con 0x7f5c5007bde0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.421+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f5c44019900 con 0x7f5c5007bde0 2023-07-21T22:12:37.593 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.421+0000 7f5c4dffb700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.421+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 --> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c4003b320 con 0x7f5c40038760 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.421+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5c4404c640 con 0x7f5c5007bde0 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.421+0000 7f5c4ffff700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.421+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.622+0000 7f5c4ffff700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:29.622+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:12:37.594 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:30.023+0000 7f5c4ffff700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:30.023+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:30.824+0000 7f5c4ffff700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:30.824+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:32.426+0000 7f5c4ffff700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:32.426+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:35.629+0000 7f5c4ffff700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/3065709937 2023-07-21T22:12:37.595 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:35.629+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:36.102+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f5c44021d60 con 0x7f5c5007bde0 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:36.102+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 msgr2=0x7f5c4003ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:36.102+0000 7f5c4dffb700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.102+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f5c4402d7b0 con 0x7f5c5007bde0 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.102+0000 7f5c4dffb700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c4003bff0 0x7f5c4003e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.102+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5c4003b320 con 0x7f5c4003bff0 2023-07-21T22:12:37.596 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.103+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c4003bff0 0x7f5c4003e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.104+0000 7f5c4ffff700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c4003bff0 0x7f5c4003e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c003a10 tx=0x7f5c3c009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.104+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f5c4003b320 con 0x7f5c4003bff0 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5c5019c660 con 0x7f5c4003bff0 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c4dffb700 1 -- 172.21.15.66:0/4273164753 <== mgr.14150 v2:172.21.15.66:6800/1262827329 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f5c5019c660 con 0x7f5c4003bff0 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c4003bff0 msgr2=0x7f5c4003e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c4003bff0 0x7f5c4003e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c003a10 tx=0x7f5c3c009270 comp rx=0 tx=0).stop 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 msgr2=0x7f5c5019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:37.597 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5019b320 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006af0 tx=0x7f5c44011a10 comp rx=0 tx=0).stop 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 shutdown_connections 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c4003bff0 0x7f5c4003e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c5007bde0 0x7f5c5019b320 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 --2- 172.21.15.66:0/4273164753 >> [v2:172.21.15.66:6800/3065709937,v1:172.21.15.66:6801/3065709937] conn(0x7f5c40038760 0x7f5c4003ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.112+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 >> 172.21.15.66:0/4273164753 conn(0x7f5c50100970 msgr2=0x7f5c50102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.113+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 shutdown_connections 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:37.113+0000 7f5c570c5700 1 -- 172.21.15.66:0/4273164753 wait complete. 2023-07-21T22:12:37.598 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 9 is available 2023-07-21T22:12:37.599 INFO:teuthology.orchestra.run.smithi066.stdout:Generating a dashboard self-signed certificate... 2023-07-21T22:12:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:38 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:36] ENGINE Bus STARTING 2023-07-21T22:12:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:38 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:36] ENGINE Serving on https://172.21.15.66:7150 2023-07-21T22:12:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:38 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:36] ENGINE Serving on http://172.21.15.66:8765 2023-07-21T22:12:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:38 smithi066 ceph-mon[126379]: [21/Jul/2023:22:12:36] ENGINE Bus STARTED 2023-07-21T22:12:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:38 smithi066 ceph-mon[126379]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T22:12:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:38 smithi066 ceph-mon[126379]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T22:12:38.977 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-21T22:12:38.977 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.178+0000 7fd8bcf98700 1 Processor -- start 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.178+0000 7fd8bcf98700 1 -- start start 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.179+0000 7fd8bcf98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.179+0000 7fd8bcf98700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b807a860 con 0x7fd8b807be20 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.179+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.179+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38784/0 (socket says 172.21.15.66:38784) 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.179+0000 7fd8b659c700 1 -- 172.21.15.66:0/3449175108 learned_addr learned my addr 172.21.15.66:0/3449175108 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:38.978 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.180+0000 7fd8b659c700 1 -- 172.21.15.66:0/3449175108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b807a9a0 con 0x7fd8b807be20 2023-07-21T22:12:38.979 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.180+0000 7fd8b659c700 1 --2- 172.21.15.66:0/3449175108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b807a320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098a0 tx=0x7fd8a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce28c1f2730a8b8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:38.979 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.180+0000 7fd8b559a700 1 -- 172.21.15.66:0/3449175108 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a8004160 con 0x7fd8b807be20 2023-07-21T22:12:38.979 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.180+0000 7fd8b559a700 1 -- 172.21.15.66:0/3449175108 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8a80042c0 con 0x7fd8b807be20 2023-07-21T22:12:38.979 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.180+0000 7fd8b559a700 1 -- 172.21.15.66:0/3449175108 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a80119b0 con 0x7fd8b807be20 2023-07-21T22:12:38.979 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 -- 172.21.15.66:0/3449175108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 msgr2=0x7fd8b807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:38.979 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 --2- 172.21.15.66:0/3449175108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b807a320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098a0 tx=0x7fd8a8009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 -- 172.21.15.66:0/3449175108 shutdown_connections 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 --2- 172.21.15.66:0/3449175108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b807a320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 -- 172.21.15.66:0/3449175108 >> 172.21.15.66:0/3449175108 conn(0x7fd8b8100970 msgr2=0x7fd8b8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 -- 172.21.15.66:0/3449175108 shutdown_connections 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.181+0000 7fd8bcf98700 1 -- 172.21.15.66:0/3449175108 wait complete. 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.182+0000 7fd8bcf98700 1 Processor -- start 2023-07-21T22:12:38.980 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.182+0000 7fd8bcf98700 1 -- start start 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8bcf98700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8bcf98700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b819b890 con 0x7fd8b807be20 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8b659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b819b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38798/0 (socket says 172.21.15.66:38798) 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8b659c700 1 -- 172.21.15.66:0/1642840546 learned_addr learned my addr 172.21.15.66:0/1642840546 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8b659c700 1 -- 172.21.15.66:0/1642840546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8009550 con 0x7fd8b807be20 2023-07-21T22:12:38.981 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.183+0000 7fd8b659c700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b819b350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8000c00 tx=0x7fd8a8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.184+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a8003aa0 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.184+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8a8003c00 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.184+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b819ba90 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.184+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8a8024800 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.184+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b819beb0 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.185+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b8195330 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.185+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fd8a801ab60 con 0x7fd8b807be20 2023-07-21T22:12:38.982 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.186+0000 7fd8a77fe700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8a0038630 0x7fd8a003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.186+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd8a804c760 con 0x7fd8b807be20 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.186+0000 7fd8b5d9b700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8a0038630 0x7fd8a003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.187+0000 7fd8b5d9b700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8a0038630 0x7fd8a003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac006fd0 tx=0x7fd8ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.191+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd8a8018530 con 0x7fd8b807be20 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.412+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b8060950 con 0x7fd8a0038630 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.549+0000 7fd8a77fe700 1 -- 172.21.15.66:0/1642840546 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd8b8060950 con 0x7fd8a0038630 2023-07-21T22:12:38.983 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8a0038630 msgr2=0x7fd8a003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8a0038630 0x7fd8a003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac006fd0 tx=0x7fd8ac006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 msgr2=0x7fd8b819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b819b350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8000c00 tx=0x7fd8a8011a30 comp rx=0 tx=0).stop 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 shutdown_connections 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8a0038630 0x7fd8a003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 --2- 172.21.15.66:0/1642840546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8b807be20 0x7fd8b819b350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:38.984 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 >> 172.21.15.66:0/1642840546 conn(0x7fd8b8100970 msgr2=0x7fd8b8102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:38.985 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 shutdown_connections 2023-07-21T22:12:38.985 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:38.552+0000 7fd8bcf98700 1 -- 172.21.15.66:0/1642840546 wait complete. 2023-07-21T22:12:38.985 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial admin user... 2023-07-21T22:12:39.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:39 smithi066 ceph-mon[126379]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:39.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:39.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:39.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:39.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:39 smithi066 ceph-mon[126379]: mgrmap e12: x(active, since 2s) 2023-07-21T22:12:40.365 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Pin.1gwEK6FzNQWBwvyhV./aMwZSBxdSv5/gOu/4I6oxOtP73pZa2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689977559, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-21T22:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.477+0000 7f8a6e48b700 1 Processor -- start 2023-07-21T22:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.477+0000 7f8a6e48b700 1 -- start start 2023-07-21T22:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.477+0000 7f8a6e48b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 0x7f8a68107ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.477+0000 7f8a6e48b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68108400 con 0x7f8a68079d20 2023-07-21T22:12:40.366 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.478+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 0x7f8a68107ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.478+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 0x7f8a68107ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38810/0 (socket says 172.21.15.66:38810) 2023-07-21T22:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.478+0000 7f8a67fff700 1 -- 172.21.15.66:0/713275421 learned_addr learned my addr 172.21.15.66:0/713275421 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:40.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.478+0000 7f8a67fff700 1 -- 172.21.15.66:0/713275421 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68108540 con 0x7f8a68079d20 2023-07-21T22:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.478+0000 7f8a67fff700 1 --2- 172.21.15.66:0/713275421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 0x7f8a68107ec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a580098a0 tx=0x7f8a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dad62d58dda545 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.478+0000 7f8a66ffd700 1 -- 172.21.15.66:0/713275421 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a58004160 con 0x7f8a68079d20 2023-07-21T22:12:40.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a66ffd700 1 -- 172.21.15.66:0/713275421 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a580042c0 con 0x7f8a68079d20 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a66ffd700 1 -- 172.21.15.66:0/713275421 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a580119b0 con 0x7f8a68079d20 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 -- 172.21.15.66:0/713275421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 msgr2=0x7f8a68107ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 --2- 172.21.15.66:0/713275421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 0x7f8a68107ec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a580098a0 tx=0x7f8a58009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 -- 172.21.15.66:0/713275421 shutdown_connections 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 --2- 172.21.15.66:0/713275421 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a68079d20 0x7f8a68107ec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 -- 172.21.15.66:0/713275421 >> 172.21.15.66:0/713275421 conn(0x7f8a680ff890 msgr2=0x7f8a68101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 -- 172.21.15.66:0/713275421 shutdown_connections 2023-07-21T22:12:40.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.479+0000 7f8a6e48b700 1 -- 172.21.15.66:0/713275421 wait complete. 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.480+0000 7f8a6e48b700 1 Processor -- start 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.481+0000 7f8a6e48b700 1 -- start start 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.481+0000 7f8a6e48b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 0x7f8a68103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.481+0000 7f8a6e48b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68105be0 con 0x7f8a681057d0 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.481+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 0x7f8a68103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.481+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 0x7f8a68103e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38820/0 (socket says 172.21.15.66:38820) 2023-07-21T22:12:40.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.481+0000 7f8a67fff700 1 -- 172.21.15.66:0/3693732995 learned_addr learned my addr 172.21.15.66:0/3693732995 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a67fff700 1 -- 172.21.15.66:0/3693732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58009550 con 0x7f8a681057d0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a67fff700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 0x7f8a68103e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006af0 tx=0x7f8a58011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a58003a40 con 0x7f8a681057d0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a681043a0 con 0x7f8a681057d0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a58003ba0 con 0x7f8a681057d0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a58019690 con 0x7f8a681057d0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.482+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a681047c0 con 0x7f8a681057d0 2023-07-21T22:12:40.373 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.483+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f8a58019950 con 0x7f8a681057d0 2023-07-21T22:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.484+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68193110 con 0x7f8a681057d0 2023-07-21T22:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.484+0000 7f8a657fa700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8a50038710 0x7f8a5003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.484+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a5804cda0 con 0x7f8a681057d0 2023-07-21T22:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.484+0000 7f8a677fe700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8a50038710 0x7f8a5003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:40.374 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.485+0000 7f8a677fe700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8a50038710 0x7f8a5003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006fd0 tx=0x7f8a5c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:40.375 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.493+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8a5802dcd0 con 0x7f8a681057d0 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.724+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8a68061960 con 0x7f8a50038710 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.975+0000 7f8a657fa700 1 -- 172.21.15.66:0/3693732995 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8a68061960 con 0x7f8a50038710 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8a50038710 msgr2=0x7f8a5003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8a50038710 0x7f8a5003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c006fd0 tx=0x7f8a5c006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 msgr2=0x7f8a68103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 0x7f8a68103e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006af0 tx=0x7f8a58011a10 comp rx=0 tx=0).stop 2023-07-21T22:12:40.376 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 shutdown_connections 2023-07-21T22:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8a50038710 0x7f8a5003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.977+0000 7f8a6e48b700 1 --2- 172.21.15.66:0/3693732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8a681057d0 0x7f8a68103e60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:40.377 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.978+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 >> 172.21.15.66:0/3693732995 conn(0x7f8a680ff890 msgr2=0x7f8a68100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.978+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 shutdown_connections 2023-07-21T22:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:39.978+0000 7f8a6e48b700 1 -- 172.21.15.66:0/3693732995 wait complete. 2023-07-21T22:12:40.378 INFO:teuthology.orchestra.run.smithi066.stdout:Fetching dashboard port number... 2023-07-21T22:12:41.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:40 smithi066 ceph-mon[126379]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:41.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:41.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:41.749 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 8443 2023-07-21T22:12:41.754 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.900+0000 7fd43291d700 1 Processor -- start 2023-07-21T22:12:41.755 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.901+0000 7fd43291d700 1 -- start start 2023-07-21T22:12:41.755 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.901+0000 7fd43291d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:41.756 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.901+0000 7fd43291d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c105c70 con 0x7fd42c105320 2023-07-21T22:12:41.756 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.902+0000 7fd42bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:41.756 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.902+0000 7fd42bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38832/0 (socket says 172.21.15.66:38832) 2023-07-21T22:12:41.756 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.902+0000 7fd42bfff700 1 -- 172.21.15.66:0/727171146 learned_addr learned my addr 172.21.15.66:0/727171146 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:41.757 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.902+0000 7fd42bfff700 1 -- 172.21.15.66:0/727171146 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c105db0 con 0x7fd42c105320 2023-07-21T22:12:41.757 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.902+0000 7fd42bfff700 1 --2- 172.21.15.66:0/727171146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c105730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0098a0 tx=0x7fd41c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c375fd7be49ff4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:41.757 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd42affd700 1 -- 172.21.15.66:0/727171146 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd41c004160 con 0x7fd42c105320 2023-07-21T22:12:41.758 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd42affd700 1 -- 172.21.15.66:0/727171146 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41c0042c0 con 0x7fd42c105320 2023-07-21T22:12:41.758 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd42affd700 1 -- 172.21.15.66:0/727171146 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd41c0119b0 con 0x7fd42c105320 2023-07-21T22:12:41.758 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 -- 172.21.15.66:0/727171146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 msgr2=0x7fd42c105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:41.758 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 --2- 172.21.15.66:0/727171146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c105730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0098a0 tx=0x7fd41c009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:41.759 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 -- 172.21.15.66:0/727171146 shutdown_connections 2023-07-21T22:12:41.759 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 --2- 172.21.15.66:0/727171146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c105730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:41.759 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 -- 172.21.15.66:0/727171146 >> 172.21.15.66:0/727171146 conn(0x7fd42c100970 msgr2=0x7fd42c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:41.760 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 -- 172.21.15.66:0/727171146 shutdown_connections 2023-07-21T22:12:41.760 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.903+0000 7fd43291d700 1 -- 172.21.15.66:0/727171146 wait complete. 2023-07-21T22:12:41.760 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.905+0000 7fd43291d700 1 Processor -- start 2023-07-21T22:12:41.760 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.905+0000 7fd43291d700 1 -- start start 2023-07-21T22:12:41.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.905+0000 7fd43291d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:41.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.905+0000 7fd43291d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c07c1b0 con 0x7fd42c105320 2023-07-21T22:12:41.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.906+0000 7fd42bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:41.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.906+0000 7fd42bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c07bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:38848/0 (socket says 172.21.15.66:38848) 2023-07-21T22:12:41.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.906+0000 7fd42bfff700 1 -- 172.21.15.66:0/1105367163 learned_addr learned my addr 172.21.15.66:0/1105367163 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:41.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.906+0000 7fd42bfff700 1 -- 172.21.15.66:0/1105367163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c009550 con 0x7fd42c105320 2023-07-21T22:12:41.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.906+0000 7fd42bfff700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c07bc70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd41c003f00 tx=0x7fd41c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:41.763 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.907+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd41c003b20 con 0x7fd42c105320 2023-07-21T22:12:41.763 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.907+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd41c003c80 con 0x7fd42c105320 2023-07-21T22:12:41.763 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.907+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd41c019770 con 0x7fd42c105320 2023-07-21T22:12:41.763 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.907+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42c07a2c0 con 0x7fd42c105320 2023-07-21T22:12:41.764 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.907+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd42c07a6e0 con 0x7fd42c105320 2023-07-21T22:12:41.764 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.908+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42c1097b0 con 0x7fd42c105320 2023-07-21T22:12:41.764 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.908+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fd41c01ab60 con 0x7fd42c105320 2023-07-21T22:12:41.765 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.909+0000 7fd4297fa700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd414038800 0x7fd41403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:41.765 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.909+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd41c04cb70 con 0x7fd42c105320 2023-07-21T22:12:41.765 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.910+0000 7fd42b7fe700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd414038800 0x7fd41403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:41.765 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.910+0000 7fd42b7fe700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd414038800 0x7fd41403acb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd420006fd0 tx=0x7fd420006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:41.766 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:40.914+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd41c04ee60 con 0x7fd42c105320 2023-07-21T22:12:41.766 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.124+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fd42c07ade0 con 0x7fd42c105320 2023-07-21T22:12:41.767 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.125+0000 7fd4297fa700 1 -- 172.21.15.66:0/1105367163 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fd41c02d7c0 con 0x7fd42c105320 2023-07-21T22:12:41.767 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd414038800 msgr2=0x7fd41403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:41.767 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd414038800 0x7fd41403acb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd420006fd0 tx=0x7fd420006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:41.767 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 msgr2=0x7fd42c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:41.768 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c07bc70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd41c003f00 tx=0x7fd41c011a30 comp rx=0 tx=0).stop 2023-07-21T22:12:41.768 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 shutdown_connections 2023-07-21T22:12:41.768 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd414038800 0x7fd41403acb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:41.768 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 --2- 172.21.15.66:0/1105367163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd42c105320 0x7fd42c07bc70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:41.769 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.127+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 >> 172.21.15.66:0/1105367163 conn(0x7fd42c100970 msgr2=0x7fd42c1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:41.769 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.128+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 shutdown_connections 2023-07-21T22:12:41.769 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:41.128+0000 7fd43291d700 1 -- 172.21.15.66:0/1105367163 wait complete. 2023-07-21T22:12:41.774 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T22:12:41.774 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2023-07-21T22:12:41.793 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T22:12:41.793 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2023-07-21T22:12:41.793 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2023-07-21T22:12:41.793 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-21T22:12:41.796 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph Dashboard is now available at: 2023-07-21T22:12:41.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:41.797 INFO:teuthology.orchestra.run.smithi066.stdout: URL: https://smithi066.front.sepia.ceph.com:8443/ 2023-07-21T22:12:41.797 INFO:teuthology.orchestra.run.smithi066.stdout: User: admin 2023-07-21T22:12:41.797 INFO:teuthology.orchestra.run.smithi066.stdout: Password: fzrncfoa2l 2023-07-21T22:12:41.797 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:41.797 INFO:teuthology.orchestra.run.smithi066.stdout:Saving cluster configuration to /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config directory 2023-07-21T22:12:41.798 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling autotune for osd_memory_target 2023-07-21T22:12:42.102 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:41 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1105367163' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-21T22:12:43.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.377+0000 7f7257165700 1 Processor -- start 2023-07-21T22:12:43.083 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.377+0000 7f7257165700 1 -- start start 2023-07-21T22:12:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.377+0000 7f7257165700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f7250105730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.377+0000 7f7257165700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7250105c70 con 0x7f7250105320 2023-07-21T22:12:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.378+0000 7f7254f01700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f7250105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.378+0000 7f7254f01700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f7250105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33010/0 (socket says 172.21.15.66:33010) 2023-07-21T22:12:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.378+0000 7f7254f01700 1 -- 172.21.15.66:0/3822788585 learned_addr learned my addr 172.21.15.66:0/3822788585 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:43.084 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.378+0000 7f7254f01700 1 -- 172.21.15.66:0/3822788585 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250105db0 con 0x7f7250105320 2023-07-21T22:12:43.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.378+0000 7f7254f01700 1 --2- 172.21.15.66:0/3822788585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f7250105730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72440098a0 tx=0x7f7244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e57c87c512ae12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:43.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f724f7fe700 1 -- 172.21.15.66:0/3822788585 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7244004160 con 0x7f7250105320 2023-07-21T22:12:43.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f724f7fe700 1 -- 172.21.15.66:0/3822788585 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72440042c0 con 0x7f7250105320 2023-07-21T22:12:43.085 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f724f7fe700 1 -- 172.21.15.66:0/3822788585 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72440119b0 con 0x7f7250105320 2023-07-21T22:12:43.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f7257165700 1 -- 172.21.15.66:0/3822788585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 msgr2=0x7f7250105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:43.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f7257165700 1 --2- 172.21.15.66:0/3822788585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f7250105730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f72440098a0 tx=0x7f7244009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:43.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f7257165700 1 -- 172.21.15.66:0/3822788585 shutdown_connections 2023-07-21T22:12:43.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f7257165700 1 --2- 172.21.15.66:0/3822788585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f7250105730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:43.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f7257165700 1 -- 172.21.15.66:0/3822788585 >> 172.21.15.66:0/3822788585 conn(0x7f7250100970 msgr2=0x7f7250102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:43.086 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.379+0000 7f7257165700 1 -- 172.21.15.66:0/3822788585 shutdown_connections 2023-07-21T22:12:43.087 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.380+0000 7f7257165700 1 -- 172.21.15.66:0/3822788585 wait complete. 2023-07-21T22:12:43.087 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7257165700 1 Processor -- start 2023-07-21T22:12:43.087 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7257165700 1 -- start start 2023-07-21T22:12:43.087 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7257165700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f725019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:43.087 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7257165700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725019b840 con 0x7f7250105320 2023-07-21T22:12:43.087 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7254f01700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f725019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:43.088 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7254f01700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f725019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33020/0 (socket says 172.21.15.66:33020) 2023-07-21T22:12:43.088 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.381+0000 7f7254f01700 1 -- 172.21.15.66:0/350432635 learned_addr learned my addr 172.21.15.66:0/350432635 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:43.088 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f7254f01700 1 -- 172.21.15.66:0/350432635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7244009550 con 0x7f7250105320 2023-07-21T22:12:43.088 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f7254f01700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f725019b300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7244006af0 tx=0x7f7244011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:43.088 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7244003a40 con 0x7f7250105320 2023-07-21T22:12:43.088 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7244003ba0 con 0x7f7250105320 2023-07-21T22:12:43.089 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725019ba40 con 0x7f7250105320 2023-07-21T22:12:43.089 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72440196c0 con 0x7f7250105320 2023-07-21T22:12:43.089 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.382+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725019be60 con 0x7f7250105320 2023-07-21T22:12:43.089 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.383+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f724401a430 con 0x7f7250105320 2023-07-21T22:12:43.089 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.383+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72501097b0 con 0x7f7250105320 2023-07-21T22:12:43.090 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.384+0000 7f724dffb700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7240038760 0x7f724003ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:43.090 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.384+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f724404cbb0 con 0x7f7250105320 2023-07-21T22:12:43.090 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.384+0000 7f724ffff700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7240038760 0x7f724003ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:43.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.385+0000 7f724ffff700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7240038760 0x7f724003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f723c006fd0 tx=0x7f723c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:43.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.389+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f724401ac60 con 0x7f7250105320 2023-07-21T22:12:43.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.599+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7250061b40 con 0x7f7250105320 2023-07-21T22:12:43.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.599+0000 7f724dffb700 1 -- 172.21.15.66:0/350432635 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7244023390 con 0x7f7250105320 2023-07-21T22:12:43.091 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7240038760 msgr2=0x7f724003ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7240038760 0x7f724003ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f723c006fd0 tx=0x7f723c006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 msgr2=0x7f725019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f725019b300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7244006af0 tx=0x7f7244011a10 comp rx=0 tx=0).stop 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 shutdown_connections 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7240038760 0x7f724003ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 --2- 172.21.15.66:0/350432635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7250105320 0x7f725019b300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:43.092 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 >> 172.21.15.66:0/350432635 conn(0x7f7250100970 msgr2=0x7f72501020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:43.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 shutdown_connections 2023-07-21T22:12:43.093 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:42.602+0000 7f7257165700 1 -- 172.21.15.66:0/350432635 wait complete. 2023-07-21T22:12:44.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:43 smithi066 ceph-mon[126379]: mgrmap e13: x(active, since 6s) 2023-07-21T22:12:44.361 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.559+0000 7ff96ac17700 1 Processor -- start 2023-07-21T22:12:44.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.559+0000 7ff96ac17700 1 -- start start 2023-07-21T22:12:44.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.560+0000 7ff96ac17700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96407a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:44.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.560+0000 7ff96ac17700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96407a8a0 con 0x7ff96407be60 2023-07-21T22:12:44.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.560+0000 7ff9689b3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96407a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:44.362 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.560+0000 7ff9689b3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96407a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33034/0 (socket says 172.21.15.66:33034) 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.560+0000 7ff9689b3700 1 -- 172.21.15.66:0/451153195 learned_addr learned my addr 172.21.15.66:0/451153195 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.560+0000 7ff9689b3700 1 -- 172.21.15.66:0/451153195 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96407a9e0 con 0x7ff96407be60 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.561+0000 7ff9689b3700 1 --2- 172.21.15.66:0/451153195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96407a360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9580098a0 tx=0x7ff958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e374ba967d95c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.561+0000 7ff9637fe700 1 -- 172.21.15.66:0/451153195 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff958004160 con 0x7ff96407be60 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.561+0000 7ff9637fe700 1 -- 172.21.15.66:0/451153195 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9580042c0 con 0x7ff96407be60 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.561+0000 7ff9637fe700 1 -- 172.21.15.66:0/451153195 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9580119b0 con 0x7ff96407be60 2023-07-21T22:12:44.363 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.561+0000 7ff96ac17700 1 -- 172.21.15.66:0/451153195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 msgr2=0x7ff96407a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:44.364 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.561+0000 7ff96ac17700 1 --2- 172.21.15.66:0/451153195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96407a360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9580098a0 tx=0x7ff958009bb0 comp rx=0 tx=0).stop 2023-07-21T22:12:44.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.562+0000 7ff96ac17700 1 -- 172.21.15.66:0/451153195 shutdown_connections 2023-07-21T22:12:44.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.562+0000 7ff96ac17700 1 --2- 172.21.15.66:0/451153195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96407a360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:44.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.562+0000 7ff96ac17700 1 -- 172.21.15.66:0/451153195 >> 172.21.15.66:0/451153195 conn(0x7ff964100970 msgr2=0x7ff964102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:44.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.562+0000 7ff96ac17700 1 -- 172.21.15.66:0/451153195 shutdown_connections 2023-07-21T22:12:44.367 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.562+0000 7ff96ac17700 1 -- 172.21.15.66:0/451153195 wait complete. 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff96ac17700 1 Processor -- start 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff96ac17700 1 -- start start 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff96ac17700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff96ac17700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96419fc60 con 0x7ff96407be60 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff9689b3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff9689b3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33044/0 (socket says 172.21.15.66:33044) 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.563+0000 7ff9689b3700 1 -- 172.21.15.66:0/954394281 learned_addr learned my addr 172.21.15.66:0/954394281 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:44.368 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff9689b3700 1 -- 172.21.15.66:0/954394281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958009550 con 0x7ff96407be60 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff9689b3700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96419f720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff958006af0 tx=0x7ff958011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff958003a40 con 0x7ff96407be60 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff958003ba0 con 0x7ff96407be60 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96419fe60 con 0x7ff96407be60 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff958019690 con 0x7ff96407be60 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.564+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9641a0280 con 0x7ff96407be60 2023-07-21T22:12:44.369 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.565+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff964107260 con 0x7ff96407be60 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.566+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ff95801aa60 con 0x7ff96407be60 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.566+0000 7ff961ffb700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff9540387b0 0x7ff95403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.566+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff95804cb40 con 0x7ff96407be60 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.566+0000 7ff963fff700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff9540387b0 0x7ff95403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.567+0000 7ff963fff700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff9540387b0 0x7ff95403ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff950006fd0 tx=0x7ff950006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.571+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff95801a430 con 0x7ff96407be60 2023-07-21T22:12:44.370 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.894+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7ff964107450 con 0x7ff96407be60 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.899+0000 7ff961ffb700 1 -- 172.21.15.66:0/954394281 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7ff958018350 con 0x7ff96407be60 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff9540387b0 msgr2=0x7ff95403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff9540387b0 0x7ff95403ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff950006fd0 tx=0x7ff950006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 msgr2=0x7ff96419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96419f720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff958006af0 tx=0x7ff958011a10 comp rx=0 tx=0).stop 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 shutdown_connections 2023-07-21T22:12:44.371 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff9540387b0 0x7ff95403ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 --2- 172.21.15.66:0/954394281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff96407be60 0x7ff96419f720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 >> 172.21.15.66:0/954394281 conn(0x7ff964100970 msgr2=0x7ff964106580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.901+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 shutdown_connections 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr 2023-07-21T22:12:43.902+0000 7ff96ac17700 1 -- 172.21.15.66:0/954394281 wait complete. 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-21T22:12:44.372 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout:Or, if you are only running a single cluster on this host: 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.373 INFO:teuthology.orchestra.run.smithi066.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout: ceph telemetry on 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout:For more information see: 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:44.374 INFO:teuthology.orchestra.run.smithi066.stdout:Bootstrap complete. 2023-07-21T22:12:44.425 INFO:tasks.cephadm:Fetching config... 2023-07-21T22:12:44.426 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:12:44.426 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-21T22:12:44.471 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-21T22:12:44.471 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:12:44.472 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-21T22:12:44.559 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-21T22:12:44.560 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:12:44.560 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-21T22:12:44.672 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-21T22:12:44.673 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:12:44.673 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-21T22:12:44.756 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-21T22:12:44.757 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:12:44.903 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:12:44.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:12:44.981 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:12:44.993 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T22:12:45.038 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBt4uCv0m3ejgqxiqsbGBi6nH2CTb3LY8U53cQpni9gnhoL0zW31GmS1WSB0HSllCTkuZ9ph+QrokSFvwVFlLUo2Urv0104+rRA3KRPGsSRz5wu3eK0RUd5YxjZ+0e8k69ss0BrlpFtPQvWo9RlO/dy8WXmxy7nR8jD5zJIkdQ7UZSELSkpgx9RwIizvcrs4f8dQOkrxxjNd8CVV0loen6pXEd1dbKdvzpAW5n40/5eE618hSbVrib0+j1olCi5++zjGLi7WjuYabzEIzPznqACSma8nfaE8+rWw3HIlNr08KfSYH1g+FDwA2G8XY2aP0jnozkCt6dKUmOEwfhlNS3bp8I51plGNz5IBR66RQ0gHbst4dm+WV56oDkfs+vQynVETVdSMMhmnAYECqyIhcOfQx61oivhXUVrEEs8ea/uxwLVhxafB0yG9ZMJJioFQDZHo/e8qpYud9QD0PWQdIZduURyEqu59IR/f7g9z4hPC53BiUt2FuxtQtG54VjtsE= ceph-6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:12:45.051 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-21T22:12:45.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:44 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/954394281' entity='client.admin' 2023-07-21T22:12:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:12:47.282 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:47.282 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:47.282 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:12:47.283 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:47.283 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:47.283 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:47.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 -- 172.21.15.66:0/956691063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 msgr2=0x7f6c480ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:47.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 --2- 172.21.15.66:0/956691063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c480ff9f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009910 tx=0x7f6c38009c20 comp rx=0 tx=0).stop 2023-07-21T22:12:47.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 -- 172.21.15.66:0/956691063 shutdown_connections 2023-07-21T22:12:47.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 --2- 172.21.15.66:0/956691063 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c480ff9f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:47.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 -- 172.21.15.66:0/956691063 >> 172.21.15.66:0/956691063 conn(0x7f6c480fab90 msgr2=0x7f6c480fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:47.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 -- 172.21.15.66:0/956691063 shutdown_connections 2023-07-21T22:12:47.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.495+0000 7f6c4ca11700 1 -- 172.21.15.66:0/956691063 wait complete. 2023-07-21T22:12:47.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.496+0000 7f6c4ca11700 1 Processor -- start 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.496+0000 7f6c4ca11700 1 -- start start 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4ca11700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c48196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4ca11700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48196eb0 con 0x7f6c480ff5e0 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c48196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c48196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33066/0 (socket says 172.21.15.66:33066) 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4659c700 1 -- 172.21.15.66:0/736953274 learned_addr learned my addr 172.21.15.66:0/736953274 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:47.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4659c700 1 -- 172.21.15.66:0/736953274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380095f0 con 0x7f6c480ff5e0 2023-07-21T22:12:47.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.497+0000 7f6c4659c700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c48196970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004f40 tx=0x7f6c380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:47.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.498+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c3801d070 con 0x7f6c480ff5e0 2023-07-21T22:12:47.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.498+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c481970b0 con 0x7f6c480ff5e0 2023-07-21T22:12:47.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.498+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c3800bc10 con 0x7f6c480ff5e0 2023-07-21T22:12:47.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.498+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c38003a70 con 0x7f6c480ff5e0 2023-07-21T22:12:47.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.498+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c48197550 con 0x7f6c480ff5e0 2023-07-21T22:12:47.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.499+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c481905b0 con 0x7f6c480ff5e0 2023-07-21T22:12:47.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.499+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6c38003d30 con 0x7f6c480ff5e0 2023-07-21T22:12:47.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.499+0000 7f6c377fe700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c300387b0 0x7f6c3003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:47.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.499+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c3804c570 con 0x7f6c480ff5e0 2023-07-21T22:12:47.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.500+0000 7f6c45d9b700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c300387b0 0x7f6c3003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:47.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.500+0000 7f6c45d9b700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c300387b0 0x7f6c3003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006fd0 tx=0x7f6c3c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:47.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.504+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6c3802abf0 con 0x7f6c480ff5e0 2023-07-21T22:12:47.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.711+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f6c4802c850 con 0x7f6c480ff5e0 2023-07-21T22:12:47.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.717+0000 7f6c377fe700 1 -- 172.21.15.66:0/736953274 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6c38026070 con 0x7f6c480ff5e0 2023-07-21T22:12:47.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c300387b0 msgr2=0x7f6c3003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:47.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c300387b0 0x7f6c3003ac60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c006fd0 tx=0x7f6c3c006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:47.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 msgr2=0x7f6c48196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:47.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c48196970 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c38004f40 tx=0x7f6c380057c0 comp rx=0 tx=0).stop 2023-07-21T22:12:47.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 shutdown_connections 2023-07-21T22:12:47.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c300387b0 0x7f6c3003ac60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:47.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 --2- 172.21.15.66:0/736953274 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c480ff5e0 0x7f6c48196970 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:47.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.719+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 >> 172.21.15.66:0/736953274 conn(0x7f6c480fab90 msgr2=0x7f6c480fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:47.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.720+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 shutdown_connections 2023-07-21T22:12:47.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:47.720+0000 7f6c4ca11700 1 -- 172.21.15.66:0/736953274 wait complete. 2023-07-21T22:12:48.245 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-21T22:12:48.245 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-21T22:12:48.982 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:12:49.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:48 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/736953274' entity='client.admin' 2023-07-21T22:12:49.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:49.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:50.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.339+0000 7f43286a0700 1 -- 172.21.15.66:0/2965367284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 msgr2=0x7f43200ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:50.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.339+0000 7f43286a0700 1 --2- 172.21.15.66:0/2965367284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f43200ff9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f431c009910 tx=0x7f431c009c20 comp rx=0 tx=0).stop 2023-07-21T22:12:50.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.340+0000 7f43286a0700 1 -- 172.21.15.66:0/2965367284 shutdown_connections 2023-07-21T22:12:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.340+0000 7f43286a0700 1 --2- 172.21.15.66:0/2965367284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f43200ff9f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.340+0000 7f43286a0700 1 -- 172.21.15.66:0/2965367284 >> 172.21.15.66:0/2965367284 conn(0x7f43200fab90 msgr2=0x7f43200fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.340+0000 7f43286a0700 1 -- 172.21.15.66:0/2965367284 shutdown_connections 2023-07-21T22:12:50.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.340+0000 7f43286a0700 1 -- 172.21.15.66:0/2965367284 wait complete. 2023-07-21T22:12:50.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.341+0000 7f43286a0700 1 Processor -- start 2023-07-21T22:12:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.341+0000 7f43286a0700 1 -- start start 2023-07-21T22:12:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.341+0000 7f43286a0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f4320196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.341+0000 7f43286a0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320196eb0 con 0x7f43200ff5e0 2023-07-21T22:12:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.341+0000 7f432643c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f4320196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.341+0000 7f432643c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f4320196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33090/0 (socket says 172.21.15.66:33090) 2023-07-21T22:12:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f432643c700 1 -- 172.21.15.66:0/3263742796 learned_addr learned my addr 172.21.15.66:0/3263742796 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f432643c700 1 -- 172.21.15.66:0/3263742796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c0095f0 con 0x7f43200ff5e0 2023-07-21T22:12:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f432643c700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f4320196970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f431c004f40 tx=0x7f431c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f431c01d070 con 0x7f43200ff5e0 2023-07-21T22:12:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f431c00bc10 con 0x7f43200ff5e0 2023-07-21T22:12:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43201970b0 con 0x7f43200ff5e0 2023-07-21T22:12:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.342+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f431c003a70 con 0x7f43200ff5e0 2023-07-21T22:12:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.343+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320197550 con 0x7f43200ff5e0 2023-07-21T22:12:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.343+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f431c003ce0 con 0x7f43200ff5e0 2023-07-21T22:12:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.344+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43201905b0 con 0x7f43200ff5e0 2023-07-21T22:12:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.344+0000 7f43177fe700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f430c038800 0x7f430c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.344+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f431c04c550 con 0x7f43200ff5e0 2023-07-21T22:12:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.344+0000 7f4325c3b700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f430c038800 0x7f430c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.345+0000 7f4325c3b700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f430c038800 0x7f430c03acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4310006fd0 tx=0x7f4310006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.349+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f431c02abf0 con 0x7f43200ff5e0 2023-07-21T22:12:50.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.565+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f43200609e0 con 0x7f430c038800 2023-07-21T22:12:50.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.569+0000 7f43177fe700 1 -- 172.21.15.66:0/3263742796 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f43200609e0 con 0x7f430c038800 2023-07-21T22:12:50.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f430c038800 msgr2=0x7f430c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:50.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f430c038800 0x7f430c03acb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4310006fd0 tx=0x7f4310006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:50.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 msgr2=0x7f4320196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:50.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f4320196970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f431c004f40 tx=0x7f431c0057c0 comp rx=0 tx=0).stop 2023-07-21T22:12:50.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 shutdown_connections 2023-07-21T22:12:50.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f430c038800 0x7f430c03acb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:50.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 --2- 172.21.15.66:0/3263742796 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f43200ff5e0 0x7f4320196970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:50.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.571+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 >> 172.21.15.66:0/3263742796 conn(0x7f43200fab90 msgr2=0x7f43200fb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:50.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.572+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 shutdown_connections 2023-07-21T22:12:50.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:50.572+0000 7f43286a0700 1 -- 172.21.15.66:0/3263742796 wait complete. 2023-07-21T22:12:51.174 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2023-07-21T22:12:51.174 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:12:51.174 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2023-07-21T22:12:51.195 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:12:51.195 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:12:51.254 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2023-07-21T22:12:51.255 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch host add smithi104 2023-07-21T22:12:51.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:51.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:51.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:51.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:12:51.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:12:51.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:12:51.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:51 smithi066 ceph-mon[126379]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:12:52.018 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:12:52.851 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:52 smithi066 ceph-mon[126379]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:12:53.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/3141999561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 msgr2=0x7f8b58105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:53.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 --2- 172.21.15.66:0/3141999561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58105a00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009910 tx=0x7f8b48009c20 comp rx=0 tx=0).stop 2023-07-21T22:12:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/3141999561 shutdown_connections 2023-07-21T22:12:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 --2- 172.21.15.66:0/3141999561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58105a00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/3141999561 >> 172.21.15.66:0/3141999561 conn(0x7f8b580fb130 msgr2=0x7f8b580fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/3141999561 shutdown_connections 2023-07-21T22:12:53.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.402+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/3141999561 wait complete. 2023-07-21T22:12:53.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.404+0000 7f8b5cc6f700 1 Processor -- start 2023-07-21T22:12:53.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.404+0000 7f8b5cc6f700 1 -- start start 2023-07-21T22:12:53.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.404+0000 7f8b5cc6f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:53.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.404+0000 7f8b5cc6f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58074790 con 0x7f8b58105630 2023-07-21T22:12:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.405+0000 7f8b5659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:53.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.405+0000 7f8b5659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57586/0 (socket says 172.21.15.66:57586) 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.405+0000 7f8b5659c700 1 -- 172.21.15.66:0/2774501906 learned_addr learned my addr 172.21.15.66:0/2774501906 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.405+0000 7f8b5659c700 1 -- 172.21.15.66:0/2774501906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480095f0 con 0x7f8b58105630 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.405+0000 7f8b5659c700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58072810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b48006010 tx=0x7f8b48004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.405+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b4801d070 con 0x7f8b58105630 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.406+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4800bc10 con 0x7f8b58105630 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.406+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b58072d50 con 0x7f8b58105630 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.406+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b48003910 con 0x7f8b58105630 2023-07-21T22:12:53.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.406+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b58073250 con 0x7f8b58105630 2023-07-21T22:12:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.407+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8b4800bd80 con 0x7f8b58105630 2023-07-21T22:12:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.407+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b581048d0 con 0x7f8b58105630 2023-07-21T22:12:53.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.407+0000 7f8b477fe700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8b40038800 0x7f8b4003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:53.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.407+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8b4804c590 con 0x7f8b58105630 2023-07-21T22:12:53.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.408+0000 7f8b55d9b700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8b40038800 0x7f8b4003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:53.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.408+0000 7f8b55d9b700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8b40038800 0x7f8b4003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c006fd0 tx=0x7f8b4c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:53.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.413+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8b4802a7c0 con 0x7f8b58105630 2023-07-21T22:12:53.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:53.621+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b580008d0 con 0x7f8b40038800 2023-07-21T22:12:54.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:53 smithi066 ceph-mon[126379]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:12:54.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:54.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:54.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:54 smithi066 ceph-mon[126379]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:12:55.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:54 smithi066 ceph-mon[126379]: Deploying cephadm binary to smithi104 2023-07-21T22:12:56.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.981+0000 7f8b477fe700 1 -- 172.21.15.66:0/2774501906 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8b580008d0 con 0x7f8b40038800 2023-07-21T22:12:56.025 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi104' with addr '172.21.15.104' 2023-07-21T22:12:56.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8b40038800 msgr2=0x7f8b4003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:56.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8b40038800 0x7f8b4003acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c006fd0 tx=0x7f8b4c006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:56.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 msgr2=0x7f8b58072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:56.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58072810 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8b48006010 tx=0x7f8b48004ff0 comp rx=0 tx=0).stop 2023-07-21T22:12:56.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 shutdown_connections 2023-07-21T22:12:56.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8b40038800 0x7f8b4003acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:56.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 --2- 172.21.15.66:0/2774501906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8b58105630 0x7f8b58072810 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:56.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 >> 172.21.15.66:0/2774501906 conn(0x7f8b580fb130 msgr2=0x7f8b580fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:56.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 shutdown_connections 2023-07-21T22:12:56.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:55.984+0000 7f8b5cc6f700 1 -- 172.21.15.66:0/2774501906 wait complete. 2023-07-21T22:12:56.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T22:12:57.156 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:12:57.408 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:57.409 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:56 smithi066 ceph-mon[126379]: Added host smithi104 2023-07-21T22:12:57.409 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:12:57.409 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:56 smithi066 ceph-mon[126379]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:12:57.409 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:58.434 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 -- 172.21.15.66:0/3184125469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 msgr2=0x7f7104105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 --2- 172.21.15.66:0/3184125469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104105a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f70f4009910 tx=0x7f70f4009c20 comp rx=0 tx=0).stop 2023-07-21T22:12:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 -- 172.21.15.66:0/3184125469 shutdown_connections 2023-07-21T22:12:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 --2- 172.21.15.66:0/3184125469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104105a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:58.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 -- 172.21.15.66:0/3184125469 >> 172.21.15.66:0/3184125469 conn(0x7f71040fb130 msgr2=0x7f71040fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:58.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 -- 172.21.15.66:0/3184125469 shutdown_connections 2023-07-21T22:12:58.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.709+0000 7f71088f8700 1 -- 172.21.15.66:0/3184125469 wait complete. 2023-07-21T22:12:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.711+0000 7f71088f8700 1 Processor -- start 2023-07-21T22:12:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.711+0000 7f71088f8700 1 -- start start 2023-07-21T22:12:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.711+0000 7f71088f8700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.712+0000 7f71088f8700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7104194eb0 con 0x7f7104105630 2023-07-21T22:12:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.712+0000 7f710259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.712+0000 7f710259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104194970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57592/0 (socket says 172.21.15.66:57592) 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.712+0000 7f710259c700 1 -- 172.21.15.66:0/67759599 learned_addr learned my addr 172.21.15.66:0/67759599 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.712+0000 7f710259c700 1 -- 172.21.15.66:0/67759599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40095f0 con 0x7f7104105630 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.712+0000 7f710259c700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104194970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006010 tx=0x7f70f4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.713+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70f401d070 con 0x7f7104105630 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.713+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70f400bc10 con 0x7f7104105630 2023-07-21T22:12:58.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.713+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f70f4003910 con 0x7f7104105630 2023-07-21T22:12:58.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.713+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7104195110 con 0x7f7104105630 2023-07-21T22:12:58.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.713+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7104195610 con 0x7f7104105630 2023-07-21T22:12:58.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.714+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f70f400bd80 con 0x7f7104105630 2023-07-21T22:12:58.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.714+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7104061b40 con 0x7f7104105630 2023-07-21T22:12:58.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.714+0000 7f70f37fe700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70ec038800 0x7f70ec03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:12:58.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.714+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f70f404c310 con 0x7f7104105630 2023-07-21T22:12:58.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.715+0000 7f7101d9b700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70ec038800 0x7f70ec03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:12:58.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.715+0000 7f7101d9b700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70ec038800 0x7f70ec03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70f8006fd0 tx=0x7f70f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:12:58.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.720+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f70f4029360 con 0x7f7104105630 2023-07-21T22:12:58.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.934+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7104195610 con 0x7f70ec038800 2023-07-21T22:12:58.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.936+0000 7f70f37fe700 1 -- 172.21.15.66:0/67759599 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f7104195610 con 0x7f70ec038800 2023-07-21T22:12:58.979 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:12:58.979 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2023-07-21T22:12:58.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70ec038800 msgr2=0x7f70ec03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:58.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70ec038800 0x7f70ec03acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f70f8006fd0 tx=0x7f70f8006e40 comp rx=0 tx=0).stop 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 msgr2=0x7f7104194970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104194970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006010 tx=0x7f70f4004ff0 comp rx=0 tx=0).stop 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 shutdown_connections 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70ec038800 0x7f70ec03acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 --2- 172.21.15.66:0/67759599 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7104105630 0x7f7104194970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.938+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 >> 172.21.15.66:0/67759599 conn(0x7f71040fb130 msgr2=0x7f71040fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:12:58.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.939+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 shutdown_connections 2023-07-21T22:12:58.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:12:58.939+0000 7f71088f8700 1 -- 172.21.15.66:0/67759599 wait complete. 2023-07-21T22:12:59.374 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:59 smithi066 ceph-mon[126379]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:12:59.375 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:12:59 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:12:59.445 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2023-07-21T22:12:59.445 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:12:59.445 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2023-07-21T22:12:59.466 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:12:59.466 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:12:59.525 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2023-07-21T22:12:59.525 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch host add smithi149 2023-07-21T22:13:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:00.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:00 smithi066 ceph-mon[126379]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:13:01.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:01.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:01.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:01.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 -- 172.21.15.66:0/3576103757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 msgr2=0x7fb2cc105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 --2- 172.21.15.66:0/3576103757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc105a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009910 tx=0x7fb2c8009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 -- 172.21.15.66:0/3576103757 shutdown_connections 2023-07-21T22:13:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 --2- 172.21.15.66:0/3576103757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc105a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:01.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 -- 172.21.15.66:0/3576103757 >> 172.21.15.66:0/3576103757 conn(0x7fb2cc0fb130 msgr2=0x7fb2cc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:01.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 -- 172.21.15.66:0/3576103757 shutdown_connections 2023-07-21T22:13:01.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.745+0000 7fb2d426d700 1 -- 172.21.15.66:0/3576103757 wait complete. 2023-07-21T22:13:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.746+0000 7fb2d426d700 1 Processor -- start 2023-07-21T22:13:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.746+0000 7fb2d426d700 1 -- start start 2023-07-21T22:13:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.747+0000 7fb2d426d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.747+0000 7fb2d426d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc19cea0 con 0x7fb2cc105630 2023-07-21T22:13:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.747+0000 7fb2d2009700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:01.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.747+0000 7fb2d2009700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57612/0 (socket says 172.21.15.66:57612) 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.747+0000 7fb2d2009700 1 -- 172.21.15.66:0/4197902308 learned_addr learned my addr 172.21.15.66:0/4197902308 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.747+0000 7fb2d2009700 1 -- 172.21.15.66:0/4197902308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80095f0 con 0x7fb2cc105630 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.748+0000 7fb2d2009700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc19c960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006010 tx=0x7fb2c8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.748+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c801d070 con 0x7fb2cc105630 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.748+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2c800bc10 con 0x7fb2cc105630 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.748+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2c8003910 con 0x7fb2cc105630 2023-07-21T22:13:01.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.748+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc19d100 con 0x7fb2cc105630 2023-07-21T22:13:01.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.748+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc1969e0 con 0x7fb2cc105630 2023-07-21T22:13:01.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.749+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb2c800bd80 con 0x7fb2cc105630 2023-07-21T22:13:01.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.749+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc061b40 con 0x7fb2cc105630 2023-07-21T22:13:01.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.750+0000 7fb2c37fe700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb2b8038800 0x7fb2b803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:01.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.750+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb2c804c560 con 0x7fb2cc105630 2023-07-21T22:13:01.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.750+0000 7fb2d1808700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb2b8038800 0x7fb2b803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:01.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.751+0000 7fb2d1808700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb2b8038800 0x7fb2b803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc006fd0 tx=0x7fb2bc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:01.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.755+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb2c802ad00 con 0x7fb2cc105630 2023-07-21T22:13:02.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:01.968+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2cc1974a0 con 0x7fb2b8038800 2023-07-21T22:13:02.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:02 smithi066 ceph-mon[126379]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:13:02.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:02 smithi066 ceph-mon[126379]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:02.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:02 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:02.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:02 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:02.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:02 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:03.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:03 smithi066 ceph-mon[126379]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:03.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:03 smithi066 ceph-mon[126379]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:13:03.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:03 smithi066 ceph-mon[126379]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:03.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:03 smithi066 ceph-mon[126379]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:04.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.370+0000 7fb2c37fe700 1 -- 172.21.15.66:0/4197902308 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb2cc1974a0 con 0x7fb2b8038800 2023-07-21T22:13:04.413 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi149' with addr '172.21.15.149' 2023-07-21T22:13:04.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb2b8038800 msgr2=0x7fb2b803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb2b8038800 0x7fb2b803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc006fd0 tx=0x7fb2bc006e40 comp rx=0 tx=0).stop 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 msgr2=0x7fb2cc19c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc19c960 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006010 tx=0x7fb2c8004ff0 comp rx=0 tx=0).stop 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 shutdown_connections 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb2b8038800 0x7fb2b803acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 --2- 172.21.15.66:0/4197902308 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb2cc105630 0x7fb2cc19c960 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:04.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 >> 172.21.15.66:0/4197902308 conn(0x7fb2cc0fb130 msgr2=0x7fb2cc0fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:04.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 shutdown_connections 2023-07-21T22:13:04.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:04.372+0000 7fb2d426d700 1 -- 172.21.15.66:0/4197902308 wait complete. 2023-07-21T22:13:04.438 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:04 smithi066 ceph-mon[126379]: Deploying cephadm binary to smithi149 2023-07-21T22:13:04.953 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T22:13:05.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:05 smithi066 ceph-mon[126379]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:05.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:05.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:05.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:05.693 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:06.243 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:06 smithi066 ceph-mon[126379]: Added host smithi149 2023-07-21T22:13:06.244 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:07.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.041+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/1853986346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 msgr2=0x7f6cb0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.041+0000 7f6cb5c5d700 1 --2- 172.21.15.66:0/1853986346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb0072c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009910 tx=0x7f6ca0009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:07.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.041+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/1853986346 shutdown_connections 2023-07-21T22:13:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.041+0000 7f6cb5c5d700 1 --2- 172.21.15.66:0/1853986346 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb0072c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.041+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/1853986346 >> 172.21.15.66:0/1853986346 conn(0x7f6cb00fb130 msgr2=0x7f6cb00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.042+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/1853986346 shutdown_connections 2023-07-21T22:13:07.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.042+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/1853986346 wait complete. 2023-07-21T22:13:07.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.043+0000 7f6cb5c5d700 1 Processor -- start 2023-07-21T22:13:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.043+0000 7f6cb5c5d700 1 -- start start 2023-07-21T22:13:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.044+0000 7f6cb5c5d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:07.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.044+0000 7f6cb5c5d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb019cf40 con 0x7f6cb00728b0 2023-07-21T22:13:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.044+0000 7f6caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.044+0000 7f6caf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55116/0 (socket says 172.21.15.66:55116) 2023-07-21T22:13:07.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.044+0000 7f6caf7fe700 1 -- 172.21.15.66:0/4252967939 learned_addr learned my addr 172.21.15.66:0/4252967939 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:13:07.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6caf7fe700 1 -- 172.21.15.66:0/4252967939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00095f0 con 0x7f6cb00728b0 2023-07-21T22:13:07.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6caf7fe700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb019ca00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0006010 tx=0x7f6ca0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:07.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ca001d070 con 0x7f6cb00728b0 2023-07-21T22:13:07.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb019d140 con 0x7f6cb00728b0 2023-07-21T22:13:07.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca00044a0 con 0x7f6cb00728b0 2023-07-21T22:13:07.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ca0003a70 con 0x7f6cb00728b0 2023-07-21T22:13:07.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.045+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb019d560 con 0x7f6cb00728b0 2023-07-21T22:13:07.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.046+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6ca0003cb0 con 0x7f6cb00728b0 2023-07-21T22:13:07.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.046+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb0061b40 con 0x7f6cb00728b0 2023-07-21T22:13:07.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.047+0000 7f6cacff9700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c980387b0 0x7f6c9803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:07.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.047+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ca004c690 con 0x7f6cb00728b0 2023-07-21T22:13:07.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.047+0000 7f6caeffd700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c980387b0 0x7f6c9803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.048+0000 7f6caeffd700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c980387b0 0x7f6c9803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4006fd0 tx=0x7f6ca4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:07.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.052+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6ca0029540 con 0x7f6cb00728b0 2023-07-21T22:13:07.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:07 smithi066 ceph-mon[126379]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:07.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:07 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.264+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cb0108c50 con 0x7f6c980387b0 2023-07-21T22:13:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.266+0000 7f6cacff9700 1 -- 172.21.15.66:0/4252967939 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f6cb0108c50 con 0x7f6c980387b0 2023-07-21T22:13:07.309 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:13:07.309 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2023-07-21T22:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c980387b0 msgr2=0x7f6c9803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c980387b0 0x7f6c9803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4006fd0 tx=0x7f6ca4006e40 comp rx=0 tx=0).stop 2023-07-21T22:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 msgr2=0x7f6cb019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb019ca00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0006010 tx=0x7f6ca0004ff0 comp rx=0 tx=0).stop 2023-07-21T22:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 shutdown_connections 2023-07-21T22:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6c980387b0 0x7f6c9803ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 --2- 172.21.15.66:0/4252967939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6cb00728b0 0x7f6cb019ca00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 >> 172.21.15.66:0/4252967939 conn(0x7f6cb00fb130 msgr2=0x7f6cb0106130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 shutdown_connections 2023-07-21T22:13:07.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:07.268+0000 7f6cb5c5d700 1 -- 172.21.15.66:0/4252967939 wait complete. 2023-07-21T22:13:07.773 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-21T22:13:07.774 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-21T22:13:08.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:08 smithi066 ceph-mon[126379]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:13:08.513 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:09.315 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:09 smithi066 ceph-mon[126379]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.875+0000 7f8ac7213700 1 -- 172.21.15.66:0/455413703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 msgr2=0x7f8ac0105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:09.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.875+0000 7f8ac7213700 1 --2- 172.21.15.66:0/455413703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac0105a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009910 tx=0x7f8ab4009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.876+0000 7f8ac7213700 1 -- 172.21.15.66:0/455413703 shutdown_connections 2023-07-21T22:13:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.876+0000 7f8ac7213700 1 --2- 172.21.15.66:0/455413703 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac0105a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.876+0000 7f8ac7213700 1 -- 172.21.15.66:0/455413703 >> 172.21.15.66:0/455413703 conn(0x7f8ac00fb130 msgr2=0x7f8ac00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.876+0000 7f8ac7213700 1 -- 172.21.15.66:0/455413703 shutdown_connections 2023-07-21T22:13:09.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.876+0000 7f8ac7213700 1 -- 172.21.15.66:0/455413703 wait complete. 2023-07-21T22:13:09.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.877+0000 7f8ac7213700 1 Processor -- start 2023-07-21T22:13:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.877+0000 7f8ac7213700 1 -- start start 2023-07-21T22:13:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.877+0000 7f8ac7213700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.877+0000 7f8ac7213700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac019ce70 con 0x7f8ac0105630 2023-07-21T22:13:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8ac4faf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8ac4faf700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac019c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55140/0 (socket says 172.21.15.66:55140) 2023-07-21T22:13:09.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8ac4faf700 1 -- 172.21.15.66:0/3827596983 learned_addr learned my addr 172.21.15.66:0/3827596983 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8ac4faf700 1 -- 172.21.15.66:0/3827596983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab40095f0 con 0x7f8ac0105630 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8ac4faf700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac019c930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006010 tx=0x7f8ab4004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ab401d070 con 0x7f8ac0105630 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac019d0d0 con 0x7f8ac0105630 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab400bc10 con 0x7f8ac0105630 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.878+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ab4003910 con 0x7f8ac0105630 2023-07-21T22:13:09.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.879+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac01969b0 con 0x7f8ac0105630 2023-07-21T22:13:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.880+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac0061b40 con 0x7f8ac0105630 2023-07-21T22:13:09.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.880+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f8ab400bd80 con 0x7f8ac0105630 2023-07-21T22:13:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.881+0000 7f8abdffb700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8ab0038800 0x7f8ab003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.881+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ab404c560 con 0x7f8ac0105630 2023-07-21T22:13:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.881+0000 7f8abffff700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8ab0038800 0x7f8ab003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:09.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.882+0000 7f8abffff700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8ab0038800 0x7f8ab003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8aac006fd0 tx=0x7f8aac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:09.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:09.886+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8ab4029360 con 0x7f8ac0105630 2023-07-21T22:13:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.100+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8ac0197ec0 con 0x7f8ac0105630 2023-07-21T22:13:10.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.391+0000 7f8abdffb700 1 -- 172.21.15.66:0/3827596983 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8ab4026020 con 0x7f8ac0105630 2023-07-21T22:13:10.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8ab0038800 msgr2=0x7f8ab003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:10.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8ab0038800 0x7f8ab003acb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8aac006fd0 tx=0x7f8aac006e40 comp rx=0 tx=0).stop 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 msgr2=0x7f8ac019c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac019c930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4006010 tx=0x7f8ab4004ff0 comp rx=0 tx=0).stop 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 shutdown_connections 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8ab0038800 0x7f8ab003acb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 --2- 172.21.15.66:0/3827596983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ac0105630 0x7f8ac019c930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 >> 172.21.15.66:0/3827596983 conn(0x7f8ac00fb130 msgr2=0x7f8ac0074200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:10.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 shutdown_connections 2023-07-21T22:13:10.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:10.393+0000 7f8ac7213700 1 -- 172.21.15.66:0/3827596983 wait complete. 2023-07-21T22:13:10.438 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2023-07-21T22:13:10.679 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.679 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:10.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:10.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:10.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3827596983' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:10.904 INFO:tasks.cephadm:Adding mon.a on smithi066 2023-07-21T22:13:10.905 INFO:tasks.cephadm:Adding mon.b on smithi104 2023-07-21T22:13:10.905 INFO:tasks.cephadm:Adding mon.c on smithi149 2023-07-21T22:13:10.905 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c' 2023-07-21T22:13:11.489 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:11.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:11 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3827596983' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:13:11.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:11 smithi066 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:12.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:12 smithi066 ceph-mon[126379]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:12.720 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.677+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/714987770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 msgr2=0x7f6b380ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:12.720 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.677+0000 7f6b3cd2e700 1 --2- 172.21.15.149:0/714987770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b380ff930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009910 tx=0x7f6b28009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:12.720 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.677+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/714987770 shutdown_connections 2023-07-21T22:13:12.720 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.677+0000 7f6b3cd2e700 1 --2- 172.21.15.149:0/714987770 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b380ff930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:12.720 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.677+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/714987770 >> 172.21.15.149:0/714987770 conn(0x7f6b38074f10 msgr2=0x7f6b38075310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:12.721 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.678+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/714987770 shutdown_connections 2023-07-21T22:13:12.721 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.678+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/714987770 wait complete. 2023-07-21T22:13:12.722 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.678+0000 7f6b3cd2e700 1 Processor -- start 2023-07-21T22:13:12.722 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.678+0000 7f6b3cd2e700 1 -- start start 2023-07-21T22:13:12.722 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.678+0000 7f6b3cd2e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b3819cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:12.722 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.678+0000 7f6b3cd2e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3819d010 con 0x7f6b380ff560 2023-07-21T22:13:12.722 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b3659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b3819cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b3659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b3819cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.149:44622/0 (socket says 172.21.15.149:44622) 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b3659c700 1 -- 172.21.15.149:0/3207243480 learned_addr learned my addr 172.21.15.149:0/3207243480 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b3659c700 1 -- 172.21.15.149:0/3207243480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b280095f0 con 0x7f6b380ff560 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b3659c700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b3819cad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b28004f40 tx=0x7f6b28005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b2801d070 con 0x7f6b380ff560 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3819d210 con 0x7f6b380ff560 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.679+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b2800be20 con 0x7f6b380ff560 2023-07-21T22:13:12.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.680+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b28022d00 con 0x7f6b380ff560 2023-07-21T22:13:12.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.680+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3819d630 con 0x7f6b380ff560 2023-07-21T22:13:12.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.680+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6b28022500 con 0x7f6b380ff560 2023-07-21T22:13:12.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.680+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b38061b40 con 0x7f6b380ff560 2023-07-21T22:13:12.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.681+0000 7f6b277fe700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b200387b0 0x7f6b2003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:12.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.681+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b2804c890 con 0x7f6b380ff560 2023-07-21T22:13:12.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.681+0000 7f6b35d9b700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b200387b0 0x7f6b2003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:12.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.682+0000 7f6b35d9b700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b200387b0 0x7f6b2003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c006fd0 tx=0x7f6b2c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:12.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.686+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6b2802a540 con 0x7f6b380ff560 2023-07-21T22:13:12.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.880+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b38102650 con 0x7f6b200387b0 2023-07-21T22:13:12.930 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.888+0000 7f6b277fe700 1 -- 172.21.15.149:0/3207243480 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6b38102650 con 0x7f6b200387b0 2023-07-21T22:13:12.930 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mon update... 2023-07-21T22:13:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.889+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b200387b0 msgr2=0x7f6b2003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.889+0000 7f6b3cd2e700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b200387b0 0x7f6b2003ac60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c006fd0 tx=0x7f6b2c006e40 comp rx=0 tx=0).stop 2023-07-21T22:13:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.889+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 msgr2=0x7f6b3819cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.889+0000 7f6b3cd2e700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b3819cad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6b28004f40 tx=0x7f6b28005020 comp rx=0 tx=0).stop 2023-07-21T22:13:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.890+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 shutdown_connections 2023-07-21T22:13:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.890+0000 7f6b3cd2e700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b200387b0 0x7f6b2003ac60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:12.933 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.890+0000 7f6b3cd2e700 1 --2- 172.21.15.149:0/3207243480 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b380ff560 0x7f6b3819cad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:12.933 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.890+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 >> 172.21.15.149:0/3207243480 conn(0x7f6b38074f10 msgr2=0x7f6b380fc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:12.933 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.890+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 shutdown_connections 2023-07-21T22:13:12.933 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:12.890+0000 7f6b3cd2e700 1 -- 172.21.15.149:0/3207243480 wait complete. 2023-07-21T22:13:13.361 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.b.service 2023-07-21T22:13:13.364 DEBUG:teuthology.orchestra.run.smithi149:mon.c> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.c.service 2023-07-21T22:13:13.366 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:13:13.367 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:13:13.396 INFO:journalctl@ceph.mon.c.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:13:13.400 INFO:journalctl@ceph.mon.b.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:13:14.132 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:14.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:14.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c;count:3 2023-07-21T22:13:14.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:14.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:13 smithi066 ceph-mon[126379]: Deploying daemon mon.c on smithi149 2023-07-21T22:13:15.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:14 smithi066 ceph-mon[126379]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:16.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 -- 172.21.15.149:0/703722262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 msgr2=0x7ff54c105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:16.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 --2- 172.21.15.149:0/703722262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c105a00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff548009910 tx=0x7ff548009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:16.191 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 -- 172.21.15.149:0/703722262 shutdown_connections 2023-07-21T22:13:16.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 --2- 172.21.15.149:0/703722262 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c105a00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:16.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 -- 172.21.15.149:0/703722262 >> 172.21.15.149:0/703722262 conn(0x7ff54c0fb0f0 msgr2=0x7ff54c0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:16.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 -- 172.21.15.149:0/703722262 shutdown_connections 2023-07-21T22:13:16.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 -- 172.21.15.149:0/703722262 wait complete. 2023-07-21T22:13:16.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 Processor -- start 2023-07-21T22:13:16.192 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.147+0000 7ff553b0d700 1 -- start start 2023-07-21T22:13:16.193 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.148+0000 7ff553b0d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:16.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.148+0000 7ff553b0d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19ad00 con 0x7ff54c105630 2023-07-21T22:13:16.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.148+0000 7ff5518a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:16.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.148+0000 7ff5518a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.149:44628/0 (socket says 172.21.15.149:44628) 2023-07-21T22:13:16.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.148+0000 7ff5518a9700 1 -- 172.21.15.149:0/2582712107 learned_addr learned my addr 172.21.15.149:0/2582712107 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:13:16.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.148+0000 7ff5518a9700 1 -- 172.21.15.149:0/2582712107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480095f0 con 0x7ff54c105630 2023-07-21T22:13:16.194 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.149+0000 7ff5518a9700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c19a7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff548004f40 tx=0x7ff548005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.149+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff54801d070 con 0x7ff54c105630 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.149+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c19af60 con 0x7ff54c105630 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.149+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff548004370 con 0x7ff54c105630 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.149+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5480039b0 con 0x7ff54c105630 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.149+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c194840 con 0x7ff54c105630 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.150+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7ff548003c70 con 0x7ff54c105630 2023-07-21T22:13:16.195 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.150+0000 7ff542ffd700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff5380387b0 0x7ff53803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:16.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.150+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c061b40 con 0x7ff54c105630 2023-07-21T22:13:16.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.150+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff54804d510 con 0x7ff54c105630 2023-07-21T22:13:16.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.150+0000 7ff5510a8700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff5380387b0 0x7ff53803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:16.196 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.151+0000 7ff5510a8700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff5380387b0 0x7ff53803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:16.199 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.156+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff548026020 con 0x7ff54c105630 2023-07-21T22:13:16.481 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 systemd[1]: Starting Ceph mon.c for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:13:16.519 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.475+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff54c195d50 con 0x7ff54c105630 2023-07-21T22:13:16.520 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.476+0000 7ff542ffd700 1 -- 172.21.15.149:0/2582712107 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7ff54802b440 con 0x7ff54c105630 2023-07-21T22:13:16.520 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:13:16.521 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":1,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:11:32.496817Z","created":"2023-07-21T22:11:32.496817Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-21T22:13:16.522 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff5380387b0 msgr2=0x7ff53803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:16.522 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff5380387b0 0x7ff53803ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006fd0 tx=0x7ff53c006e40 comp rx=0 tx=0).stop 2023-07-21T22:13:16.523 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 msgr2=0x7ff54c19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:16.523 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c19a7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff548004f40 tx=0x7ff548005020 comp rx=0 tx=0).stop 2023-07-21T22:13:16.523 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 shutdown_connections 2023-07-21T22:13:16.524 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff5380387b0 0x7ff53803ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:16.524 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 --2- 172.21.15.149:0/2582712107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff54c105630 0x7ff54c19a7c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:16.524 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 >> 172.21.15.149:0/2582712107 conn(0x7ff54c0fb0f0 msgr2=0x7ff54c0fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:16.525 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 shutdown_connections 2023-07-21T22:13:16.525 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:16.478+0000 7ff553b0d700 1 -- 172.21.15.149:0/2582712107 wait complete. 2023-07-21T22:13:16.525 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 1 2023-07-21T22:13:16.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 podman[129202]: 2023-07-21T22:13:17.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:13:17.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:13:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pidfile_write: ignore empty --pid-file 2023-07-21T22:13:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 bash[129202]: 72aaf61f4d184bf59aef24bc6471e4c1ebeb3aec1dcbba9b26d4d0b195486696 2023-07-21T22:13:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: load: jerasure load: lrc 2023-07-21T22:13:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:13:17.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Git sha 0 2023-07-21T22:13:17.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:13:17.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: DB SUMMARY 2023-07-21T22:13:17.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: DB Session ID: S2BA7AY8BOPLT5SWZ0FV 2023-07-21T22:13:17.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:13:17.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:13:17.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:13:17.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:13:17.243 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.env: 0x55658a4b2720 2023-07-21T22:13:17.243 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:13:17.243 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.info_log: 0x55658ca71260 2023-07-21T22:13:17.243 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:13:17.243 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.statistics: (nil) 2023-07-21T22:13:17.243 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.use_fsync: 0 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:13:17.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.db_log_dir: 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.wal_dir: 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:13:17.245 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.write_buffer_manager: 0x55658c9923c0 2023-07-21T22:13:17.246 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.unordered_write: 0 2023-07-21T22:13:17.248 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:13:17.248 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:13:17.248 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:13:17.248 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:13:17.248 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.row_cache: None 2023-07-21T22:13:17.248 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.wal_filter: None 2023-07-21T22:13:17.249 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:13:17.249 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:13:17.249 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:13:17.249 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:13:17.250 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.wal_compression: 0 2023-07-21T22:13:17.250 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:13:17.250 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:13:17.250 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:13:17.251 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:13:17.251 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:13:17.251 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:13:17.251 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:13:17.251 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:13:17.251 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:13:17.252 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:13:17.253 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_open_files: -1 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Compression algorithms supported: 2023-07-21T22:13:17.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kZSTD supported: 0 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:13:17.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.merge_operator: 2023-07-21T22:13:17.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_filter: None 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55658ca71360) 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:13:17.257 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_type: 0 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_index_type: 0 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_shortening: 1 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: checksum: 4 2023-07-21T22:13:17.258 INFO:journalctl@ceph.mon.c.smithi149.stdout: no_block_cache: 0 2023-07-21T22:13:17.259 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache: 0x55658ca311f0 2023-07-21T22:13:17.259 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:13:17.259 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_options: 2023-07-21T22:13:17.259 INFO:journalctl@ceph.mon.c.smithi149.stdout: capacity : 536870912 2023-07-21T22:13:17.259 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_shard_bits : 4 2023-07-21T22:13:17.259 INFO:journalctl@ceph.mon.c.smithi149.stdout: strict_capacity_limit : 0 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_cache_compressed: (nil) 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: persistent_cache: (nil) 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size: 4096 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_size_deviation: 10 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_restart_interval: 16 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: index_block_restart_interval: 1 2023-07-21T22:13:17.260 INFO:journalctl@ceph.mon.c.smithi149.stdout: metadata_block_size: 4096 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: partition_filters: 0 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: use_delta_encoding: 1 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: filter_policy: bloomfilter 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: whole_key_filtering: 1 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: verify_compression: 0 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:13:17.261 INFO:journalctl@ceph.mon.c.smithi149.stdout: format_version: 5 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_index_compression: 1 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout: block_align: 0 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout: max_auto_readahead_size: 262144 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout: prepopulate_block_cache: 0 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:13:17.262 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression: NoCompression 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.num_levels: 7 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:13:17.263 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:13:17.264 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:13:17.265 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:13:17.266 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:13:17.267 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:13:17.268 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:13:17.269 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:13:17.270 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:13:17.271 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:13:17.272 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.ttl: 2592000 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:13:17.273 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T22:13:17.274 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2b6bffbb-5447-4109-a973-617099718969 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977596850470, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977596851026, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689977596, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2b6bffbb-5447-4109-a973-617099718969", "db_session_id": "S2BA7AY8BOPLT5SWZ0FV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977596851116, "job": 1, "event": "recovery_finished"} 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T22:13:17.275 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55658caca000 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: DB pointer 0x55658cab6000 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** DB Stats ** 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:13:17.276 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2023-07-21T22:13:17.277 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** Compaction Stats [default] ** 2023-07-21T22:13:17.278 INFO:journalctl@ceph.mon.c.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:13:17.279 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: Interval compaction: 0.00 GB write, 0.45 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache BinnedLRUCache@0x55658ca311f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-07-21T22:13:17.280 INFO:journalctl@ceph.mon.c.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:13:17.281 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2023-07-21T22:13:17.281 INFO:journalctl@ceph.mon.c.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:13:17.281 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-21T22:13:17.281 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2023-07-21T22:13:17.281 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: starting mon.c rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:13:17.281 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(???) e0 preinit fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).mds e1 new map 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout: e1 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout: legacy client fscid: -1 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout: 2023-07-21T22:13:17.282 INFO:journalctl@ceph.mon.c.smithi149.stdout: No filesystems configured 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:13:17.283 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.284 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.285 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.285 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:17.285 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:17.285 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:17.285 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:13:17.285 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:17.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:17.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:13:17.286 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Deploying cephadm binary to smithi149 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.287 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Added host smithi149 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.288 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:17.289 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3827596983' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3827596983' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:13:17.290 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c;count:3 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:17.291 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: Deploying daemon mon.c on smithi149 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-21T22:13:17.292 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expand map: {default=false} 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta from 'false' to 'false' 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expanded map: {default=false} 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expand map: {default=info} 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta from 'info' to 'info' 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expanded map: {default=info} 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expand map: {default=daemon} 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T22:13:17.293 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T22:13:17.294 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expand map: {default=debug} 2023-07-21T22:13:17.294 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T22:13:17.294 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 ceph-mon[129268]: expand_channel_meta expanded map: {default=debug} 2023-07-21T22:13:17.294 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:16 smithi149 systemd[1]: Started Ceph mon.c for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:13:17.976 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:13:17.976 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:13:18.542 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:13:22.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:13:22.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: mon.a calling monitor election 2023-07-21T22:13:22.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:22.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: mon.c calling monitor election 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:13:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: fsmap 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: mgrmap e13: x(active, since 45s) 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: overall HEALTH_OK 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:22.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: mon.a calling monitor election 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: mon.c calling monitor election 2023-07-21T22:13:22.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: fsmap 2023-07-21T22:13:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: mgrmap e13: x(active, since 45s) 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: overall HEALTH_OK 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:22.762 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.718+0000 7fdb99c23700 1 -- 172.21.15.149:0/1707101251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 msgr2=0x7fdb7c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:22.763 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.718+0000 7fdb99c23700 1 --2- 172.21.15.149:0/1707101251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb7c005a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb94075020 tx=0x7fdb84010ec0 comp rx=0 tx=0).stop 2023-07-21T22:13:22.763 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.719+0000 7fdb99c23700 1 -- 172.21.15.149:0/1707101251 shutdown_connections 2023-07-21T22:13:22.763 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.719+0000 7fdb99c23700 1 --2- 172.21.15.149:0/1707101251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb7c005a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:22.763 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.719+0000 7fdb99c23700 1 -- 172.21.15.149:0/1707101251 >> 172.21.15.149:0/1707101251 conn(0x7fdb94073d70 msgr2=0x7fdb94074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:22.763 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.719+0000 7fdb99c23700 1 -- 172.21.15.149:0/1707101251 shutdown_connections 2023-07-21T22:13:22.763 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.719+0000 7fdb99c23700 1 -- 172.21.15.149:0/1707101251 wait complete. 2023-07-21T22:13:22.765 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.719+0000 7fdb99c23700 1 Processor -- start 2023-07-21T22:13:22.765 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb99c23700 1 -- start start 2023-07-21T22:13:22.765 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb99c23700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb940ffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:22.765 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb99c23700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:22.765 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb99c23700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94101ac0 con 0x7fdb7c0035c0 2023-07-21T22:13:22.765 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb99c23700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941a0de0 con 0x7fdb94100030 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb92ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb92ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35330/0 (socket says 172.21.15.149:35330) 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.720+0000 7fdb92ffd700 1 -- 172.21.15.149:0/3473959537 learned_addr learned my addr 172.21.15.149:0/3473959537 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb937fe700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb940ffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb92ffd700 1 -- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 msgr2=0x7fdb941004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb92ffd700 1 -- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 msgr2=0x7fdb941004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:13:22.766 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb92ffd700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb92ffd700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb937fe700 1 -- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 msgr2=0x7fdb941004a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb937fe700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb937fe700 1 -- 172.21.15.149:0/3473959537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84010910 con 0x7fdb7c0035c0 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb937fe700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb940ffaf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb84000c00 tx=0x7fdb84005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fdb84017050 con 0x7fdb7c0035c0 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.721+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb84012c20 con 0x7fdb7c0035c0 2023-07-21T22:13:22.767 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.722+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb941a0fe0 con 0x7fdb7c0035c0 2023-07-21T22:13:22.768 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.722+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fdb8401b610 con 0x7fdb7c0035c0 2023-07-21T22:13:22.768 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.722+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb941a14a0 con 0x7fdb7c0035c0 2023-07-21T22:13:22.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.722+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fdb8401b8d0 con 0x7fdb7c0035c0 2023-07-21T22:13:22.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.723+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb94061b40 con 0x7fdb7c0035c0 2023-07-21T22:13:22.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.723+0000 7fdb90ff9700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdb800385f0 0x7fdb8003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:22.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.723+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdb8404de70 con 0x7fdb7c0035c0 2023-07-21T22:13:22.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.723+0000 7fdb92ffd700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdb800385f0 0x7fdb8003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:22.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.724+0000 7fdb92ffd700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdb800385f0 0x7fdb8003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009180 tx=0x7fdb88008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:22.772 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.729+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdb8402dd30 con 0x7fdb7c0035c0 2023-07-21T22:13:23.038 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.995+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdb941a18d0 con 0x7fdb7c0035c0 2023-07-21T22:13:23.038 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.995+0000 7fdb90ff9700 1 -- 172.21.15.149:0/3473959537 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7fdb8400f360 con 0x7fdb7c0035c0 2023-07-21T22:13:23.039 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:13:23.039 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":2,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:13:16.882317Z","created":"2023-07-21T22:11:32.496817Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T22:13:23.041 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdb800385f0 msgr2=0x7fdb8003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:23.041 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdb800385f0 0x7fdb8003aaa0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009180 tx=0x7fdb88008040 comp rx=0 tx=0).stop 2023-07-21T22:13:23.041 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 msgr2=0x7fdb940ffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:23.041 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb940ffaf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdb84000c00 tx=0x7fdb84005c80 comp rx=0 tx=0).stop 2023-07-21T22:13:23.041 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 shutdown_connections 2023-07-21T22:13:23.042 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdb800385f0 0x7fdb8003aaa0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.042 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdb7c0035c0 0x7fdb940ffaf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.042 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 --2- 172.21.15.149:0/3473959537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb94100030 0x7fdb941004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:23.042 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 >> 172.21.15.149:0/3473959537 conn(0x7fdb94073d70 msgr2=0x7fdb940fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:23.042 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 shutdown_connections 2023-07-21T22:13:23.042 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:22.998+0000 7fdb99c23700 1 -- 172.21.15.149:0/3473959537 wait complete. 2023-07-21T22:13:23.043 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 2 2023-07-21T22:13:23.430 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:22 smithi149 ceph-mon[129268]: Deploying daemon mon.b on smithi104 2023-07-21T22:13:23.430 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:22 smithi149 ceph-mon[129268]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:23.430 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:22 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:23.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:22 smithi066 ceph-mon[126379]: Deploying daemon mon.b on smithi104 2023-07-21T22:13:23.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:22 smithi066 ceph-mon[126379]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:23.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:22 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:24.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:23 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/3473959537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:13:24.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:23 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/3473959537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:13:24.498 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T22:13:24.498 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T22:13:24.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 systemd[1]: Starting Ceph mon.b for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:13:25.059 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:13:25.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 podman[128981]: 2023-07-21T22:13:25.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T22:13:25.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev), process ceph-mon, pid 7 2023-07-21T22:13:25.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pidfile_write: ignore empty --pid-file 2023-07-21T22:13:25.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 bash[128981]: b858e006bfc8e4c1a054fa85d1fc78e933eae26d8a74e0dfb7bca1212b1f82e6 2023-07-21T22:13:25.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: load: jerasure load: lrc 2023-07-21T22:13:25.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: RocksDB version: 7.9.2 2023-07-21T22:13:25.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Git sha 0 2023-07-21T22:13:25.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Compile date 2023-07-21 19:07:44 2023-07-21T22:13:25.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: DB SUMMARY 2023-07-21T22:13:25.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: DB Session ID: KAKDZLJPV9PZL4XWM60Y 2023-07-21T22:13:25.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: CURRENT file: CURRENT 2023-07-21T22:13:25.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T22:13:25.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T22:13:25.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-21T22:13:25.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-21T22:13:25.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.error_if_exists: 0 2023-07-21T22:13:25.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.create_if_missing: 0 2023-07-21T22:13:25.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.paranoid_checks: 1 2023-07-21T22:13:25.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T22:13:25.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T22:13:25.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T22:13:25.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.env: 0x55f60b783720 2023-07-21T22:13:25.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T22:13:25.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.info_log: 0x55f60c851260 2023-07-21T22:13:25.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T22:13:25.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.statistics: (nil) 2023-07-21T22:13:25.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.use_fsync: 0 2023-07-21T22:13:25.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_log_file_size: 0 2023-07-21T22:13:25.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T22:13:25.124 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T22:13:25.124 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T22:13:25.124 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T22:13:25.124 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.allow_fallocate: 1 2023-07-21T22:13:25.125 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T22:13:25.125 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T22:13:25.125 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.use_direct_reads: 0 2023-07-21T22:13:25.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T22:13:25.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T22:13:25.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.db_log_dir: 2023-07-21T22:13:25.126 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.wal_dir: 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T22:13:25.127 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T22:13:25.128 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.write_buffer_manager: 0x55f60c7723c0 2023-07-21T22:13:25.128 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T22:13:25.128 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T22:13:25.128 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T22:13:25.128 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T22:13:25.128 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.unordered_write: 0 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T22:13:25.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.row_cache: None 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.wal_filter: None 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.two_write_queues: 0 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T22:13:25.130 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.wal_compression: 0 2023-07-21T22:13:25.131 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.atomic_flush: 0 2023-07-21T22:13:25.131 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T22:13:25.131 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T22:13:25.131 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T22:13:25.131 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.log_readahead_size: 0 2023-07-21T22:13:25.131 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T22:13:25.132 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T22:13:25.132 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T22:13:25.132 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T22:13:25.133 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T22:13:25.133 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_background_jobs: 2 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_background_compactions: -1 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_subcompactions: 1 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T22:13:25.134 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_open_files: -1 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T22:13:25.135 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T22:13:25.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T22:13:25.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_background_flushes: -1 2023-07-21T22:13:25.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Compression algorithms supported: 2023-07-21T22:13:25.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T22:13:25.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kZSTD supported: 0 2023-07-21T22:13:25.136 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kXpressCompression supported: 0 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kZlibCompression supported: 1 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kSnappyCompression supported: 1 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kLZ4Compression supported: 1 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: kBZip2Compression supported: 0 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T22:13:25.137 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.merge_operator: 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_filter: None 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T22:13:25.138 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T22:13:25.139 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f60c851360) 2023-07-21T22:13:25.139 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks: 1 2023-07-21T22:13:25.139 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T22:13:25.139 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T22:13:25.139 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_top_level_index_and_filter: 1 2023-07-21T22:13:25.139 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_type: 0 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_index_type: 0 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_shortening: 1 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: checksum: 4 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: no_block_cache: 0 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache: 0x55f60c8111f0 2023-07-21T22:13:25.140 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_name: BinnedLRUCache 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_options: 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: capacity : 536870912 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_shard_bits : 4 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: strict_capacity_limit : 0 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: high_pri_pool_ratio: 0.000 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_compressed: (nil) 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: persistent_cache: (nil) 2023-07-21T22:13:25.141 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size: 4096 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size_deviation: 10 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_restart_interval: 16 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_block_restart_interval: 1 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: metadata_block_size: 4096 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: partition_filters: 0 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: use_delta_encoding: 1 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: filter_policy: bloomfilter 2023-07-21T22:13:25.142 INFO:journalctl@ceph.mon.b.smithi104.stdout: whole_key_filtering: 1 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: verify_compression: 0 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: read_amp_bytes_per_bit: 0 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: format_version: 5 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_index_compression: 1 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_align: 0 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: max_auto_readahead_size: 262144 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: prepopulate_block_cache: 0 2023-07-21T22:13:25.143 INFO:journalctl@ceph.mon.b.smithi104.stdout: initial_auto_readahead_size: 8192 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression: NoCompression 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T22:13:25.144 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.num_levels: 7 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T22:13:25.145 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T22:13:25.146 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T22:13:25.147 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T22:13:25.148 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T22:13:25.149 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T22:13:25.150 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T22:13:25.150 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T22:13:25.150 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T22:13:25.150 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T22:13:25.150 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T22:13:25.150 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T22:13:25.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.table_properties_collectors: 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.inplace_update_support: 0 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T22:13:25.152 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.bloom_locality: 0 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.max_successive_merges: 0 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.ttl: 2592000 2023-07-21T22:13:25.153 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.enable_blob_files: false 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.min_blob_size: 0 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T22:13:25.154 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T22:13:25.155 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1eaaa5af-0079-481c-ad70-aba93daad25c 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977604951588, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977604952145, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689977604, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1eaaa5af-0079-481c-ad70-aba93daad25c", "db_session_id": "KAKDZLJPV9PZL4XWM60Y", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689977604952233, "job": 1, "event": "recovery_finished"} 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T22:13:25.156 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f60c8aa000 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: DB pointer 0x55f60c896000 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** DB Stats ** 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:13:25.157 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T22:13:25.158 INFO:journalctl@ceph.mon.b.smithi104.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T22:13:25.159 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2023-07-21T22:13:25.160 INFO:journalctl@ceph.mon.b.smithi104.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T22:13:25.160 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2023-07-21T22:13:25.160 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T22:13:25.160 INFO:journalctl@ceph.mon.b.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T22:13:25.160 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache BinnedLRUCache@0x55f60c8111f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-07-21T22:13:25.161 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: using public_addr v2:172.21.15.104:0/0 -> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: starting mon.b rank -1 at public addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] at bind addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(???) e0 preinit fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:13:25.162 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 systemd[1]: Started Ceph mon.b for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).mds e1 new map 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout: e1 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout: legacy client fscid: -1 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2023-07-21T22:13:25.163 INFO:journalctl@ceph.mon.b.smithi104.stdout: No filesystems configured 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T22:13:25.164 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.165 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.166 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Deploying cephadm binary to smithi149 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.167 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Added host smithi149 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:25.169 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3827596983' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.170 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.171 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.171 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.171 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3827596983' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T22:13:25.171 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:25.171 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.172 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:25.172 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c;count:3 2023-07-21T22:13:25.172 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.172 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:25.172 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:25.172 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Deploying daemon mon.c on smithi149 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:13:25.173 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.a calling monitor election 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.c calling monitor election 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.174 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: fsmap 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mgrmap e13: x(active, since 45s) 2023-07-21T22:13:25.175 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: overall HEALTH_OK 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: Deploying daemon mon.b on smithi104 2023-07-21T22:13:25.176 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:25.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/3473959537' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:13:25.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T22:13:25.177 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:24 smithi104 ceph-mon[129021]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-21T22:13:25.277 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:24 smithi149 ceph-mon[129268]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:25.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:24 smithi066 ceph-mon[126379]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.366 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:13:30.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: mon.a calling monitor election 2023-07-21T22:13:30.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: mon.c calling monitor election 2023-07-21T22:13:30.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:30.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: mon.b calling monitor election 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: fsmap 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: mgrmap e13: x(active, since 53s) 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: overall HEALTH_OK 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:30.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:13:30.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: mon.a calling monitor election 2023-07-21T22:13:30.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: mon.c calling monitor election 2023-07-21T22:13:30.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: mon.b calling monitor election 2023-07-21T22:13:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: fsmap 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: mgrmap e13: x(active, since 53s) 2023-07-21T22:13:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: overall HEALTH_OK 2023-07-21T22:13:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:30.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:13:30.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: mon.a calling monitor election 2023-07-21T22:13:30.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: mon.c calling monitor election 2023-07-21T22:13:30.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: mon.b calling monitor election 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0],c=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0]} removed_ranks: {} 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: fsmap 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: mgrmap e13: x(active, since 53s) 2023-07-21T22:13:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: overall HEALTH_OK 2023-07-21T22:13:30.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:30.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:30.739 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 -- 172.21.15.149:0/2347057961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 msgr2=0x7f1ab0007820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:30.740 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 --2- 172.21.15.149:0/2347057961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ab0007820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fa0 tx=0x7f1ab800ff70 comp rx=0 tx=0).stop 2023-07-21T22:13:30.740 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 -- 172.21.15.149:0/2347057961 shutdown_connections 2023-07-21T22:13:30.740 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 --2- 172.21.15.149:0/2347057961 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ab0007820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:30.740 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 -- 172.21.15.149:0/2347057961 >> 172.21.15.149:0/2347057961 conn(0x7f1ac8073d70 msgr2=0x7f1ac8074170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:30.741 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 -- 172.21.15.149:0/2347057961 shutdown_connections 2023-07-21T22:13:30.741 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 -- 172.21.15.149:0/2347057961 wait complete. 2023-07-21T22:13:30.741 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.696+0000 7f1ace79f700 1 Processor -- start 2023-07-21T22:13:30.741 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 -- start start 2023-07-21T22:13:30.743 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ac8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:30.743 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ac8198ea0 0x7f1ac81929e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:30.743 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:30.743 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac81994b0 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac819f780 con 0x7f1ac8198ea0 2023-07-21T22:13:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ace79f700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac819f8c0 con 0x7f1ac8192f20 2023-07-21T22:13:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35358/0 (socket says 172.21.15.149:35358) 2023-07-21T22:13:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 -- 172.21.15.149:0/688509226 learned_addr learned my addr 172.21.15.149:0/688509226 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:13:30.744 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ac7fff700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ac8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 -- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 msgr2=0x7f1ac81933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 -- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 msgr2=0x7f1ac81933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1accd3c700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ac7fff700 1 -- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 msgr2=0x7f1ac81933c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.697+0000 7f1ac7fff700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:30.745 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac7fff700 1 -- 172.21.15.149:0/688509226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ac8198ea0 msgr2=0x7f1ac81929e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:30.746 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac7fff700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ac8198ea0 0x7f1ac81929e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:30.746 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac7fff700 1 -- 172.21.15.149:0/688509226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab800f970 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.746 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac7fff700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ac8198960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fa0 tx=0x7f1ab80039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:30.746 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8012e40 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.746 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab80054f0 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.747 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab801d6a0 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.747 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac819fb40 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.747 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.698+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac819fff0 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.747 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.699+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f1ab8024050 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.747 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.699+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8061b40 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.748 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.699+0000 7f1ac57fa700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1ab4038c30 0x7f1ab403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:30.748 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.699+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1ab804e610 con 0x7f1ac80ff8d0 2023-07-21T22:13:30.748 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.700+0000 7f1ac77fe700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1ab4038c30 0x7f1ab403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:30.748 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.700+0000 7f1ac77fe700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1ab4038c30 0x7f1ab403b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0098e0 tx=0x7f1abc006dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:30.749 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.705+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1ab802fba0 con 0x7f1ac80ff8d0 2023-07-21T22:13:31.023 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.980+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1ac81a0380 con 0x7f1ac80ff8d0 2023-07-21T22:13:31.024 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.981+0000 7f1ac57fa700 1 -- 172.21.15.149:0/688509226 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1276 (secure 0 0 0) 0x7f1ab802b070 con 0x7f1ac80ff8d0 2023-07-21T22:13:31.024 INFO:teuthology.orchestra.run.smithi149.stdout: 2023-07-21T22:13:31.025 INFO:teuthology.orchestra.run.smithi149.stdout:{"epoch":3,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","modified":"2023-07-21T22:13:24.987658Z","created":"2023-07-21T22:11:32.496817Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-07-21T22:13:31.026 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1ab4038c30 msgr2=0x7f1ab403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:31.026 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1ab4038c30 0x7f1ab403b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0098e0 tx=0x7f1abc006dd0 comp rx=0 tx=0).stop 2023-07-21T22:13:31.027 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 msgr2=0x7f1ac8198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:31.027 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ac8198960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005fa0 tx=0x7f1ab80039e0 comp rx=0 tx=0).stop 2023-07-21T22:13:31.027 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 shutdown_connections 2023-07-21T22:13:31.027 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1ab4038c30 0x7f1ab403b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:31.027 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ac8192f20 0x7f1ac81933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:31.027 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ac80ff8d0 0x7f1ac8198960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:31.028 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 --2- 172.21.15.149:0/688509226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ac8198ea0 0x7f1ac81929e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:31.028 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 >> 172.21.15.149:0/688509226 conn(0x7f1ac8073d70 msgr2=0x7f1ac81009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:31.028 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 shutdown_connections 2023-07-21T22:13:31.028 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:30.983+0000 7f1ace79f700 1 -- 172.21.15.149:0/688509226 wait complete. 2023-07-21T22:13:31.029 INFO:teuthology.orchestra.run.smithi149.stderr:dumped monmap epoch 3 2023-07-21T22:13:31.371 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:31.372 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:31.372 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:31 smithi149 ceph-mon[129268]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:31.372 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:31.372 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/688509226' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:13:31.372 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:31.421 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-21T22:13:31.421 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-21T22:13:31.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:31.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:31 smithi066 ceph-mon[126379]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/688509226' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:13:31.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:31.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:31 smithi104 ceph-mon[129021]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/688509226' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T22:13:31.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:13:32.184 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:32.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:32.483 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:32.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:32.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:32.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:33.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:13:33.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:13:33.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:13:33.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:33.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:33.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:33.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:33.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:33.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:33 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:33.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:13:33.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:33 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:33.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:13:33.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:13:33.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:13:33.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:33 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.697+0000 7f5f111e0700 1 -- 172.21.15.66:0/2965863152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1055d0 msgr2=0x7f5f0c1059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.697+0000 7f5f111e0700 1 --2- 172.21.15.66:0/2965863152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1055d0 0x7f5f0c1059a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009910 tx=0x7f5efc009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.697+0000 7f5f111e0700 1 -- 172.21.15.66:0/2965863152 shutdown_connections 2023-07-21T22:13:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.697+0000 7f5f111e0700 1 --2- 172.21.15.66:0/2965863152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1055d0 0x7f5f0c1059a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.697+0000 7f5f111e0700 1 -- 172.21.15.66:0/2965863152 >> 172.21.15.66:0/2965863152 conn(0x7f5f0c0742b0 msgr2=0x7f5f0c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.697+0000 7f5f111e0700 1 -- 172.21.15.66:0/2965863152 shutdown_connections 2023-07-21T22:13:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.698+0000 7f5f111e0700 1 -- 172.21.15.66:0/2965863152 wait complete. 2023-07-21T22:13:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.699+0000 7f5f111e0700 1 Processor -- start 2023-07-21T22:13:33.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.699+0000 7f5f111e0700 1 -- start start 2023-07-21T22:13:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f111e0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f0c1055d0 0x7f5f0c196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f111e0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 0x7f5f0c19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f111e0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f0c19bd50 0x7f5f0c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f111e0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c197a70 con 0x7f5f0c1973a0 2023-07-21T22:13:33.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f111e0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a3ac0 con 0x7f5f0c1055d0 2023-07-21T22:13:33.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f111e0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1a3c00 con 0x7f5f0c19bd50 2023-07-21T22:13:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 0x7f5f0c19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f0a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 0x7f5f0c19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34276/0 (socket says 172.21.15.66:34276) 2023-07-21T22:13:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f0a59c700 1 -- 172.21.15.66:0/3110111303 learned_addr learned my addr 172.21.15.66:0/3110111303 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:13:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f0ad9d700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f0c1055d0 0x7f5f0c196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.700+0000 7f5f0b59e700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f0c19bd50 0x7f5f0c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:33.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f0a59c700 1 -- 172.21.15.66:0/3110111303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f0c19bd50 msgr2=0x7f5f0c19c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f0a59c700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f0c19bd50 0x7f5f0c19c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f0a59c700 1 -- 172.21.15.66:0/3110111303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f0c1055d0 msgr2=0x7f5f0c196e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f0a59c700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f0c1055d0 0x7f5f0c196e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f0a59c700 1 -- 172.21.15.66:0/3110111303 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc0095f0 con 0x7f5f0c1973a0 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f0a59c700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 0x7f5f0c19b810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000b7b0 tx=0x7f5f0000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00004230 con 0x7f5f0c1973a0 2023-07-21T22:13:33.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c1a3eb0 con 0x7f5f0c1973a0 2023-07-21T22:13:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f00004390 con 0x7f5f0c1973a0 2023-07-21T22:13:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00011ad0 con 0x7f5f0c1973a0 2023-07-21T22:13:33.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.701+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c1a4320 con 0x7f5f0c1973a0 2023-07-21T22:13:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.702+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5f00011c30 con 0x7f5f0c1973a0 2023-07-21T22:13:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.702+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c061b40 con 0x7f5f0c1973a0 2023-07-21T22:13:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.703+0000 7f5ef3fff700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5ef4038c80 0x7f5ef403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.703+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5f0004aed0 con 0x7f5f0c1973a0 2023-07-21T22:13:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.703+0000 7f5f0ad9d700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5ef4038c80 0x7f5ef403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:33.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.704+0000 7f5f0ad9d700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5ef4038c80 0x7f5ef403b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009fd0 tx=0x7f5efc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:33.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.708+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5f0001ae40 con 0x7f5f0c1973a0 2023-07-21T22:13:33.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.946+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f5f0c1981c0 con 0x7f5f0c1973a0 2023-07-21T22:13:33.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.946+0000 7f5ef3fff700 1 -- 172.21.15.66:0/3110111303 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f5f00021900 con 0x7f5f0c1973a0 2023-07-21T22:13:33.989 INFO:teuthology.orchestra.run.smithi066.stdout:# minimal ceph.conf for 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:13:33.989 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2023-07-21T22:13:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:13:33.990 INFO:teuthology.orchestra.run.smithi066.stdout: mon_host = [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2023-07-21T22:13:33.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5ef4038c80 msgr2=0x7f5ef403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:33.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5ef4038c80 0x7f5ef403b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009fd0 tx=0x7f5efc0058e0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 msgr2=0x7f5f0c19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:33.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 0x7f5f0c19b810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000b7b0 tx=0x7f5f0000bac0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 shutdown_connections 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5ef4038c80 0x7f5ef403b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f0c1055d0 0x7f5f0c196e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f0c19bd50 0x7f5f0c19c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 --2- 172.21.15.66:0/3110111303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f0c1973a0 0x7f5f0c19b810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 >> 172.21.15.66:0/3110111303 conn(0x7f5f0c0742b0 msgr2=0x7f5f0c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 shutdown_connections 2023-07-21T22:13:33.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:33.949+0000 7f5f111e0700 1 -- 172.21.15.66:0/3110111303 wait complete. 2023-07-21T22:13:34.558 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:34 smithi066 ceph-mon[126379]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:34.558 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:34 smithi066 ceph-mon[126379]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:13:34.559 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:34 smithi066 ceph-mon[126379]: Reconfiguring daemon mon.a on smithi066 2023-07-21T22:13:34.559 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3110111303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:34.559 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:34 smithi066 ceph-mon[126379]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:34.586 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-21T22:13:34.587 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:13:34.587 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:13:34.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:34 smithi104 ceph-mon[129021]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:34.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:34 smithi104 ceph-mon[129021]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:13:34.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:34 smithi104 ceph-mon[129021]: Reconfiguring daemon mon.a on smithi066 2023-07-21T22:13:34.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3110111303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:34.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:34 smithi104 ceph-mon[129021]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:34.700 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:13:34.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:34.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:34 smithi149 ceph-mon[129268]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:13:34.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:34 smithi149 ceph-mon[129268]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T22:13:34.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:34 smithi149 ceph-mon[129268]: Reconfiguring daemon mon.a on smithi066 2023-07-21T22:13:34.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3110111303' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:34.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:34 smithi149 ceph-mon[129268]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:34.811 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:13:34.812 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:13:34.848 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:13:34.848 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:34.919 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:13:34.919 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T22:13:34.954 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:13:34.955 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:13:35.026 INFO:tasks.cephadm:Adding mgr.x on smithi066 2023-07-21T22:13:35.026 INFO:tasks.cephadm:Adding mgr.y on smithi104 2023-07-21T22:13:35.026 INFO:tasks.cephadm:Adding mgr.z on smithi149 2023-07-21T22:13:35.027 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi066=x;smithi104=y;smithi149=z' 2023-07-21T22:13:35.621 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:13:36.116 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:36.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:36.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:13:36.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:36.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:36.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:36.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:35 smithi104 ceph-mon[129021]: Reconfiguring daemon mon.b on smithi104 2023-07-21T22:13:36.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:36.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:36.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:13:36.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:36.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:36.128 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:36.129 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:35 smithi149 ceph-mon[129268]: Reconfiguring daemon mon.b on smithi104 2023-07-21T22:13:36.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:36.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:36.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: Reconfiguring mon.b (monmap changed)... 2023-07-21T22:13:36.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:36.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:36.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:36.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:35 smithi066 ceph-mon[126379]: Reconfiguring daemon mon.b on smithi104 2023-07-21T22:13:36.902 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 -- 172.21.15.149:0/484395723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c0ff700 msgr2=0x7f7e4c0ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 --2- 172.21.15.149:0/484395723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c0ff700 0x7f7e4c0ffb10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009910 tx=0x7f7e3c009c20 comp rx=0 tx=0).stop 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 -- 172.21.15.149:0/484395723 shutdown_connections 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 --2- 172.21.15.149:0/484395723 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c0ff700 0x7f7e4c0ffb10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 -- 172.21.15.149:0/484395723 >> 172.21.15.149:0/484395723 conn(0x7f7e4c0facd0 msgr2=0x7f7e4c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 -- 172.21.15.149:0/484395723 shutdown_connections 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 -- 172.21.15.149:0/484395723 wait complete. 2023-07-21T22:13:36.903 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.858+0000 7f7e526f0700 1 Processor -- start 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 -- start start 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e4c0ff700 0x7f7e4c1949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 0x7f7e4c199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c195380 0x7f7e4c19c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c19cc90 con 0x7f7e4c195380 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c19ce00 con 0x7f7e4c0ff700 2023-07-21T22:13:36.904 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e526f0700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c19cfc0 con 0x7f7e4c194f10 2023-07-21T22:13:36.905 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e4b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 0x7f7e4c199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:36.905 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e4bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e4c0ff700 0x7f7e4c1949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:36.905 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e4b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 0x7f7e4c199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35366/0 (socket says 172.21.15.149:35366) 2023-07-21T22:13:36.905 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e4b7fe700 1 -- 172.21.15.149:0/1747362636 learned_addr learned my addr 172.21.15.149:0/1747362636 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:13:36.905 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e50c8d700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c195380 0x7f7e4c19c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:36.906 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e4b7fe700 1 -- 172.21.15.149:0/1747362636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e4c0ff700 msgr2=0x7f7e4c1949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.859+0000 7f7e4b7fe700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e4c0ff700 0x7f7e4c1949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e4b7fe700 1 -- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c195380 msgr2=0x7f7e4c19c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e4b7fe700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c195380 0x7f7e4c19c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e4b7fe700 1 -- 172.21.15.149:0/1747362636 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0095f0 con 0x7f7e4c194f10 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e50c8d700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c195380 0x7f7e4c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e4b7fe700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 0x7f7e4c199f80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b780 tx=0x7f7e4000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:36.907 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40004230 con 0x7f7e4c194f10 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e40004390 con 0x7f7e4c194f10 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c19d2a0 con 0x7f7e4c194f10 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40011ad0 con 0x7f7e4c194f10 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.860+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c19d820 con 0x7f7e4c194f10 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.861+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f7e40011c30 con 0x7f7e4c194f10 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.861+0000 7f7e497fa700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e34038c30 0x7f7e3403b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:36.908 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.861+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4c18e280 con 0x7f7e4c194f10 2023-07-21T22:13:36.909 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.861+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7e4004acc0 con 0x7f7e4c194f10 2023-07-21T22:13:36.909 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.862+0000 7f7e4bfff700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e34038c30 0x7f7e3403b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:36.909 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.862+0000 7f7e4bfff700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e34038c30 0x7f7e3403b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005130 tx=0x7f7e3c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:36.909 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:36.866+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7e4001a400 con 0x7f7e4c194f10 2023-07-21T22:13:37.120 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.077+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi104=y;smithi149=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e4c0609e0 con 0x7f7e34038c30 2023-07-21T22:13:37.127 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.085+0000 7f7e497fa700 1 -- 172.21.15.149:0/1747362636 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7e4c0609e0 con 0x7f7e34038c30 2023-07-21T22:13:37.127 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mgr update... 2023-07-21T22:13:37.129 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e34038c30 msgr2=0x7f7e3403b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:37.129 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e34038c30 0x7f7e3403b0e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005130 tx=0x7f7e3c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:13:37.131 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 msgr2=0x7f7e4c199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:37.131 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 0x7f7e4c199f80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000b780 tx=0x7f7e4000ba90 comp rx=0 tx=0).stop 2023-07-21T22:13:37.131 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 shutdown_connections 2023-07-21T22:13:37.131 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e34038c30 0x7f7e3403b0e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:37.132 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e4c0ff700 0x7f7e4c1949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:37.132 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e4c195380 0x7f7e4c19c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:37.132 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 --2- 172.21.15.149:0/1747362636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c194f10 0x7f7e4c199f80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:37.133 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 >> 172.21.15.149:0/1747362636 conn(0x7f7e4c0facd0 msgr2=0x7f7e4c0fb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:37.133 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 shutdown_connections 2023-07-21T22:13:37.133 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:13:37.087+0000 7f7e526f0700 1 -- 172.21.15.149:0/1747362636 wait complete. 2023-07-21T22:13:37.345 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:37.345 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:37.345 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:13:37.345 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:37.346 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:37.346 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:37.346 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: Reconfiguring daemon mon.c on smithi149 2023-07-21T22:13:37.346 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:36 smithi149 ceph-mon[129268]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:37.366 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:37.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:37.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:13:37.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:37.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:37.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:37.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: Reconfiguring daemon mon.c on smithi149 2023-07-21T22:13:37.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:36 smithi104 ceph-mon[129021]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:37.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:37.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:37.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: Reconfiguring mon.c (monmap changed)... 2023-07-21T22:13:37.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T22:13:37.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T22:13:37.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:37.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: Reconfiguring daemon mon.c on smithi149 2023-07-21T22:13:37.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:36 smithi066 ceph-mon[126379]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:37.679 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-21T22:13:37.682 DEBUG:teuthology.orchestra.run.smithi149:mgr.z> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-21T22:13:37.684 INFO:tasks.cephadm:Deploying OSDs... 2023-07-21T22:13:37.685 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:13:37.685 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:13:37.714 INFO:journalctl@ceph.mgr.z.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:13:37.715 INFO:journalctl@ceph.mgr.y.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:13:37.735 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:13:37.735 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2023-07-21T22:13:37.783 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:13:37.783 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:37.783 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-07-21T22:13:37.783 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:37.783 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:37.784 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-07-21 22:12:45.371679994 +0000 2023-07-21T22:13:37.784 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-07-21 22:01:51.310200216 +0000 2023-07-21T22:13:37.784 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-07-21 22:01:51.310200216 +0000 2023-07-21T22:13:37.784 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-07-21 22:01:51.280200426 +0000 2023-07-21T22:13:37.785 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:13:37.855 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-07-21T22:13:37.856 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-07-21T22:13:37.856 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.00031481 s, 1.6 MB/s 2023-07-21T22:13:37.858 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:13:37.905 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2023-07-21T22:13:37.950 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:13:37.951 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:37.951 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-07-21T22:13:37.951 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:37.951 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:37.951 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-07-21 22:12:45.371679994 +0000 2023-07-21T22:13:37.952 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-07-21 22:01:52.143194397 +0000 2023-07-21T22:13:37.952 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-07-21 22:01:52.143194397 +0000 2023-07-21T22:13:37.952 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-07-21 22:01:52.116194586 +0000 2023-07-21T22:13:37.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:13:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-07-21T22:13:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-07-21T22:13:38.058 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000337874 s, 1.5 MB/s 2023-07-21T22:13:38.061 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:13:38.106 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2023-07-21T22:13:38.193 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:13:38.193 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:38.193 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-07-21T22:13:38.194 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:38.194 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:38.194 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-07-21 22:12:45.371679994 +0000 2023-07-21T22:13:38.195 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-07-21 22:01:52.979188557 +0000 2023-07-21T22:13:38.195 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-07-21 22:01:52.979188557 +0000 2023-07-21T22:13:38.195 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-07-21 22:01:52.952188746 +0000 2023-07-21T22:13:38.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:13:38.301 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-07-21T22:13:38.301 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-07-21T22:13:38.301 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000291376 s, 1.8 MB/s 2023-07-21T22:13:38.303 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:13:38.350 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2023-07-21T22:13:38.395 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:13:38.395 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 468 Links: 1 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-07-21 22:12:45.371679994 +0000 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-07-21 22:01:53.807182773 +0000 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-07-21 22:01:53.807182773 +0000 2023-07-21T22:13:38.396 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-07-21 22:01:53.781182955 +0000 2023-07-21T22:13:38.397 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:13:38.427 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi104=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:38.428 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: Saving service mgr spec with placement smithi066=x;smithi104=y;smithi149=z;count:3 2023-07-21T22:13:38.428 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.428 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.428 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.428 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:38.428 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.429 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:38.429 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.429 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:38.429 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:13:38.429 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:38.429 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi104=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:38.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: Saving service mgr spec with placement smithi066=x;smithi104=y;smithi149=z;count:3 2023-07-21T22:13:38.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:38.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:38.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:38.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:13:38.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:38.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2023-07-21T22:13:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2023-07-21T22:13:38.503 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000523649 s, 978 kB/s 2023-07-21T22:13:38.505 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:13:38.591 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2023-07-21T22:13:38.592 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:13:38.612 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:13:38.613 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2023-07-21T22:13:38.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi066=x;smithi104=y;smithi149=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: Saving service mgr spec with placement smithi066=x;smithi104=y;smithi149=z;count:3 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:38.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:38.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:38.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:13:38.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:38.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:38.638 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:13:38.638 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:38.638 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 381999 Links: 1 2023-07-21T22:13:38.639 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:38.639 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:38.639 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-21 22:13:24.435124517 +0000 2023-07-21T22:13:38.639 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-21 21:58:13.912476474 +0000 2023-07-21T22:13:38.640 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-21 21:58:13.912476474 +0000 2023-07-21T22:13:38.640 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-21T22:13:38.640 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:13:38.707 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-21T22:13:38.707 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-21T22:13:38.707 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000272539 s, 1.9 MB/s 2023-07-21T22:13:38.708 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:13:38.770 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2023-07-21T22:13:38.829 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:13:38.829 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:38.829 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 384045 Links: 1 2023-07-21T22:13:38.830 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:38.830 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:38.830 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-21 22:13:24.435124517 +0000 2023-07-21T22:13:38.830 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-21 21:58:14.450470280 +0000 2023-07-21T22:13:38.830 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-21 21:58:14.450470280 +0000 2023-07-21T22:13:38.830 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-21T22:13:38.831 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:13:38.897 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-21T22:13:38.897 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-21T22:13:38.897 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000226828 s, 2.3 MB/s 2023-07-21T22:13:38.898 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:13:38.957 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2023-07-21T22:13:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:13:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:39.015 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 385347 Links: 1 2023-07-21T22:13:39.015 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:39.016 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:39.016 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-21 22:13:24.435124517 +0000 2023-07-21T22:13:39.016 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-21 21:58:14.966464339 +0000 2023-07-21T22:13:39.016 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-21 21:58:14.966464339 +0000 2023-07-21T22:13:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-21T22:13:39.016 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:13:39.081 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-21T22:13:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-21T22:13:39.082 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000197814 s, 2.6 MB/s 2023-07-21T22:13:39.083 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:13:39.132 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:38 smithi149 systemd[1]: Starting Ceph mgr.z for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:13:39.141 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2023-07-21T22:13:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:13:39.199 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:39.199 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 6h/6d Inode: 386080 Links: 1 2023-07-21T22:13:39.199 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:39.200 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:39.200 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2023-07-21 22:13:24.436124505 +0000 2023-07-21T22:13:39.200 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2023-07-21 21:58:15.494458259 +0000 2023-07-21T22:13:39.200 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2023-07-21 21:58:15.494458259 +0000 2023-07-21T22:13:39.200 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: - 2023-07-21T22:13:39.200 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:13:39.264 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2023-07-21T22:13:39.265 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2023-07-21T22:13:39.265 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000197208 s, 2.6 MB/s 2023-07-21T22:13:39.266 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:13:39.323 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2023-07-21T22:13:39.324 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T22:13:39.343 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T22:13:39.344 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2023-07-21T22:13:39.366 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:39 smithi104 ceph-mon[129021]: Deploying daemon mgr.z on smithi149 2023-07-21T22:13:39.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:39 smithi104 ceph-mon[129021]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:39.401 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T22:13:39.402 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:39.402 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 380016 Links: 1 2023-07-21T22:13:39.402 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:39.403 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:39.403 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-07-21 22:13:38.914499674 +0000 2023-07-21T22:13:39.403 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-07-21 21:58:26.271999603 +0000 2023-07-21T22:13:39.403 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-07-21 21:58:26.271999603 +0000 2023-07-21T22:13:39.404 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-07-21T22:13:39.404 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T22:13:39.437 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-mon[129268]: Deploying daemon mgr.z on smithi149 2023-07-21T22:13:39.438 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-mon[129268]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:39.438 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 podman[131091]: 2023-07-21T22:13:39.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:39 smithi066 ceph-mon[126379]: Deploying daemon mgr.z on smithi149 2023-07-21T22:13:39.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:39 smithi066 ceph-mon[126379]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:39.466 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-07-21T22:13:39.467 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-07-21T22:13:39.467 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000262476 s, 2.0 MB/s 2023-07-21T22:13:39.469 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T22:13:39.527 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2023-07-21T22:13:39.585 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T22:13:39.585 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:39.585 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 381174 Links: 1 2023-07-21T22:13:39.585 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:39.585 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:39.586 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-07-21 22:13:38.914499674 +0000 2023-07-21T22:13:39.586 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-07-21 21:58:26.800989063 +0000 2023-07-21T22:13:39.586 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-07-21 21:58:26.800989063 +0000 2023-07-21T22:13:39.586 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-07-21T22:13:39.586 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T22:13:39.654 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-07-21T22:13:39.654 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-07-21T22:13:39.654 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000224299 s, 2.3 MB/s 2023-07-21T22:13:39.655 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T22:13:39.713 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2023-07-21T22:13:39.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 bash[131091]: d12e652ab94c0ce2efb064dc63c934caf59b77a0171d18c2e5f06a82b63b286f 2023-07-21T22:13:39.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-mgr[131212]: -- 172.21.15.149:0/1197895414 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55df239351e0 con 0x55df2390ac00 2023-07-21T22:13:39.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 systemd[1]: Started Ceph mgr.z for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:13:39.738 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:39.667+0000 7fbc1973b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:13:39.753 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T22:13:39.754 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:39.754 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 369651 Links: 1 2023-07-21T22:13:39.754 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:39.754 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:39.754 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-07-21 22:13:38.914499674 +0000 2023-07-21T22:13:39.755 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-07-21 21:58:27.317978762 +0000 2023-07-21T22:13:39.755 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-07-21 21:58:27.317978762 +0000 2023-07-21T22:13:39.755 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-07-21T22:13:39.755 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T22:13:39.820 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-07-21T22:13:39.820 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-07-21T22:13:39.820 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000226194 s, 2.3 MB/s 2023-07-21T22:13:39.821 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T22:13:39.879 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2023-07-21T22:13:39.937 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T22:13:39.937 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T22:13:39.937 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 6h/6d Inode: 381330 Links: 1 2023-07-21T22:13:39.937 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:13:39.937 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T22:13:39.938 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2023-07-21 22:13:38.914499674 +0000 2023-07-21T22:13:39.938 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2023-07-21 21:58:27.844968261 +0000 2023-07-21T22:13:39.938 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2023-07-21 21:58:27.844968261 +0000 2023-07-21T22:13:39.938 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: - 2023-07-21T22:13:39.938 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T22:13:40.002 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2023-07-21T22:13:40.002 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2023-07-21T22:13:40.002 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000187964 s, 2.7 MB/s 2023-07-21T22:13:40.003 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T22:13:40.060 INFO:tasks.cephadm:Deploying osd.0 on smithi066 with /dev/vg_nvme/lv_4... 2023-07-21T22:13:40.060 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:13:40.156 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:39.749+0000 7fbc1973b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:13:40.157 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:39.843+0000 7fbc1973b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:13:40.157 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:39.940+0000 7fbc1973b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:13:40.157 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:40.114+0000 7fbc1973b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:13:40.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:40.421+0000 7fbc1973b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:13:40.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:40.510+0000 7fbc1973b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:13:40.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:40.640+0000 7fbc1973b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:13:40.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:40.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: Deploying daemon mgr.y on smithi104 2023-07-21T22:13:40.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-mon[129268]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:40.801 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:40.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:40.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:13:40.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:40.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:40.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: Deploying daemon mgr.y on smithi104 2023-07-21T22:13:40.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:40 smithi104 ceph-mon[129021]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:40.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:40.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:40.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T22:13:40.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:40.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:40.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: Deploying daemon mgr.y on smithi104 2023-07-21T22:13:40.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:40 smithi066 ceph-mon[126379]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:41.196 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:40.734+0000 7fbc1973b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:13:41.367 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 systemd[1]: Starting Ceph mgr.y for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:13:41.487 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:41.154+0000 7fbc1973b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:13:41.679 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 podman[130461]: 2023-07-21T22:13:41.975 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 ceph-mgr[130501]: -- 172.21.15.104:0/1662636989 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596fc49f1e0 con 0x5596fc475000 2023-07-21T22:13:41.975 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 bash[130461]: 83d91cef76409611af5634ee04e910eb2ed53cf0d1b3c1c64d6368c474ecd25a 2023-07-21T22:13:41.975 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 systemd[1]: Started Ceph mgr.y for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:13:41.975 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:41.759+0000 7fc33e559200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:13:41.975 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:41.838+0000 7fc33e559200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:13:41.975 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:41 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:41.932+0000 7fc33e559200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:13:42.237 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:41.734+0000 7fbc1973b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:13:42.237 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:41.908+0000 7fbc1973b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:13:42.251 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:42.024+0000 7fc33e559200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:13:42.575 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:42.209+0000 7fc33e559200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:13:42.575 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:42.532+0000 7fc33e559200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:13:42.866 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:42.620+0000 7fc33e559200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:13:42.867 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:42.755+0000 7fc33e559200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:13:42.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:42.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:42.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:42.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:42.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:42.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-mon[129021]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:42.896 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:13:42.896 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:13:42.897 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-07-21T22:13:42.897 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-07-21T22:13:42.897 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0398659 s, 263 MB/s 2023-07-21T22:13:42.897 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-07-21T22:13:43.036 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:42.724+0000 7fbc1973b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:13:43.037 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:42.819+0000 7fbc1973b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:13:43.037 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.038 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.038 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.038 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.038 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:43.039 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-mon[129268]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:43.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:43.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:43.040 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:42 smithi066 ceph-mon[126379]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:43.290 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi066:vg_nvme/lv_4 2023-07-21T22:13:43.329 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:42 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:42.994+0000 7fbc1973b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:13:43.343 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:42.855+0000 7fc33e559200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:13:43.344 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:43 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:43.300+0000 7fc33e559200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:13:43.721 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:43.287+0000 7fbc1973b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:13:43.721 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:43.528+0000 7fbc1973b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:13:43.986 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:43.678+0000 7fbc1973b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:13:44.043 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:13:44.237 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:43.944+0000 7fbc1973b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:13:44.366 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:43 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:43.884+0000 7fc33e559200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:13:44.367 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:44.052+0000 7fc33e559200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:13:44.848 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.848 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:13:44.849 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:44.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:44.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:44.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: Reconfiguring daemon mgr.x on smithi066 2023-07-21T22:13:44.850 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-mon[129021]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:44.892 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:44.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:44.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: Reconfiguring daemon mgr.x on smithi066 2023-07-21T22:13:44.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:44 smithi066 ceph-mon[126379]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:44.987 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:44.650+0000 7fbc1973b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:13:44.987 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:44.780+0000 7fbc1973b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:13:44.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:44.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T22:13:44.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T22:13:44.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T22:13:44.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:44.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: Reconfiguring daemon mgr.x on smithi066 2023-07-21T22:13:44.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-mon[129268]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:45.105 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:44.805+0000 7fc33e559200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:13:45.105 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:44 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:44.894+0000 7fc33e559200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:13:45.337 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:44 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:44.970+0000 7fbc1973b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:13:45.337 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:45.083+0000 7fbc1973b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:13:45.366 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:45.062+0000 7fc33e559200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:13:45.737 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:45.294+0000 7fbc1973b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:13:45.742 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:45.333+0000 7fc33e559200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:13:45.743 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:45.555+0000 7fc33e559200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:13:45.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 -- 172.21.15.66:0/2601099109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0101720 msgr2=0x7f64a0107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:45.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 --2- 172.21.15.66:0/2601099109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0101720 0x7f64a0107fe0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f649800b0a0 tx=0x7f649800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:13:45.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 -- 172.21.15.66:0/2601099109 shutdown_connections 2023-07-21T22:13:45.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 --2- 172.21.15.66:0/2601099109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0101720 0x7f64a0107fe0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:45.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 --2- 172.21.15.66:0/2601099109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:45.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 --2- 172.21.15.66:0/2601099109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a0100d90 0x7f64a01011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:45.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 -- 172.21.15.66:0/2601099109 >> 172.21.15.66:0/2601099109 conn(0x7f64a00fb1a0 msgr2=0x7f64a00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:13:45.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 -- 172.21.15.66:0/2601099109 shutdown_connections 2023-07-21T22:13:45.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.794+0000 7f64a7f66700 1 -- 172.21.15.66:0/2601099109 wait complete. 2023-07-21T22:13:45.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.795+0000 7f64a7f66700 1 Processor -- start 2023-07-21T22:13:45.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.795+0000 7f64a7f66700 1 -- start start 2023-07-21T22:13:45.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a7f66700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a0197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a7f66700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0100d90 0x7f64a0198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a7f66700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a0101720 0x7f64a019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a7f66700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a019f750 con 0x7f64a0101720 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a7f66700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a019f890 con 0x7f64a0100d90 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a7f66700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a019fa20 con 0x7f64a00ffb90 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a0197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:45.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a6503700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a0101720 0x7f64a019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a0197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50204/0 (socket says 172.21.15.66:50204) 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 -- 172.21.15.66:0/108110536 learned_addr learned my addr 172.21.15.66:0/108110536 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5501700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0100d90 0x7f64a0198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 -- 172.21.15.66:0/108110536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0100d90 msgr2=0x7f64a0198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0100d90 0x7f64a0198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 -- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a0101720 msgr2=0x7f64a019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a0101720 0x7f64a019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:13:45.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.796+0000 7f64a5d02700 1 -- 172.21.15.66:0/108110536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c009520 con 0x7f64a00ffb90 2023-07-21T22:13:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.797+0000 7f64a5d02700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a0197bc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f649c00ea80 tx=0x7f649c00ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.797+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c00cbd0 con 0x7f64a00ffb90 2023-07-21T22:13:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.797+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f649c00cd30 con 0x7f64a00ffb90 2023-07-21T22:13:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.797+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6490009520 con 0x7f64a00ffb90 2023-07-21T22:13:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.797+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c018680 con 0x7f64a00ffb90 2023-07-21T22:13:45.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.797+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6498009d00 con 0x7f64a00ffb90 2023-07-21T22:13:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.798+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a010b3b0 con 0x7f64a00ffb90 2023-07-21T22:13:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.798+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f649c010430 con 0x7f64a00ffb90 2023-07-21T22:13:45.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.799+0000 7f6496ffd700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f648c0389e0 0x7f648c03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:13:45.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.799+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f649c014070 con 0x7f64a00ffb90 2023-07-21T22:13:45.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.799+0000 7f64a5501700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f648c0389e0 0x7f648c03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:13:45.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.799+0000 7f64a5501700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f648c0389e0 0x7f648c03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6490009df0 tx=0x7f6490009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:13:45.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:45.804+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f649c01f720 con 0x7f64a00ffb90 2023-07-21T22:13:46.007 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:45.700+0000 7fc33e559200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:13:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:46.019+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f64a0107650 con 0x7f648c0389e0 2023-07-21T22:13:46.366 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:45.964+0000 7fc33e559200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:13:46.487 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:13:46.155+0000 7fbc1973b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:13:46.867 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:46.653+0000 7fc33e559200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:13:46.867 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:46.781+0000 7fc33e559200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:13:46.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:46.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:46.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:46.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:46.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:46.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:46.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:46.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:46.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:13:46.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:13:46.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: Standby manager daemon z started 2023-07-21T22:13:46.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:13:46.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:13:46.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:46.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:46.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:46.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:46.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:46.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:46.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:46.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:46.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:13:46.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:13:46.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: Standby manager daemon z started 2023-07-21T22:13:46.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:13:46.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:46 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:13:46.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:13:46.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:46.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:13:46.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:13:46.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:13:46.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:13:46.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: Standby manager daemon z started 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:13:46.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:46 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/3212243405' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:13:47.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:47.175+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f649c018d00 con 0x7f64a00ffb90 2023-07-21T22:13:47.335 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:46 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:46.968+0000 7fc33e559200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:13:47.335 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:47.080+0000 7fc33e559200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:13:47.616 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:47.292+0000 7fc33e559200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:13:48.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-21T22:13:48.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:13:48.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: Standby manager daemon y started 2023-07-21T22:13:48.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:13:48.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:13:48.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:13:48.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:48 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:13:48.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-21T22:13:48.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:13:48.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: Standby manager daemon y started 2023-07-21T22:13:48.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:13:48.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:13:48.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:13:48.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:48 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:13:48.616 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:13:48.135+0000 7fc33e559200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:13:48.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: mgrmap e14: x(active, since 71s), standbys: z 2023-07-21T22:13:48.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:13:48.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: Standby manager daemon y started 2023-07-21T22:13:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:13:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:13:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:13:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:48 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/2599897666' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:13:48.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:13:48.646+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mon.1 v2:172.21.15.149:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f649c0187e0 con 0x7f64a00ffb90 2023-07-21T22:13:49.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:49.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-07-21T22:13:49.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:13:49.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2598009492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]: dispatch 2023-07-21T22:13:49.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]: dispatch 2023-07-21T22:13:49.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]': finished 2023-07-21T22:13:49.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:13:49.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:13:49.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:49.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-07-21T22:13:49.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:13:49.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2598009492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]: dispatch 2023-07-21T22:13:49.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]: dispatch 2023-07-21T22:13:49.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]': finished 2023-07-21T22:13:49.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:13:49.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:13:49.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:49.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: mgrmap e15: x(active, since 72s), standbys: z, y 2023-07-21T22:13:49.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:13:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2598009492' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]: dispatch 2023-07-21T22:13:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]: dispatch 2023-07-21T22:13:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "165f7676-76ef-4250-b74b-48159136d422"}]': finished 2023-07-21T22:13:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T22:13:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:13:50.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:50 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3543766748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:50.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:50 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3543766748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:50.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:50 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3543766748' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:13:51.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:51 smithi149 ceph-mon[129268]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:51.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:51 smithi104 ceph-mon[129021]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:51.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:51 smithi066 ceph-mon[126379]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:53.471 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:53 smithi066 ceph-mon[126379]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:53.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:53 smithi149 ceph-mon[129268]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:53.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:53 smithi104 ceph-mon[129021]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:55.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:55 smithi066 ceph-mon[126379]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:55.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:55 smithi149 ceph-mon[129268]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:55.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:55 smithi104 ceph-mon[129021]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:57.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:57 smithi066 ceph-mon[126379]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:57.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:13:57.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:57.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:57 smithi149 ceph-mon[129268]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:57.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:13:57.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:57.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:57 smithi104 ceph-mon[129021]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:57.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T22:13:57.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:13:58.262 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:58 smithi066 ceph-mon[126379]: Deploying daemon osd.0 on smithi066 2023-07-21T22:13:58.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:58 smithi149 ceph-mon[129268]: Deploying daemon osd.0 on smithi066 2023-07-21T22:13:58.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:58 smithi104 ceph-mon[129021]: Deploying daemon osd.0 on smithi066 2023-07-21T22:13:59.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:13:59 smithi149 ceph-mon[129268]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:59.579 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:13:59 smithi066 ceph-mon[126379]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:13:59.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:13:59 smithi104 ceph-mon[129021]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:01.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:01 smithi149 ceph-mon[129268]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:01.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:01 smithi104 ceph-mon[129021]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:01.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:01 smithi066 ceph-mon[126379]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:03.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:03 smithi066 ceph-mon[126379]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:03.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:03 smithi104 ceph-mon[129021]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:03.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:03 smithi149 ceph-mon[129268]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:04.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:04 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:04.574 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:04 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:04.575 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:04 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:04.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:04 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:04.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:04 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:04.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:04 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:04.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:04 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:04.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:04 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:04.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:04 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:05.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:05 smithi066 ceph-mon[126379]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:05.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:05 smithi104 ceph-mon[129021]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:05.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:05 smithi149 ceph-mon[129268]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:06.771 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 0 on host 'smithi066' 2023-07-21T22:14:06.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.729+0000 7f6496ffd700 1 -- 172.21.15.66:0/108110536 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f64a0107650 con 0x7f648c0389e0 2023-07-21T22:14:06.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.731+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f648c0389e0 msgr2=0x7f648c03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:06.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.731+0000 7f64a7f66700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f648c0389e0 0x7f648c03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6490009df0 tx=0x7f6490009260 comp rx=0 tx=0).stop 2023-07-21T22:14:06.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 msgr2=0x7f64a0197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:06.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a0197bc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f649c00ea80 tx=0x7f649c00ed90 comp rx=0 tx=0).stop 2023-07-21T22:14:06.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 shutdown_connections 2023-07-21T22:14:06.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f648c0389e0 0x7f648c03ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:06.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64a0101720 0x7f64a019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:06.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a00ffb90 0x7f64a0197bc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:06.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 --2- 172.21.15.66:0/108110536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64a0100d90 0x7f64a0198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:06.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 >> 172.21.15.66:0/108110536 conn(0x7f64a00fb1a0 msgr2=0x7f64a00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:06.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 shutdown_connections 2023-07-21T22:14:06.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:06.732+0000 7f64a7f66700 1 -- 172.21.15.66:0/108110536 wait complete. 2023-07-21T22:14:07.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:07.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:07.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:07.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:07.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:07.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:07.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:07.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:07.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:07.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:07.257 DEBUG:teuthology.orchestra.run.smithi066:osd.0> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.0.service 2023-07-21T22:14:07.264 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2023-07-21T22:14:07.265 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:14:07.346 INFO:journalctl@ceph.osd.0.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:14:07.823 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:14:08.068 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:07 smithi104 ceph-mon[129021]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:14:08.070 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:07 smithi066 ceph-mon[126379]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:14:08.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:07 smithi149 ceph-mon[129268]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T22:14:09.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:08 smithi104 ceph-mon[129021]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:14:09.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:08 smithi104 ceph-mon[129021]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:14:09.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:09.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:08 smithi104 ceph-mon[129021]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:14:09.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:08 smithi104 ceph-mon[129021]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:09.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-mon[126379]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:14:09.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-mon[126379]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:14:09.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:09.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-mon[126379]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:14:09.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-mon[126379]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:09.194 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-0[138663]: 2023-07-21T22:14:08.800+0000 7fd37025e700 -1 osd.0 0 waiting for initial osdmap 2023-07-21T22:14:09.195 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-0[138663]: 2023-07-21T22:14:08.804+0000 7fd36a850700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:14:09.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:08 smithi149 ceph-mon[129268]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T22:14:09.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:08 smithi149 ceph-mon[129268]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T22:14:09.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:09.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:08 smithi149 ceph-mon[129268]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:14:09.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:08 smithi149 ceph-mon[129268]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T22:14:09.334 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:14:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:14:09.335 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2023-07-21T22:14:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2023-07-21T22:14:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109821 s, 955 MB/s 2023-07-21T22:14:09.335 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2023-07-21T22:14:09.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2023-07-21T22:14:09.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:09 smithi066 ceph-mon[126379]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:14:09.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:09 smithi066 ceph-mon[126379]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:14:09.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:09.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:10.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:09 smithi104 ceph-mon[129021]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:14:10.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:09 smithi104 ceph-mon[129021]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:14:10.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:10.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:10.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:09 smithi149 ceph-mon[129268]: from='osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:14:10.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:09 smithi149 ceph-mon[129268]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T22:14:10.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:10.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:10.327 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:14:11.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:14:11.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:14:11.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] boot 2023-07-21T22:14:11.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi066 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi066 to 17267M 2023-07-21T22:14:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:10 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:14:11.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:14:11.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] boot 2023-07-21T22:14:11.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:14:11.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:11.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi066 2023-07-21T22:14:11.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi066 to 17267M 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:10 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:14:11.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: osd.0 [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] boot 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi066 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:11.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi066 to 17267M 2023-07-21T22:14:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:10 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:11.518 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.475+0000 7f514b622700 1 -- 172.21.15.104:0/2079774503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 msgr2=0x7f51441004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.475+0000 7f514b622700 1 --2- 172.21.15.104:0/2079774503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f51441004b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5140009990 tx=0x7f5140009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- 172.21.15.104:0/2079774503 shutdown_connections 2023-07-21T22:14:11.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 --2- 172.21.15.104:0/2079774503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f51441004b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 --2- 172.21.15.104:0/2079774503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51440ff6b0 0x7f5144106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.519 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 --2- 172.21.15.104:0/2079774503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51440fed00 0x7f51440ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- 172.21.15.104:0/2079774503 >> 172.21.15.104:0/2079774503 conn(0x7f51440f9860 msgr2=0x7f51440fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:11.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- 172.21.15.104:0/2079774503 shutdown_connections 2023-07-21T22:14:11.520 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- 172.21.15.104:0/2079774503 wait complete. 2023-07-21T22:14:11.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 Processor -- start 2023-07-21T22:14:11.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- start start 2023-07-21T22:14:11.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51440fed00 0x7f5144102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.522 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51440ff6b0 0x7f51441010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f5144101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5144101db0 con 0x7f51440fed00 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5144101f20 con 0x7f51441000a0 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f514b622700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514419fa00 con 0x7f51440ff6b0 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.476+0000 7f5149bbf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f5144101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f5144101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56832/0 (socket says 172.21.15.104:56832) 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 -- 172.21.15.104:0/444472913 learned_addr learned my addr 172.21.15.104:0/444472913 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-21T22:14:11.523 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5148bbd700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51440ff6b0 0x7f51441010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 -- 172.21.15.104:0/444472913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51440ff6b0 msgr2=0x7f51441010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51440ff6b0 0x7f51441010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 -- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51440fed00 msgr2=0x7f5144102aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51440fed00 0x7f5144102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 -- 172.21.15.104:0/444472913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51400095f0 con 0x7f51441000a0 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f5149bbf700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f5144101630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f513c00b970 tx=0x7f513c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:11.524 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c00c9b0 con 0x7f51441000a0 2023-07-21T22:14:11.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f513c014440 con 0x7f51441000a0 2023-07-21T22:14:11.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514419fce0 con 0x7f51441000a0 2023-07-21T22:14:11.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0075d0 con 0x7f51441000a0 2023-07-21T22:14:11.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.477+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51441a0250 con 0x7f51441000a0 2023-07-21T22:14:11.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.478+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514410a350 con 0x7f51441000a0 2023-07-21T22:14:11.525 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.479+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f513c0145b0 con 0x7f51441000a0 2023-07-21T22:14:11.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.480+0000 7f513a7fc700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51300a11c0 0x7f51300a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:11.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.480+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f513c006e40 con 0x7f51441000a0 2023-07-21T22:14:11.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.480+0000 7f51493be700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51300a11c0 0x7f51300a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:11.526 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.481+0000 7f51493be700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51300a11c0 0x7f51300a3670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51400095c0 tx=0x7f5140009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:11.527 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.483+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f513c0958b0 con 0x7f51441000a0 2023-07-21T22:14:11.718 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:11.675+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f51440fe340 con 0x7f51300a11c0 2023-07-21T22:14:12.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:12 smithi066 ceph-mon[126379]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:14:12.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:12.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:12.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:12.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:12 smithi149 ceph-mon[129268]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:14:12.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:12.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:12.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:12.575 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:12 smithi104 ceph-mon[129021]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T22:14:12.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:12.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:12.576 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:13.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:13 smithi104 ceph-mon[129021]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:13.434 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:13 smithi104 ceph-mon[129021]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:13.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:13 smithi066 ceph-mon[126379]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:13.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:13 smithi066 ceph-mon[126379]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:13.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:13 smithi149 ceph-mon[129268]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:13.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:13 smithi149 ceph-mon[129268]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:14.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/1517507022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]: dispatch 2023-07-21T22:14:14.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:14 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]: dispatch 2023-07-21T22:14:14.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:14 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]': finished 2023-07-21T22:14:14.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:14 smithi066 ceph-mon[126379]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:14:14.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:14 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:14.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/1517507022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]: dispatch 2023-07-21T22:14:14.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:14 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]: dispatch 2023-07-21T22:14:14.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:14 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]': finished 2023-07-21T22:14:14.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:14 smithi149 ceph-mon[129268]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:14:14.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:14 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:14.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/1517507022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]: dispatch 2023-07-21T22:14:14.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:14 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]: dispatch 2023-07-21T22:14:14.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:14 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77b37050-b8b8-4218-b605-b362f6375262"}]': finished 2023-07-21T22:14:14.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:14 smithi104 ceph-mon[129021]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T22:14:14.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:14 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:15.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:15 smithi066 ceph-mon[126379]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:15.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/259244398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:14:15.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:15 smithi149 ceph-mon[129268]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:15.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/259244398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:14:15.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:15 smithi104 ceph-mon[129021]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:15.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/259244398' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:14:17.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:17 smithi066 ceph-mon[126379]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:17.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:17 smithi149 ceph-mon[129268]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:17.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:17 smithi104 ceph-mon[129021]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:18.943 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:18 smithi066 ceph-mon[126379]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:18.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:18 smithi149 ceph-mon[129268]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:19.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:18 smithi104 ceph-mon[129021]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:21.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:21 smithi149 ceph-mon[129268]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:21.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:14:21.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:21.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:21 smithi104 ceph-mon[129021]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:21.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:21 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:14:21.526 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:21 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:21.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:21 smithi066 ceph-mon[126379]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:21.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T22:14:21.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:22.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:22 smithi104 ceph-mon[129021]: Deploying daemon osd.1 on smithi104 2023-07-21T22:14:22.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:22 smithi149 ceph-mon[129268]: Deploying daemon osd.1 on smithi104 2023-07-21T22:14:22.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:22 smithi066 ceph-mon[126379]: Deploying daemon osd.1 on smithi104 2023-07-21T22:14:23.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:23 smithi104 ceph-mon[129021]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:23.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:23 smithi149 ceph-mon[129268]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:23.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:23 smithi066 ceph-mon[126379]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:25.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:25 smithi104 ceph-mon[129021]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:25.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:25 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:25.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:25 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:25.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:25 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:25.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:25 smithi149 ceph-mon[129268]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:25.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:25 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:25.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:25 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:25.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:25 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:25.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:25 smithi066 ceph-mon[126379]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:25.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:25 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:25.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:25 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:25.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:25 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:27.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:27 smithi104 ceph-mon[129021]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:27.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:27.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:27.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:27 smithi149 ceph-mon[129268]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:27.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:27.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:27.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:27 smithi066 ceph-mon[126379]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:27.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.943+0000 7f513a7fc700 1 -- 172.21.15.104:0/444472913 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f51440fe340 con 0x7f51300a11c0 2023-07-21T22:14:27.986 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2023-07-21T22:14:27.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51300a11c0 msgr2=0x7f51300a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51300a11c0 0x7f51300a3670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51400095c0 tx=0x7f5140009530 comp rx=0 tx=0).stop 2023-07-21T22:14:27.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 msgr2=0x7f5144101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:27.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f5144101630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f513c00b970 tx=0x7f513c00bd30 comp rx=0 tx=0).stop 2023-07-21T22:14:27.988 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 shutdown_connections 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51300a11c0 0x7f51300a3670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51441000a0 0x7f5144101630 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51440fed00 0x7f5144102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 --2- 172.21.15.104:0/444472913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51440ff6b0 0x7f51441010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 >> 172.21.15.104:0/444472913 conn(0x7f51440f9860 msgr2=0x7f51440fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 shutdown_connections 2023-07-21T22:14:27.989 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:14:27.945+0000 7f514b622700 1 -- 172.21.15.104:0/444472913 wait complete. 2023-07-21T22:14:28.414 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.1.service 2023-07-21T22:14:28.417 INFO:tasks.cephadm:Deploying osd.2 on smithi149 with /dev/vg_nvme/lv_4... 2023-07-21T22:14:28.417 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T22:14:28.447 INFO:journalctl@ceph.osd.1.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:14:28.982 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:14:29.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:28 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:29.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:28 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:29.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:28 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:29.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:28 smithi149 ceph-mon[129268]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:29.274 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1[134104]: 2023-07-21T22:14:28.964+0000 7f1cae65e640 -1 osd.1 0 log_to_monitors true 2023-07-21T22:14:29.275 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:28 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:29.275 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:28 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:29.275 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:28 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:29.275 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:28 smithi104 ceph-mon[129021]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:29.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:28 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:29.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:28 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:29.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:28 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:29.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:28 smithi066 ceph-mon[126379]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:30.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:29 smithi104 ceph-mon[129021]: from='osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:14:30.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:29 smithi104 ceph-mon[129021]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:14:30.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:29 smithi149 ceph-mon[129268]: from='osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:14:30.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:29 smithi149 ceph-mon[129268]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:14:30.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:29 smithi066 ceph-mon[126379]: from='osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:14:30.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:29 smithi066 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T22:14:30.560 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T22:14:30.560 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T22:14:30.560 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-07-21T22:14:30.560 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-07-21T22:14:30.560 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111181 s, 943 MB/s 2023-07-21T22:14:30.561 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2023-07-21T22:14:30.561 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-07-21T22:14:30.935 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2023-07-21T22:14:31.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:14:31.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:14:31.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:31.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:31.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.367 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1[134104]: 2023-07-21T22:14:30.952+0000 7f1ca44d7700 -1 osd.1 0 waiting for initial osdmap 2023-07-21T22:14:31.367 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1[134104]: 2023-07-21T22:14:30.955+0000 7f1c9dac7700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:14:31.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:14:31.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T22:14:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T22:14:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:14:31.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:14:31.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:31.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:31.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:31.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:31.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:31.490 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:14:32.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:31 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi104 2023-07-21T22:14:32.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:31 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi104 to 17168M 2023-07-21T22:14:32.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:31 smithi104 ceph-mon[129021]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:14:32.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:31 smithi104 ceph-mon[129021]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:14:32.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:32.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:32.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:31 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi104 2023-07-21T22:14:32.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:31 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi104 to 17168M 2023-07-21T22:14:32.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:31 smithi066 ceph-mon[126379]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:14:32.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:31 smithi066 ceph-mon[126379]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:14:32.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:32.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:32.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:31 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi104 2023-07-21T22:14:32.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:31 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi104 to 17168M 2023-07-21T22:14:32.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:31 smithi149 ceph-mon[129268]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:14:32.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:31 smithi149 ceph-mon[129268]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T22:14:32.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:32.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:32.703 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 -- 172.21.15.149:0/4023432560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f00fefb0 msgr2=0x7f53f00ff3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.703 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 --2- 172.21.15.149:0/4023432560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f00fefb0 0x7f53f00ff3c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009990 tx=0x7f53e4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.703 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 -- 172.21.15.149:0/4023432560 shutdown_connections 2023-07-21T22:14:32.704 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 --2- 172.21.15.149:0/4023432560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f0100680 0x7f53f0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.704 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 --2- 172.21.15.149:0/4023432560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f00fefb0 0x7f53f00ff3c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.704 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 --2- 172.21.15.149:0/4023432560 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 0x7f53f0100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.705 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 -- 172.21.15.149:0/4023432560 >> 172.21.15.149:0/4023432560 conn(0x7f53f00f9860 msgr2=0x7f53f00fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:32.705 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 -- 172.21.15.149:0/4023432560 shutdown_connections 2023-07-21T22:14:32.705 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.657+0000 7f53f51a4700 1 -- 172.21.15.149:0/4023432560 wait complete. 2023-07-21T22:14:32.706 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 Processor -- start 2023-07-21T22:14:32.706 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 -- start start 2023-07-21T22:14:32.706 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f00fefb0 0x7f53f01948f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.706 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 0x7f53f0194e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.707 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 0x7f53f019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.707 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f019c480 con 0x7f53f00ffcd0 2023-07-21T22:14:32.707 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f019c5c0 con 0x7f53f00fefb0 2023-07-21T22:14:32.707 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53f51a4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f019c750 con 0x7f53f0100680 2023-07-21T22:14:32.707 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53ef59e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 0x7f53f019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53ef59e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 0x7f53f019beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35402/0 (socket says 172.21.15.149:35402) 2023-07-21T22:14:32.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.658+0000 7f53ef59e700 1 -- 172.21.15.149:0/2965881050 learned_addr learned my addr 172.21.15.149:0/2965881050 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:14:32.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53eed9d700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f00fefb0 0x7f53f01948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ee59c700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 0x7f53f0194e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ef59e700 1 -- 172.21.15.149:0/2965881050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f00fefb0 msgr2=0x7f53f01948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ef59e700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f00fefb0 0x7f53f01948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ef59e700 1 -- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 msgr2=0x7f53f0194e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ef59e700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 0x7f53f0194e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ef59e700 1 -- 172.21.15.149:0/2965881050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40095f0 con 0x7f53f0100680 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ee59c700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 0x7f53f0194e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53eed9d700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f00fefb0 0x7f53f01948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53ef59e700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 0x7f53f019beb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b9a0 tx=0x7f53e800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:32.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e800c9b0 con 0x7f53f0100680 2023-07-21T22:14:32.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f019ca30 con 0x7f53f0100680 2023-07-21T22:14:32.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e8014440 con 0x7f53f0100680 2023-07-21T22:14:32.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80075d0 con 0x7f53f0100680 2023-07-21T22:14:32.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.659+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f019cfb0 con 0x7f53f0100680 2023-07-21T22:14:32.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.660+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f01060e0 con 0x7f53f0100680 2023-07-21T22:14:32.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.661+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f53e80145b0 con 0x7f53f0100680 2023-07-21T22:14:32.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.662+0000 7f53d7fff700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f53d80a10c0 0x7f53d80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:32.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.662+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f53e8006e40 con 0x7f53f0100680 2023-07-21T22:14:32.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.662+0000 7f53eed9d700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f53d80a10c0 0x7f53d80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:32.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.663+0000 7f53eed9d700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f53d80a10c0 0x7f53d80a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53e4006010 tx=0x7f53e400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:32.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.666+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f53e8095c30 con 0x7f53f0100680 2023-07-21T22:14:32.902 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:32.859+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f53f0060950 con 0x7f53d80a10c0 2023-07-21T22:14:33.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:14:33.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:14:33.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] boot 2023-07-21T22:14:33.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:14:33.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:33.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:33.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:33.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:33.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:33.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:14:33.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:14:33.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] boot 2023-07-21T22:14:33.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:14:33.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:33.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:33.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:33.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:33.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:33.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: osd.1 [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] boot 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:34.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:33 smithi104 ceph-mon[129021]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:34.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:33 smithi104 ceph-mon[129021]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:14:34.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:33 smithi066 ceph-mon[126379]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:34.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:33 smithi066 ceph-mon[126379]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:14:34.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:33 smithi149 ceph-mon[129268]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:34.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:33 smithi149 ceph-mon[129268]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T22:14:35.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:34 smithi104 ceph-mon[129021]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:35.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:34 smithi066 ceph-mon[126379]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:35.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:34 smithi149 ceph-mon[129268]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:36.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:35 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/564485370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]: dispatch 2023-07-21T22:14:36.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:35 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]: dispatch 2023-07-21T22:14:36.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:35 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]': finished 2023-07-21T22:14:36.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:35 smithi104 ceph-mon[129021]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:14:36.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:36.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:35 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/2881486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:14:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:35 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/564485370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]: dispatch 2023-07-21T22:14:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:35 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]: dispatch 2023-07-21T22:14:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:35 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]': finished 2023-07-21T22:14:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:35 smithi066 ceph-mon[126379]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:14:36.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:36.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:35 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/2881486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:14:36.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:35 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/564485370' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]: dispatch 2023-07-21T22:14:36.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:35 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]: dispatch 2023-07-21T22:14:36.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:35 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "502c7d03-af4c-4cbb-ab10-7bd70bb07c12"}]': finished 2023-07-21T22:14:36.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:35 smithi149 ceph-mon[129268]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T22:14:36.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:36.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:35 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/2881486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:14:37.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:36 smithi104 ceph-mon[129021]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:37.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:36 smithi066 ceph-mon[126379]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:37.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:36 smithi149 ceph-mon[129268]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:38.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:38 smithi149 ceph-mon[129268]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:39.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:38 smithi104 ceph-mon[129021]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:39.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:38 smithi066 ceph-mon[126379]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:41.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:41 smithi149 ceph-mon[129268]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:41.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:41 smithi104 ceph-mon[129021]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:41.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:41 smithi066 ceph-mon[126379]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:42.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:14:42.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:42.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:14:42.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:42.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T22:14:42.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:43.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:43 smithi149 ceph-mon[129268]: Deploying daemon osd.2 on smithi149 2023-07-21T22:14:43.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:43 smithi149 ceph-mon[129268]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:43.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:43 smithi104 ceph-mon[129021]: Deploying daemon osd.2 on smithi149 2023-07-21T22:14:43.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:43 smithi104 ceph-mon[129021]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:43.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:43 smithi066 ceph-mon[126379]: Deploying daemon osd.2 on smithi149 2023-07-21T22:14:43.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:43 smithi066 ceph-mon[126379]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:45.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:45 smithi149 ceph-mon[129268]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:45.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:45 smithi104 ceph-mon[129021]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:45.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:45 smithi066 ceph-mon[126379]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:47.192 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:47.193 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:47.193 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.193 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:47 smithi149 ceph-mon[129268]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:47.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:47.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:47.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:47 smithi104 ceph-mon[129021]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:47.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:47.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:47.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:47.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:47 smithi066 ceph-mon[126379]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:48.767 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 2 on host 'smithi149' 2023-07-21T22:14:48.768 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.724+0000 7f53d7fff700 1 -- 172.21.15.149:0/2965881050 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f53f0060950 con 0x7f53d80a10c0 2023-07-21T22:14:48.769 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f53d80a10c0 msgr2=0x7f53d80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:48.770 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f53d80a10c0 0x7f53d80a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f53e4006010 tx=0x7f53e400b540 comp rx=0 tx=0).stop 2023-07-21T22:14:48.770 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 msgr2=0x7f53f019beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:48.770 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 0x7f53f019beb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53e800b9a0 tx=0x7f53e800bd60 comp rx=0 tx=0).stop 2023-07-21T22:14:48.770 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 shutdown_connections 2023-07-21T22:14:48.770 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f53d80a10c0 0x7f53d80a3570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.770 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f00fefb0 0x7f53f01948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.771 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f0100680 0x7f53f019beb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.771 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 --2- 172.21.15.149:0/2965881050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f00ffcd0 0x7f53f0194e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:48.771 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 >> 172.21.15.149:0/2965881050 conn(0x7f53f00f9860 msgr2=0x7f53f0103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:48.771 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 shutdown_connections 2023-07-21T22:14:48.771 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:14:48.727+0000 7f53f51a4700 1 -- 172.21.15.149:0/2965881050 wait complete. 2023-07-21T22:14:48.917 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:48 smithi149 ceph-mon[129268]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:48.917 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:48.918 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:48.918 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:48.918 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:48.918 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:48 smithi104 ceph-mon[129021]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:49.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:49.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:49.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.169 DEBUG:teuthology.orchestra.run.smithi149:osd.2> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.2.service 2023-07-21T22:14:49.171 INFO:tasks.cephadm:Deploying osd.3 on smithi066 with /dev/vg_nvme/lv_3... 2023-07-21T22:14:49.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:14:49.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:48 smithi066 ceph-mon[126379]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:49.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:49.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:49.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.199 INFO:journalctl@ceph.osd.2.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:14:49.917 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:14:49.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:49.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:50.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:50.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:50.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:50.168 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:50.169 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:50.169 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:14:50.487 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2[135042]: 2023-07-21T22:14:50.021+0000 7f87f4d32640 -1 osd.2 0 log_to_monitors true 2023-07-21T22:14:50.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:50 smithi149 ceph-mon[129268]: from='osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:14:50.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:50 smithi149 ceph-mon[129268]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:14:50.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:50 smithi149 ceph-mon[129268]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:51.071 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:50 smithi066 ceph-mon[126379]: from='osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:14:51.071 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:50 smithi066 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:14:51.072 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:50 smithi066 ceph-mon[126379]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:51.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:50 smithi104 ceph-mon[129021]: from='osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:14:51.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:50 smithi104 ceph-mon[129021]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T22:14:51.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:50 smithi104 ceph-mon[129021]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:51.921 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:14:51.921 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:14:51.921 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-07-21T22:14:51.921 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-07-21T22:14:51.921 INFO:teuthology.orchestra.run.smithi066.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0306897 s, 342 MB/s 2023-07-21T22:14:51.922 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-07-21T22:14:52.097 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:14:52.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:14:52.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:52.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:14:52.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:14:52.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.099 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:52.099 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.099 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:52.099 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:52.099 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:14:52.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:14:52.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:52.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:14:52.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:14:52.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:52.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:52.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:52.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.238 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2[135042]: 2023-07-21T22:14:51.719+0000 7f87ea3aa700 -1 osd.2 0 waiting for initial osdmap 2023-07-21T22:14:52.238 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2[135042]: 2023-07-21T22:14:51.722+0000 7f87e619f700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:14:52.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T22:14:52.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:14:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:52.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:14:52.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:52.404 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi066:vg_nvme/lv_3 2023-07-21T22:14:53.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi149 2023-07-21T22:14:53.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi149 to 5879M 2023-07-21T22:14:53.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:14:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:14:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:52 smithi104 ceph-mon[129021]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:53.185 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:14:53.193 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi149 2023-07-21T22:14:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi149 to 5879M 2023-07-21T22:14:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:14:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:14:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:52 smithi066 ceph-mon[126379]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:53.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi149 2023-07-21T22:14:53.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi149 to 5879M 2023-07-21T22:14:53.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:14:53.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T22:14:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:52 smithi149 ceph-mon[129268]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T22:14:54.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:53 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:14:54.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:53 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:14:54.025 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:53 smithi066 ceph-mon[126379]: osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] boot 2023-07-21T22:14:54.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:53 smithi066 ceph-mon[126379]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:14:54.026 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:54.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:53 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:14:54.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:53 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:14:54.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:53 smithi104 ceph-mon[129021]: osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] boot 2023-07-21T22:14:54.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:53 smithi104 ceph-mon[129021]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:14:54.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:54.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:53 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:14:54.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:53 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:14:54.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:53 smithi149 ceph-mon[129268]: osd.2 [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] boot 2023-07-21T22:14:54.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:53 smithi149 ceph-mon[129268]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T22:14:54.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:14:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 -- 172.21.15.66:0/213659301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 msgr2=0x7f8c6c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/213659301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c107fe0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009f90 tx=0x7f8c6400b330 comp rx=0 tx=0).stop 2023-07-21T22:14:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 -- 172.21.15.66:0/213659301 shutdown_connections 2023-07-21T22:14:54.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/213659301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c107fe0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/213659301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c6c0ffb90 0x7f8c6c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/213659301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 0x7f8c6c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 -- 172.21.15.66:0/213659301 >> 172.21.15.66:0/213659301 conn(0x7f8c6c0fb1a0 msgr2=0x7f8c6c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:14:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 -- 172.21.15.66:0/213659301 shutdown_connections 2023-07-21T22:14:54.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.649+0000 7f8c70f4a700 1 -- 172.21.15.66:0/213659301 wait complete. 2023-07-21T22:14:54.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 Processor -- start 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 -- start start 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c6c0ffb90 0x7f8c6c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 0x7f8c6c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19f710 con 0x7f8c6c100d90 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19f850 con 0x7f8c6c101720 2023-07-21T22:14:54.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.651+0000 7f8c70f4a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19f9e0 con 0x7f8c6c0ffb90 2023-07-21T22:14:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c69d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 0x7f8c6c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c69d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 0x7f8c6c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56890/0 (socket says 172.21.15.66:56890) 2023-07-21T22:14:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c69d9b700 1 -- 172.21.15.66:0/3137172840 learned_addr learned my addr 172.21.15.66:0/3137172840 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:14:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6a59c700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c6c0ffb90 0x7f8c6c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 -- 172.21.15.66:0/3137172840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c6c0ffb90 msgr2=0x7f8c6c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c6c0ffb90 0x7f8c6c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 -- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 msgr2=0x7f8c6c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:14:54.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 0x7f8c6c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:14:54.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 -- 172.21.15.66:0/3137172840 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c0095f0 con 0x7f8c6c101720 2023-07-21T22:14:54.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.652+0000 7f8c6ad9d700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c19f140 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b6c0 tx=0x7f8c64004cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:54.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.653+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400e040 con 0x7f8c6c101720 2023-07-21T22:14:54.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.653+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c6400bbf0 con 0x7f8c6c101720 2023-07-21T22:14:54.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.653+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c64009c40 con 0x7f8c6c101720 2023-07-21T22:14:54.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.653+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640052a0 con 0x7f8c6c101720 2023-07-21T22:14:54.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.653+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c19ff90 con 0x7f8c6c101720 2023-07-21T22:14:54.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.654+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c192220 con 0x7f8c6c101720 2023-07-21T22:14:54.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.656+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f8c6401d470 con 0x7f8c6c101720 2023-07-21T22:14:54.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.657+0000 7f8c5b7fe700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c540a1140 0x7f8c540a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:14:54.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.658+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f8c640cc5d0 con 0x7f8c6c101720 2023-07-21T22:14:54.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.658+0000 7f8c6a59c700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c540a1140 0x7f8c540a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:14:54.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.658+0000 7f8c6a59c700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c540a1140 0x7f8c540a35f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c100bf0 tx=0x7f8c5c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:14:54.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.662+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8c64097340 con 0x7f8c6c101720 2023-07-21T22:14:54.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:54.900+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c6c060950 con 0x7f8c540a1140 2023-07-21T22:14:55.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:54 smithi149 ceph-mon[129268]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:14:55.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:54 smithi104 ceph-mon[129021]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:14:55.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:54 smithi066 ceph-mon[126379]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T22:14:56.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:14:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:14:56.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:14:56.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:14:56.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:14:56.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:14:56.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: pgmap v78: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='client.24185 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T22:14:56.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T22:14:56.374 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:14:57.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:14:56.982+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8c64097710 con 0x7f8c6c101720 2023-07-21T22:14:57.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:14:57.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:56 smithi066 ceph-mon[126379]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:14:57.108 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:56 smithi066 ceph-mon[126379]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:14:57.108 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:56 smithi066 sudo[141781]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:14:57.108 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:56 smithi066 sudo[141781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:57.108 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:56 smithi066 sudo[141781]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:57.108 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:56 smithi066 sudo[141781]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:14:57.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:14:57.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:56 smithi104 ceph-mon[129021]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:14:57.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:56 smithi104 ceph-mon[129021]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:14:57.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:14:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:56 smithi149 ceph-mon[129268]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T22:14:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:56 smithi149 ceph-mon[129268]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:14:57.944 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:57 smithi066 sudo[141823]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:14:57.944 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:57 smithi066 sudo[141823]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:57.945 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:57 smithi066 sudo[141823]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:57.945 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:14:57 smithi066 sudo[141823]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:14:58.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:57 smithi104 ceph-mon[129021]: osdmap e22: 3 total, 3 up, 3 in 2023-07-21T22:14:58.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:57 smithi104 ceph-mon[129021]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:14:58.129 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:14:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:58.129 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:57 smithi104 sudo[136329]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:14:58.130 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:57 smithi104 sudo[136329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:58.130 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:57 smithi104 sudo[136329]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:58.130 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:57 smithi104 sudo[136329]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:14:58.395 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:57 smithi149 ceph-mon[129268]: osdmap e22: 3 total, 3 up, 3 in 2023-07-21T22:14:58.395 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:57 smithi149 ceph-mon[129268]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:14:58.395 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:14:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:58.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:57 smithi066 ceph-mon[126379]: osdmap e22: 3 total, 3 up, 3 in 2023-07-21T22:14:58.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:57 smithi066 ceph-mon[126379]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T22:14:58.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:14:58.617 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:58 smithi104 sudo[136332]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:14:58.617 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:58 smithi104 sudo[136332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:58.618 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:58 smithi104 sudo[136332]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:58.618 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:14:58 smithi104 sudo[136332]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:14:58.649 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137172]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:14:58.649 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:58.649 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137172]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:58.650 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137172]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:14:58.987 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137175]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:14:58.988 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137175]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:58.988 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137175]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:58.988 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:14:58 smithi149 sudo[137175]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:14:59.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:59 smithi066 sudo[141938]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:14:59.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:59 smithi066 sudo[141938]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:14:59.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:59 smithi066 sudo[141938]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:14:59.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:14:59 smithi066 sudo[141938]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:15:00.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3666125701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3f495e8-3d23-4125-96ea-0e4459f72a13"}]: dispatch 2023-07-21T22:15:00.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3666125701' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3f495e8-3d23-4125-96ea-0e4459f72a13"}]': finished 2023-07-21T22:15:00.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:15:00.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:00.432 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:00.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:00.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:00.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137178]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:15:00.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137178]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:15:00.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137178]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:15:00.433 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137178]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:15:00.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 sudo[142315]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 sudo[142315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 sudo[142315]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 sudo[142315]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3666125701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3f495e8-3d23-4125-96ea-0e4459f72a13"}]: dispatch 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3666125701' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3f495e8-3d23-4125-96ea-0e4459f72a13"}]': finished 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:15:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:00.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:00.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:00.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:00.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3666125701' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3f495e8-3d23-4125-96ea-0e4459f72a13"}]: dispatch 2023-07-21T22:15:00.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3666125701' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3f495e8-3d23-4125-96ea-0e4459f72a13"}]': finished 2023-07-21T22:15:00.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T22:15:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:00.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137181]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:15:00.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137181]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:15:00.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137181]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:15:00.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:00 smithi149 sudo[137181]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:15:01.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 sudo[136335]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T22:15:01.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 sudo[136335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:15:01.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 sudo[136335]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:15:01.163 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:00 smithi104 sudo[136335]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:15:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3812906180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:01 smithi066 ceph-mon[126379]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3812906180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:01 smithi149 ceph-mon[129268]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 sudo[136338]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T22:15:01.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 sudo[136338]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 sudo[136338]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 sudo[136338]: pam_unix(sudo:session): session closed for user root 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3812906180' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:01.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T22:15:01.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:15:01.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:15:01.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:15:01.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:01 smithi104 ceph-mon[129021]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T22:15:03.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:03 smithi066 ceph-mon[126379]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:03.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:03 smithi149 ceph-mon[129268]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:03.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:03 smithi104 ceph-mon[129021]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:05.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:05 smithi066 ceph-mon[126379]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.9 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-07-21T22:15:05.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:05 smithi149 ceph-mon[129268]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.9 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-07-21T22:15:05.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:05 smithi104 ceph-mon[129021]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.9 KiB/s rd, 213 KiB/s wr, 14 op/s 2023-07-21T22:15:06.465 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:15:06.465 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:06.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:15:06.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:06.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T22:15:06.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:07.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:07 smithi149 ceph-mon[129268]: Deploying daemon osd.3 on smithi066 2023-07-21T22:15:07.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:07 smithi149 ceph-mon[129268]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-07-21T22:15:07.512 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:07 smithi066 ceph-mon[126379]: Deploying daemon osd.3 on smithi066 2023-07-21T22:15:07.513 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:07 smithi066 ceph-mon[126379]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-07-21T22:15:07.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:07 smithi104 ceph-mon[129021]: Deploying daemon osd.3 on smithi066 2023-07-21T22:15:07.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:07 smithi104 ceph-mon[129021]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 160 KiB/s wr, 11 op/s 2023-07-21T22:15:08.796 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:08 smithi066 ceph-mon[126379]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 129 KiB/s wr, 8 op/s 2023-07-21T22:15:08.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:08 smithi149 ceph-mon[129268]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 129 KiB/s wr, 8 op/s 2023-07-21T22:15:09.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:08 smithi104 ceph-mon[129021]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 129 KiB/s wr, 8 op/s 2023-07-21T22:15:11.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:11 smithi149 ceph-mon[129268]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:11.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:11 smithi104 ceph-mon[129021]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:11.689 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:11 smithi066 ceph-mon[126379]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:13.320 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:13.321 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:13.321 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:13.321 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:12 smithi066 ceph-mon[126379]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:13.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:13.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:13.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:13.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:12 smithi104 ceph-mon[129021]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:13.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:13.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:13.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:13.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:12 smithi149 ceph-mon[129268]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:15.461 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 3 on host 'smithi066' 2023-07-21T22:15:15.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.416+0000 7f8c5b7fe700 1 -- 172.21.15.66:0/3137172840 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8c6c060950 con 0x7f8c540a1140 2023-07-21T22:15:15.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.419+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c540a1140 msgr2=0x7f8c540a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:15.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.419+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c540a1140 0x7f8c540a35f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c100bf0 tx=0x7f8c5c009310 comp rx=0 tx=0).stop 2023-07-21T22:15:15.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 msgr2=0x7f8c6c19f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c19f140 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400b6c0 tx=0x7f8c64004cc0 comp rx=0 tx=0).stop 2023-07-21T22:15:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 shutdown_connections 2023-07-21T22:15:15.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c540a1140 0x7f8c540a35f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c6c101720 0x7f8c6c19f140 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c6c0ffb90 0x7f8c6c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 --2- 172.21.15.66:0/3137172840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c6c100d90 0x7f8c6c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 >> 172.21.15.66:0/3137172840 conn(0x7f8c6c0fb1a0 msgr2=0x7f8c6c106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 shutdown_connections 2023-07-21T22:15:15.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:15:15.420+0000 7f8c70f4a700 1 -- 172.21.15.66:0/3137172840 wait complete. 2023-07-21T22:15:15.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:15 smithi149 ceph-mon[129268]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:15.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:15.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:15.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.489 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:15 smithi066 ceph-mon[126379]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:15.489 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.489 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.489 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:15.490 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:15.490 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:15 smithi104 ceph-mon[129021]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T22:15:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:15.916 DEBUG:teuthology.orchestra.run.smithi066:osd.3> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.3.service 2023-07-21T22:15:15.921 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2023-07-21T22:15:15.921 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:15:16.006 INFO:journalctl@ceph.osd.3.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:15:16.481 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:15:16.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:16 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:16.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:16 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:16.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:16 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:16.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:16 smithi066 ceph-mon[126379]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:15:16.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:16 smithi066 ceph-mon[126379]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:16.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:16.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:16.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:16 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:16.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:16 smithi149 ceph-mon[129268]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:15:16.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:16 smithi149 ceph-mon[129268]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:16.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:16 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:16.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:16 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:16.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:16 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:16.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:16 smithi104 ceph-mon[129021]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T22:15:16.739 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:16 smithi104 ceph-mon[129021]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:17.611 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:17 smithi104 ceph-mon[129021]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:15:17.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:17 smithi104 ceph-mon[129021]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:15:17.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:17 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:17.612 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:17 smithi104 ceph-mon[129021]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:15:17.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:17 smithi066 ceph-mon[126379]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:15:17.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:17 smithi066 ceph-mon[126379]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:15:17.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:17 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:17.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:17 smithi066 ceph-mon[126379]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:15:17.694 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:15:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-3[144116]: 2023-07-21T22:15:17.383+0000 7f5be314f700 -1 osd.3 0 waiting for initial osdmap 2023-07-21T22:15:17.695 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:15:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-3[144116]: 2023-07-21T22:15:17.392+0000 7f5bdb73d700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:15:17.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:17 smithi149 ceph-mon[129268]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T22:15:17.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:17 smithi149 ceph-mon[129268]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T22:15:17.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:17 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:17.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:17 smithi149 ceph-mon[129268]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:15:18.058 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:15:18.059 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:15:18.059 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2023-07-21T22:15:18.059 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2023-07-21T22:15:18.059 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137263 s, 764 MB/s 2023-07-21T22:15:18.059 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2023-07-21T22:15:18.424 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2023-07-21T22:15:18.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:15:18.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: osdmap e25: 4 total, 3 up, 4 in 2023-07-21T22:15:18.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.627 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] boot 2023-07-21T22:15:18.627 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:15:18.627 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.627 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:18 smithi066 ceph-mon[126379]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:18.737 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: osdmap e25: 4 total, 3 up, 4 in 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] boot 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:18 smithi149 ceph-mon[129268]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:18.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: from='osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:15:18.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: osdmap e25: 4 total, 3 up, 4 in 2023-07-21T22:15:18.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: osd.3 [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] boot 2023-07-21T22:15:18.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T22:15:18.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:15:18.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:18 smithi104 ceph-mon[129021]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T22:15:18.974 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:15:19.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi066 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:19.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:19.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi066 to 8633M 2023-07-21T22:15:19.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:19.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:19.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:15:19.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:15:19.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:15:19.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi066 2023-07-21T22:15:19.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi066 to 8633M 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:19.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:20.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:15:20.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:15:20.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T22:15:20.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi066 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi066 to 8633M 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:20.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:20.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 -- 172.21.15.104:0/2895892559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc970100020 msgr2=0x7fc970100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 --2- 172.21.15.104:0/2895892559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc970100020 0x7fc970100430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc960009990 tx=0x7fc960009ca0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 -- 172.21.15.104:0/2895892559 shutdown_connections 2023-07-21T22:15:20.213 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 --2- 172.21.15.104:0/2895892559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc970100020 0x7fc970100430 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 --2- 172.21.15.104:0/2895892559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9700ff6f0 0x7fc970106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 --2- 172.21.15.104:0/2895892559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc9700fed60 0x7fc9700ff1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 -- 172.21.15.104:0/2895892559 >> 172.21.15.104:0/2895892559 conn(0x7fc9700f9860 msgr2=0x7fc9700fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 -- 172.21.15.104:0/2895892559 shutdown_connections 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.169+0000 7fc977aab700 1 -- 172.21.15.104:0/2895892559 wait complete. 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 Processor -- start 2023-07-21T22:15:20.214 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 -- start start 2023-07-21T22:15:20.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 0x7fc970194900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.217 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9700ff6f0 0x7fc970194e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc970100020 0x7fc97019bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97019c490 con 0x7fc9700ff6f0 2023-07-21T22:15:20.218 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97019c5d0 con 0x7fc9700fed60 2023-07-21T22:15:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc977aab700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97019c760 con 0x7fc970100020 2023-07-21T22:15:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc975847700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 0x7fc970194900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc975847700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 0x7fc970194900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:56898/0 (socket says 172.21.15.104:56898) 2023-07-21T22:15:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc975847700 1 -- 172.21.15.104:0/3760919125 learned_addr learned my addr 172.21.15.104:0/3760919125 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-21T22:15:20.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc976048700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc970100020 0x7fc97019bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.170+0000 7fc975046700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9700ff6f0 0x7fc970194e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.220 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc975847700 1 -- 172.21.15.104:0/3760919125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc970100020 msgr2=0x7fc97019bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc975847700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc970100020 0x7fc97019bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc975847700 1 -- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9700ff6f0 msgr2=0x7fc970194e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc975847700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9700ff6f0 0x7fc970194e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc975847700 1 -- 172.21.15.104:0/3760919125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600095f0 con 0x7fc9700fed60 2023-07-21T22:15:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc975847700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 0x7fc970194900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc96000b5c0 tx=0x7fc96001a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96001f070 con 0x7fc9700fed60 2023-07-21T22:15:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc960003e20 con 0x7fc9700fed60 2023-07-21T22:15:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9700fdc50 con 0x7fc9700fed60 2023-07-21T22:15:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960026470 con 0x7fc9700fed60 2023-07-21T22:15:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.171+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9700fe1c0 con 0x7fc9700fed60 2023-07-21T22:15:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.172+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97010a360 con 0x7fc9700fed60 2023-07-21T22:15:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.173+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc960023070 con 0x7fc9700fed60 2023-07-21T22:15:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.173+0000 7fc966ffd700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc95c0a11d0 0x7fc95c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:20.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.174+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fc9600ceb40 con 0x7fc9700fed60 2023-07-21T22:15:20.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.174+0000 7fc975046700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc95c0a11d0 0x7fc95c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:20.224 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.175+0000 7fc975046700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc95c0a11d0 0x7fc95c0a3680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006fd0 tx=0x7fc96c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:20.225 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.178+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc960099520 con 0x7fc9700fed60 2023-07-21T22:15:20.414 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:20.371+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc970060950 con 0x7fc95c0a11d0 2023-07-21T22:15:20.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:20 smithi149 ceph-mon[129268]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:15:20.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:20 smithi149 ceph-mon[129268]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:20.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:20 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:15:20.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:20 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:15:20.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:20 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:21.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:20 smithi104 ceph-mon[129021]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:15:21.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:20 smithi104 ceph-mon[129021]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:21.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:20 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:15:21.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:20 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:15:21.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:20 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:21.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:20 smithi066 ceph-mon[126379]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T22:15:21.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:20 smithi066 ceph-mon[126379]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:21.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:20 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:15:21.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:20 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:15:21.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:20 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:21.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:21 smithi104 ceph-mon[129021]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:21.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:21 smithi104 ceph-mon[129021]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T22:15:21.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:21 smithi149 ceph-mon[129268]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:21.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:21 smithi149 ceph-mon[129268]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T22:15:22.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:21 smithi066 ceph-mon[126379]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:22.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:21 smithi066 ceph-mon[126379]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T22:15:22.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: osdmap e30: 4 total, 4 up, 4 in 2023-07-21T22:15:22.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:22.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/164659801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]: dispatch 2023-07-21T22:15:22.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]: dispatch 2023-07-21T22:15:22.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]': finished 2023-07-21T22:15:22.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:15:22.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:22 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:23.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: osdmap e30: 4 total, 4 up, 4 in 2023-07-21T22:15:23.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:23.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/164659801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]: dispatch 2023-07-21T22:15:23.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]: dispatch 2023-07-21T22:15:23.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]': finished 2023-07-21T22:15:23.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:15:23.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:22 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:23.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: osdmap e30: 4 total, 4 up, 4 in 2023-07-21T22:15:23.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:23.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/164659801' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]: dispatch 2023-07-21T22:15:23.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]: dispatch 2023-07-21T22:15:23.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0864a013-76a0-4e35-91a1-2405091d259e"}]': finished 2023-07-21T22:15:23.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T22:15:23.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:22 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:23.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:23 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/1142398820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:24.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:23 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/1142398820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:24.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:23 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/1142398820' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:25.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:24 smithi104 ceph-mon[129021]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-07-21T22:15:25.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:24 smithi066 ceph-mon[126379]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-07-21T22:15:25.237 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:24 smithi149 ceph-mon[129268]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2023-07-21T22:15:27.487 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:27 smithi149 ceph-mon[129268]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:15:27.559 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:27 smithi104 ceph-mon[129021]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:15:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:27 smithi066 ceph-mon[126379]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:15:28.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:28 smithi149 ceph-mon[129268]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-21T22:15:29.114 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:28 smithi104 ceph-mon[129021]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-21T22:15:29.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:28 smithi066 ceph-mon[126379]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-07-21T22:15:29.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:15:29.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:29.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:29 smithi149 ceph-mon[129268]: Deploying daemon osd.4 on smithi104 2023-07-21T22:15:30.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:15:30.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:30.029 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:29 smithi104 ceph-mon[129021]: Deploying daemon osd.4 on smithi104 2023-07-21T22:15:30.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T22:15:30.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:30.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:29 smithi066 ceph-mon[126379]: Deploying daemon osd.4 on smithi104 2023-07-21T22:15:30.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:30 smithi104 ceph-mon[129021]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-21T22:15:30.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:30 smithi149 ceph-mon[129268]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-21T22:15:31.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:30 smithi066 ceph-mon[126379]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-07-21T22:15:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:33 smithi149 ceph-mon[129268]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:15:33.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:33 smithi104 ceph-mon[129021]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:15:33.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:33 smithi066 ceph-mon[126379]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:15:34.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:34 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:34.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:34 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:34.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:34 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:34 smithi104 ceph-mon[129021]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-07-21T22:15:34.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:34 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:34.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:34 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:34.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:34 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:34.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:34 smithi066 ceph-mon[126379]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-07-21T22:15:34.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:34 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:34.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:34 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:34.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:34 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:34.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:34 smithi149 ceph-mon[129268]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-07-21T22:15:36.331 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.288+0000 7fc966ffd700 1 -- 172.21.15.104:0/3760919125 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc970060950 con 0x7fc95c0a11d0 2023-07-21T22:15:36.332 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2023-07-21T22:15:36.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc95c0a11d0 msgr2=0x7fc95c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:36.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc95c0a11d0 0x7fc95c0a3680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006fd0 tx=0x7fc96c008040 comp rx=0 tx=0).stop 2023-07-21T22:15:36.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 msgr2=0x7fc970194900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:36.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 0x7fc970194900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc96000b5c0 tx=0x7fc96001a8b0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 shutdown_connections 2023-07-21T22:15:36.334 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc95c0a11d0 0x7fc95c0a3680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9700fed60 0x7fc970194900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc970100020 0x7fc97019bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 --2- 172.21.15.104:0/3760919125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc9700ff6f0 0x7fc970194e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:36.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 >> 172.21.15.104:0/3760919125 conn(0x7fc9700f9860 msgr2=0x7fc9700fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:36.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 shutdown_connections 2023-07-21T22:15:36.335 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:15:36.290+0000 7fc977aab700 1 -- 172.21.15.104:0/3760919125 wait complete. 2023-07-21T22:15:36.771 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.4.service 2023-07-21T22:15:36.775 INFO:tasks.cephadm:Deploying osd.5 on smithi149 with /dev/vg_nvme/lv_3... 2023-07-21T22:15:36.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T22:15:36.829 INFO:journalctl@ceph.osd.4.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:15:36.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:36.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:36.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:36.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:36.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:36.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:36.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:36.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:36.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:37.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:37.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:37.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:37.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:37.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:37.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:37.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:37.350 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:15:37.740 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:15:37 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:15:37.463+0000 7f0f70b29640 -1 osd.4 0 log_to_monitors true 2023-07-21T22:15:37.877 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:37 smithi149 ceph-mon[129268]: from='osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:15:37.877 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:37 smithi149 ceph-mon[129268]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:15:38.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:37 smithi104 ceph-mon[129021]: from='osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:15:38.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:37 smithi104 ceph-mon[129021]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:15:38.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:37 smithi066 ceph-mon[126379]: from='osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:15:38.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:37 smithi066 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T22:15:38.882 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T22:15:38.883 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T22:15:38.883 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-07-21T22:15:38.883 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-07-21T22:15:38.883 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110632 s, 948 MB/s 2023-07-21T22:15:38.883 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-07-21T22:15:39.043 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:38 smithi149 ceph-mon[129268]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:15:39.043 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:38 smithi149 ceph-mon[129268]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T22:15:39.043 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:39.043 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:38 smithi149 ceph-mon[129268]: from='osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:15:39.044 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:38 smithi149 ceph-mon[129268]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:15:39.044 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:38 smithi149 ceph-mon[129268]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-mon[129021]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:15:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-mon[129021]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T22:15:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-mon[129021]: from='osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:15:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-mon[129021]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:15:39.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-mon[129021]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:39.119 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:15:38.712+0000 7f0f661a1700 -1 osd.4 0 waiting for initial osdmap 2023-07-21T22:15:39.119 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:15:38 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:15:38.717+0000 7f0f5ef90700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:15:39.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:38 smithi066 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T22:15:39.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:38 smithi066 ceph-mon[126379]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T22:15:39.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:38 smithi066 ceph-mon[126379]: from='osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:15:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:38 smithi066 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:15:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:38 smithi066 ceph-mon[126379]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:39.306 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2023-07-21T22:15:39.863 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:15:40.083 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: osdmap e33: 5 total, 4 up, 5 in 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi104 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.084 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:40.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:40.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi104 to 8584M 2023-07-21T22:15:40.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:40.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:40.085 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:39 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:15:40.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: osdmap e33: 5 total, 4 up, 5 in 2023-07-21T22:15:40.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:40.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:40.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi104 2023-07-21T22:15:40.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi104 to 8584M 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:40.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:39 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:15:40.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: osdmap e33: 5 total, 4 up, 5 in 2023-07-21T22:15:40.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:40.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi104 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi104 to 8584M 2023-07-21T22:15:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:40.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:40.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:40.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:39 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:41.092 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 -- 172.21.15.149:0/2721631661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c8101740 msgr2=0x7fb5c8108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.092 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/2721631661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c8101740 0x7fb5c8108000 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000b0a0 tx=0x7fb5c000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.092 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 -- 172.21.15.149:0/2721631661 shutdown_connections 2023-07-21T22:15:41.092 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/2721631661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.093 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/2721631661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c8101740 0x7fb5c8108000 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.093 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/2721631661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c8100db0 0x7fb5c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.093 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 -- 172.21.15.149:0/2721631661 >> 172.21.15.149:0/2721631661 conn(0x7fb5c80fb1c0 msgr2=0x7fb5c80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:41.093 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 -- 172.21.15.149:0/2721631661 shutdown_connections 2023-07-21T22:15:41.093 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.048+0000 7fb5cdda2700 1 -- 172.21.15.149:0/2721631661 wait complete. 2023-07-21T22:15:41.093 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 Processor -- start 2023-07-21T22:15:41.096 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 -- start start 2023-07-21T22:15:41.096 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c8195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.096 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c8100db0 0x7fb5c8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.096 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5c8101740 0x7fb5c819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.096 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c819d5d0 con 0x7fb5c8100db0 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c819d710 con 0x7fb5c8101740 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5cdda2700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c819d8a0 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.049+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c8195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c8195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35478/0 (socket says 172.21.15.149:35478) 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 -- 172.21.15.149:0/1657918840 learned_addr learned my addr 172.21.15.149:0/1657918840 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c6ffd700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c8100db0 0x7fb5c8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.097 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c7fff700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5c8101740 0x7fb5c819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 -- 172.21.15.149:0/1657918840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5c8101740 msgr2=0x7fb5c819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5c8101740 0x7fb5c819d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 -- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c8100db0 msgr2=0x7fb5c8195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c8100db0 0x7fb5c8195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 -- 172.21.15.149:0/1657918840 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009d00 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c77fe700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c8195a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000e980 tx=0x7fb5b000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.098 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0009770 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5b00044f0 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0003e40 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c819db80 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.050+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c819e030 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.051+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c8107100 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.052+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb5b0004000 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.099 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.053+0000 7fb5c4ff9700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb5b40a11d0 0x7fb5b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:15:41.100 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.053+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fb5b0014070 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.100 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.053+0000 7fb5c6ffd700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb5b40a11d0 0x7fb5b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:15:41.100 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.054+0000 7fb5c6ffd700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb5b40a11d0 0x7fb5b40a3680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006fd0 tx=0x7fb5b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:15:41.100 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.056+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb5b0099e80 con 0x7fb5c80ffbb0 2023-07-21T22:15:41.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:40 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:15:41.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:40 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:15:41.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:40 smithi104 ceph-mon[129021]: osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] boot 2023-07-21T22:15:41.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:40 smithi104 ceph-mon[129021]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T22:15:41.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:41.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:40 smithi104 ceph-mon[129021]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:41.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:40 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:15:41.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:40 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:15:41.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:40 smithi066 ceph-mon[126379]: osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] boot 2023-07-21T22:15:41.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:40 smithi066 ceph-mon[126379]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T22:15:41.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:41.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:40 smithi066 ceph-mon[126379]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:41.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:40 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:15:41.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:40 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:15:41.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:40 smithi149 ceph-mon[129268]: osd.4 [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] boot 2023-07-21T22:15:41.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:40 smithi149 ceph-mon[129268]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T22:15:41.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:15:41.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:40 smithi149 ceph-mon[129268]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T22:15:41.296 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:41.252+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5c8060950 con 0x7fb5b40a11d0 2023-07-21T22:15:42.106 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:41 smithi149 ceph-mon[129268]: osdmap e35: 5 total, 5 up, 5 in 2023-07-21T22:15:42.106 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:41 smithi149 ceph-mon[129268]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:42.107 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:41 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:15:42.107 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:41 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:15:42.107 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:41 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:42.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:41 smithi104 ceph-mon[129021]: osdmap e35: 5 total, 5 up, 5 in 2023-07-21T22:15:42.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:41 smithi104 ceph-mon[129021]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:42.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:41 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:15:42.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:41 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:15:42.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:41 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:42.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:41 smithi066 ceph-mon[126379]: osdmap e35: 5 total, 5 up, 5 in 2023-07-21T22:15:42.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:41 smithi066 ceph-mon[126379]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:15:42.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:41 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:15:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:41 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:15:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:41 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:43.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:42 smithi104 ceph-mon[129021]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:43.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:42 smithi066 ceph-mon[126379]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:43.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:42 smithi149 ceph-mon[129268]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:44.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:43 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/2815802149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]: dispatch 2023-07-21T22:15:44.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:43 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]: dispatch 2023-07-21T22:15:44.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:43 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]': finished 2023-07-21T22:15:44.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:43 smithi104 ceph-mon[129021]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:15:44.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:43 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:15:44.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:43 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/2815802149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]: dispatch 2023-07-21T22:15:44.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:43 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]: dispatch 2023-07-21T22:15:44.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:43 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]': finished 2023-07-21T22:15:44.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:43 smithi066 ceph-mon[126379]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:15:44.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:43 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:15:44.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:43 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/2815802149' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]: dispatch 2023-07-21T22:15:44.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:43 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]: dispatch 2023-07-21T22:15:44.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:43 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d8b253aa-c95f-488c-af70-0d2f6a83e718"}]': finished 2023-07-21T22:15:44.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:43 smithi149 ceph-mon[129268]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T22:15:44.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:43 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:15:45.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:44 smithi104 ceph-mon[129021]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:45.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:44 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/374520152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:45.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:44 smithi066 ceph-mon[126379]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:45.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:44 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/374520152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:45.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:44 smithi149 ceph-mon[129268]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:45.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:44 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/374520152' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:15:47.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:47 smithi149 ceph-mon[129268]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:47.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:47 smithi104 ceph-mon[129021]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:47.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:47 smithi066 ceph-mon[126379]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:48.987 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:48 smithi149 ceph-mon[129268]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:49.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:48 smithi104 ceph-mon[129021]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:49.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:48 smithi066 ceph-mon[126379]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:50.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:50 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:15:50.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:50 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:50 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:15:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:50 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:50.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:50 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T22:15:50.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:50 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:51.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:51 smithi066 ceph-mon[126379]: Deploying daemon osd.5 on smithi149 2023-07-21T22:15:51.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:51 smithi066 ceph-mon[126379]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:51.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:51 smithi149 ceph-mon[129268]: Deploying daemon osd.5 on smithi149 2023-07-21T22:15:51.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:51 smithi149 ceph-mon[129268]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:51.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:51 smithi104 ceph-mon[129021]: Deploying daemon osd.5 on smithi149 2023-07-21T22:15:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:51 smithi104 ceph-mon[129021]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:53.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:53 smithi149 ceph-mon[129268]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:53.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:53 smithi104 ceph-mon[129021]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:53.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:53 smithi066 ceph-mon[126379]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:55.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:55 smithi149 ceph-mon[129268]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:55.590 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:55.591 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:55.591 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:55 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:55.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:55 smithi104 ceph-mon[129021]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:55.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:55 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:55.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:55 smithi066 ceph-mon[126379]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:55.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:55.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:55.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:55 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:57.261 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.217+0000 7fb5c4ff9700 1 -- 172.21.15.149:0/1657918840 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb5c8060950 con 0x7fb5b40a11d0 2023-07-21T22:15:57.261 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 5 on host 'smithi149' 2023-07-21T22:15:57.263 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb5b40a11d0 msgr2=0x7fb5b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.263 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb5b40a11d0 0x7fb5b40a3680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006fd0 tx=0x7fb5b8008040 comp rx=0 tx=0).stop 2023-07-21T22:15:57.264 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 msgr2=0x7fb5c8195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:15:57.264 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c8195a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000e980 tx=0x7fb5b000ed40 comp rx=0 tx=0).stop 2023-07-21T22:15:57.264 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 shutdown_connections 2023-07-21T22:15:57.264 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb5b40a11d0 0x7fb5b40a3680 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.264 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5c8101740 0x7fb5c819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.264 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5c80ffbb0 0x7fb5c8195a40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.265 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 --2- 172.21.15.149:0/1657918840 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5c8100db0 0x7fb5c8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:15:57.265 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 >> 172.21.15.149:0/1657918840 conn(0x7fb5c80fb1c0 msgr2=0x7fb5c8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:15:57.265 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 shutdown_connections 2023-07-21T22:15:57.265 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:15:57.219+0000 7fb5cdda2700 1 -- 172.21.15.149:0/1657918840 wait complete. 2023-07-21T22:15:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:57.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:57.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:57.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.680 DEBUG:teuthology.orchestra.run.smithi149:osd.5> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.5.service 2023-07-21T22:15:57.683 INFO:tasks.cephadm:Deploying osd.6 on smithi066 with /dev/vg_nvme/lv_2... 2023-07-21T22:15:57.683 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:15:57.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:57.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:15:57.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:15:57.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:15:57.715 INFO:journalctl@ceph.osd.5.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:15:58.450 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:15:58.465 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:58.465 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:15:58 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:15:58.304+0000 7f6557249640 -1 osd.5 0 log_to_monitors true 2023-07-21T22:15:58.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:58.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:15:59.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:59 smithi066 ceph-mon[126379]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:59.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:59 smithi066 ceph-mon[126379]: from='osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:15:59.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:15:59 smithi066 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:15:59.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:59 smithi149 ceph-mon[129268]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:59.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:59 smithi149 ceph-mon[129268]: from='osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:15:59.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:15:59 smithi149 ceph-mon[129268]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:15:59.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:59 smithi104 ceph-mon[129021]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:15:59.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:59 smithi104 ceph-mon[129021]: from='osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:15:59.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:15:59 smithi104 ceph-mon[129021]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T22:16:00.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:16:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T22:16:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:16:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:16:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:00 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.634 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:16:00.635 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:16:00.635 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-07-21T22:16:00.635 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-07-21T22:16:00.635 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.031872 s, 329 MB/s 2023-07-21T22:16:00.635 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-07-21T22:16:00.672 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:16:00.672 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T22:16:00.672 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:00.672 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.673 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:00.674 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:00.674 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:00 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.738 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:16:00.223+0000 7f654c8c1700 -1 osd.5 0 waiting for initial osdmap 2023-07-21T22:16:00.738 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:16:00.229+0000 7f65486b6700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:16:00.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T22:16:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T22:16:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:16:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:16:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:00 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:01.141 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi066:vg_nvme/lv_2 2023-07-21T22:16:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi149 2023-07-21T22:16:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi149 to 2939M 2023-07-21T22:16:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:16:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:16:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: osdmap e38: 6 total, 5 up, 6 in 2023-07-21T22:16:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:01.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi149 2023-07-21T22:16:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi149 to 2939M 2023-07-21T22:16:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:16:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:16:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: osdmap e38: 6 total, 5 up, 6 in 2023-07-21T22:16:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:01.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi149 2023-07-21T22:16:01.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi149 to 2939M 2023-07-21T22:16:01.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:16:01.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:16:01.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: osdmap e38: 6 total, 5 up, 6 in 2023-07-21T22:16:01.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:01.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:01.925 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:16:02.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:16:02.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:16:02.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] boot 2023-07-21T22:16:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:16:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:02 smithi066 ceph-mon[126379]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:16:02.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:16:02.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:16:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] boot 2023-07-21T22:16:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:16:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:02 smithi104 ceph-mon[129021]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:16:02.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:16:02.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:16:02.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:02.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: osd.5 [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] boot 2023-07-21T22:16:02.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T22:16:02.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:16:02.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:02 smithi149 ceph-mon[129268]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T22:16:03.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 -- 172.21.15.66:0/3000277392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8008101720 msgr2=0x7f8008107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 --2- 172.21.15.66:0/3000277392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8008101720 0x7f8008107fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8000009f90 tx=0x7f800000b330 comp rx=0 tx=0).stop 2023-07-21T22:16:03.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 -- 172.21.15.66:0/3000277392 shutdown_connections 2023-07-21T22:16:03.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 --2- 172.21.15.66:0/3000277392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80080ffb90 0x7f80080fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 --2- 172.21.15.66:0/3000277392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8008101720 0x7f8008107fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 --2- 172.21.15.66:0/3000277392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008100d90 0x7f80081011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 -- 172.21.15.66:0/3000277392 >> 172.21.15.66:0/3000277392 conn(0x7f80080fb1a0 msgr2=0x7f80080fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:03.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 -- 172.21.15.66:0/3000277392 shutdown_connections 2023-07-21T22:16:03.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.277+0000 7f800c983700 1 -- 172.21.15.66:0/3000277392 wait complete. 2023-07-21T22:16:03.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.278+0000 7f800c983700 1 Processor -- start 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 -- start start 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80080ffb90 0x7f8008197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8008100d90 0x7f8008198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008101720 0x7f800819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800819f780 con 0x7f80080ffb90 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800819f8c0 con 0x7f8008100d90 2023-07-21T22:16:03.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800c983700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800819fa80 con 0x7f8008101720 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f8005d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8008100d90 0x7f8008198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f8006d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008101720 0x7f800819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f8005d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8008100d90 0x7f8008198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57216/0 (socket says 172.21.15.66:57216) 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f8005d9b700 1 -- 172.21.15.66:0/884315370 learned_addr learned my addr 172.21.15.66:0/884315370 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.279+0000 7f800659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80080ffb90 0x7f8008197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f8006d9d700 1 -- 172.21.15.66:0/884315370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8008100d90 msgr2=0x7f8008198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f8006d9d700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8008100d90 0x7f8008198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f8006d9d700 1 -- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80080ffb90 msgr2=0x7f8008197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f8006d9d700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80080ffb90 0x7f8008197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f8006d9d700 1 -- 172.21.15.66:0/884315370 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009c40 con 0x7f8008101720 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f8006d9d700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008101720 0x7f800819f1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f800000b6c0 tx=0x7f8000004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800000e040 con 0x7f8008101720 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800819fd00 con 0x7f8008101720 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f800000bbf0 con 0x7f8008101720 2023-07-21T22:16:03.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000005040 con 0x7f8008101720 2023-07-21T22:16:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.280+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80081a01c0 con 0x7f8008101720 2023-07-21T22:16:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.281+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800810b3b0 con 0x7f8008101720 2023-07-21T22:16:03.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.284+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f800001d470 con 0x7f8008101720 2023-07-21T22:16:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.285+0000 7f7ff77fe700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7ff00a1150 0x7f7ff00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.285+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f80000cd2e0 con 0x7f8008101720 2023-07-21T22:16:03.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.285+0000 7f800659c700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7ff00a1150 0x7f7ff00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:03.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.286+0000 7f800659c700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7ff00a1150 0x7f7ff00a3600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8008199110 tx=0x7f7ff8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:03.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.288+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f80000978c0 con 0x7f8008101720 2023-07-21T22:16:03.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:03 smithi066 ceph-mon[126379]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:16:03.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:03 smithi066 ceph-mon[126379]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:16:03.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:03.503+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f800810b5d0 con 0x7f7ff00a1150 2023-07-21T22:16:03.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:03 smithi104 ceph-mon[129021]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:16:03.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:03 smithi104 ceph-mon[129021]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:16:03.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:03 smithi149 ceph-mon[129268]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T22:16:03.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:03 smithi149 ceph-mon[129268]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T22:16:04.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:04 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:04.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:04 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:04.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:04 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:04.653 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:04 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:04.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:04 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:04.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:04 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:04.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:04 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:04.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:04 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:04.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:04 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:05.575 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:05 smithi066 ceph-mon[126379]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:05.576 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:05 smithi066 ceph-mon[126379]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:05.576 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:05 smithi066 ceph-mon[126379]: osdmap e42: 6 total, 6 up, 6 in 2023-07-21T22:16:05.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:05 smithi104 ceph-mon[129021]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:05.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:05 smithi104 ceph-mon[129021]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:05.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:05 smithi104 ceph-mon[129021]: osdmap e42: 6 total, 6 up, 6 in 2023-07-21T22:16:05.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:05 smithi149 ceph-mon[129268]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:05.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:05 smithi149 ceph-mon[129268]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:05.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:05 smithi149 ceph-mon[129268]: osdmap e42: 6 total, 6 up, 6 in 2023-07-21T22:16:06.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:06 smithi104 ceph-mon[129021]: osdmap e43: 6 total, 6 up, 6 in 2023-07-21T22:16:06.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:06 smithi066 ceph-mon[126379]: osdmap e43: 6 total, 6 up, 6 in 2023-07-21T22:16:06.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:06 smithi149 ceph-mon[129268]: osdmap e43: 6 total, 6 up, 6 in 2023-07-21T22:16:07.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:07 smithi104 ceph-mon[129021]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:07 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2068884424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]: dispatch 2023-07-21T22:16:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:07 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]: dispatch 2023-07-21T22:16:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:07 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]': finished 2023-07-21T22:16:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:07 smithi104 ceph-mon[129021]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:16:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:07 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:07.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:07 smithi066 ceph-mon[126379]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:07 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2068884424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]: dispatch 2023-07-21T22:16:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:07 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]: dispatch 2023-07-21T22:16:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:07 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]': finished 2023-07-21T22:16:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:07 smithi066 ceph-mon[126379]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:16:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:07 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:07 smithi149 ceph-mon[129268]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:07 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2068884424' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]: dispatch 2023-07-21T22:16:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:07 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]: dispatch 2023-07-21T22:16:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:07 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60ab1523-87ee-4f6c-a457-cc311b885a18"}]': finished 2023-07-21T22:16:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:07 smithi149 ceph-mon[129268]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T22:16:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:07 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:08.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:08 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3989332164' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:08.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:08 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3989332164' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:08.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:08 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3989332164' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:09.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:09 smithi104 ceph-mon[129021]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-21T22:16:09.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:09 smithi066 ceph-mon[126379]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-21T22:16:09.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:09 smithi149 ceph-mon[129268]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2023-07-21T22:16:11.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:11 smithi104 ceph-mon[129021]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:16:11.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:11 smithi066 ceph-mon[126379]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:16:11.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:11 smithi149 ceph-mon[129268]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:16:13.604 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:13 smithi066 ceph-mon[126379]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-21T22:16:13.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:13 smithi104 ceph-mon[129021]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-21T22:16:13.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:13 smithi149 ceph-mon[129268]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-07-21T22:16:14.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:14 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:16:14.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:14 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:14.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:14 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:16:14.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:14 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:14.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:14 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T22:16:14.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:14 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:15.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:15 smithi104 ceph-mon[129021]: Deploying daemon osd.6 on smithi066 2023-07-21T22:16:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:15 smithi104 ceph-mon[129021]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-21T22:16:15.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:15 smithi066 ceph-mon[126379]: Deploying daemon osd.6 on smithi066 2023-07-21T22:16:15.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:15 smithi066 ceph-mon[126379]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-21T22:16:15.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:15 smithi149 ceph-mon[129268]: Deploying daemon osd.6 on smithi066 2023-07-21T22:16:15.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:15 smithi149 ceph-mon[129268]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-07-21T22:16:17.423 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:17 smithi066 ceph-mon[126379]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:16:17.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:17 smithi104 ceph-mon[129021]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:16:17.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:17 smithi149 ceph-mon[129268]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T22:16:18.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:18 smithi149 ceph-mon[129268]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-21T22:16:19.106 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:18 smithi066 ceph-mon[126379]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-21T22:16:19.117 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:18 smithi104 ceph-mon[129021]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-07-21T22:16:21.502 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:21 smithi066 ceph-mon[126379]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:21.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:21.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:21.503 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:21 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:21.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:21 smithi104 ceph-mon[129021]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:21.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:21 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:21.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:21 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:21.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:21 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:21.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:21 smithi149 ceph-mon[129268]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:21.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:21.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:21.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:21 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:23.507 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:23 smithi066 ceph-mon[126379]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:23.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:23 smithi104 ceph-mon[129021]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:23.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:23 smithi149 ceph-mon[129268]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:23.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.778+0000 7f7ff77fe700 1 -- 172.21.15.66:0/884315370 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f800810b5d0 con 0x7f7ff00a1150 2023-07-21T22:16:23.822 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 6 on host 'smithi066' 2023-07-21T22:16:23.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7ff00a1150 msgr2=0x7f7ff00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7ff00a1150 0x7f7ff00a3600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8008199110 tx=0x7f7ff8009260 comp rx=0 tx=0).stop 2023-07-21T22:16:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008101720 msgr2=0x7f800819f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008101720 0x7f800819f1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f800000b6c0 tx=0x7f8000004c60 comp rx=0 tx=0).stop 2023-07-21T22:16:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 shutdown_connections 2023-07-21T22:16:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7ff00a1150 0x7f7ff00a3600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:23.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8008101720 0x7f800819f1b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:23.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80080ffb90 0x7f8008197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:23.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 --2- 172.21.15.66:0/884315370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8008100d90 0x7f8008198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:23.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.780+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 >> 172.21.15.66:0/884315370 conn(0x7f80080fb1a0 msgr2=0x7f8008106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:23.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.781+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 shutdown_connections 2023-07-21T22:16:23.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:16:23.781+0000 7f800c983700 1 -- 172.21.15.66:0/884315370 wait complete. 2023-07-21T22:16:24.261 DEBUG:teuthology.orchestra.run.smithi066:osd.6> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.6.service 2023-07-21T22:16:24.266 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2023-07-21T22:16:24.266 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:16:24.342 INFO:journalctl@ceph.osd.6.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:16:24.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:24 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:24.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:24.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:24.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:24.697 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:16:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:16:24.358+0000 7f443ec9b640 -1 osd.6 0 log_to_monitors true 2023-07-21T22:16:24.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:24.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:24.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:24.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:24 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:24.830 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:16:25.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:25 smithi104 ceph-mon[129021]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:25.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:25 smithi104 ceph-mon[129021]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:16:25.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:25 smithi066 ceph-mon[126379]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:25.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:25 smithi066 ceph-mon[126379]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:16:25.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:25 smithi149 ceph-mon[129268]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:25.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:25 smithi149 ceph-mon[129268]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T22:16:26.304 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:16:26.304 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:16:26.304 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2023-07-21T22:16:26.304 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2023-07-21T22:16:26.305 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0171236 s, 612 MB/s 2023-07-21T22:16:26.305 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2023-07-21T22:16:26.611 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:16:26.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: osdmap e45: 7 total, 6 up, 7 in 2023-07-21T22:16:26.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:26.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:16:26.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:16:26.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: osdmap e46: 7 total, 6 up, 7 in 2023-07-21T22:16:26.612 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:26.613 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:16:26.316+0000 7f4434313700 -1 osd.6 0 waiting for initial osdmap 2023-07-21T22:16:26.613 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:16:26 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:16:26.328+0000 7f442d903700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:16:26.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:16:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: osdmap e45: 7 total, 6 up, 7 in 2023-07-21T22:16:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:16:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:16:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: osdmap e46: 7 total, 6 up, 7 in 2023-07-21T22:16:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:26 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:26.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2023-07-21T22:16:26.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T22:16:26.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: osdmap e45: 7 total, 6 up, 7 in 2023-07-21T22:16:26.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:26.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:16:26.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: from='osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:16:26.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: osdmap e46: 7 total, 6 up, 7 in 2023-07-21T22:16:26.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:26 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.237 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:16:27.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:27 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:27.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:27.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:27 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:27.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:27 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:28.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 -- 172.21.15.104:0/168357688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 msgr2=0x7f14f40fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.435 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 --2- 172.21.15.104:0/168357688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40fffd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009990 tx=0x7f14e4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 -- 172.21.15.104:0/168357688 shutdown_connections 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 --2- 172.21.15.104:0/168357688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40fffd0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 --2- 172.21.15.104:0/168357688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4101750 0x7f14f4108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 --2- 172.21.15.104:0/168357688 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4100dc0 0x7f14f4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 -- 172.21.15.104:0/168357688 >> 172.21.15.104:0/168357688 conn(0x7f14f40fb230 msgr2=0x7f14f40fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 -- 172.21.15.104:0/168357688 shutdown_connections 2023-07-21T22:16:28.436 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.390+0000 7f14fb721700 1 -- 172.21.15.104:0/168357688 wait complete. 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 Processor -- start 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 -- start start 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4100dc0 0x7f14f4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4101750 0x7f14f4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f40735c0 con 0x7f14f4101750 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f4073730 con 0x7f14f40ffbc0 2023-07-21T22:16:28.437 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14fb721700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f419f9a0 con 0x7f14f4100dc0 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57010/0 (socket says 172.21.15.104:57010) 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 -- 172.21.15.104:0/2440092235 learned_addr learned my addr 172.21.15.104:0/2440092235 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f8cbc700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4100dc0 0x7f14f4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 -- 172.21.15.104:0/2440092235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4100dc0 msgr2=0x7f14f4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f9cbe700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4101750 0x7f14f4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4100dc0 0x7f14f4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.438 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 -- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4101750 msgr2=0x7f14f4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4101750 0x7f14f4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 -- 172.21.15.104:0/2440092235 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e40095f0 con 0x7f14f40ffbc0 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f8cbc700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4100dc0 0x7f14f4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f9cbe700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4101750 0x7f14f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14f94bd700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40742b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14e4000c00 tx=0x7f14e400f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e401d070 con 0x7f14f40ffbc0 2023-07-21T22:16:28.439 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.391+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e400fca0 con 0x7f14f40ffbc0 2023-07-21T22:16:28.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.392+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f419fc20 con 0x7f14f40ffbc0 2023-07-21T22:16:28.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.392+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4017760 con 0x7f14f40ffbc0 2023-07-21T22:16:28.440 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.392+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f41a0030 con 0x7f14f40ffbc0 2023-07-21T22:16:28.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.392+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f4107160 con 0x7f14f40ffbc0 2023-07-21T22:16:28.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.393+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f14e40178c0 con 0x7f14f40ffbc0 2023-07-21T22:16:28.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.394+0000 7f14ea7fc700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14e00a1150 0x7f14e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:28.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.394+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f14e40cd330 con 0x7f14f40ffbc0 2023-07-21T22:16:28.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.394+0000 7f14f8cbc700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14e00a1150 0x7f14e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:28.441 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.395+0000 7f14f8cbc700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14e00a1150 0x7f14e00a3600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14f0007900 tx=0x7f14f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:28.442 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.398+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f14e4097820 con 0x7f14f40ffbc0 2023-07-21T22:16:28.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:16:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:16:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi066 2023-07-21T22:16:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi066 to 5755M 2023-07-21T22:16:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] boot 2023-07-21T22:16:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: osdmap e47: 7 total, 7 up, 7 in 2023-07-21T22:16:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:28 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:28.635 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:28.591+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f14f4060950 con 0x7f14e00a1150 2023-07-21T22:16:28.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:16:28.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:16:28.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi066 2023-07-21T22:16:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi066 to 5755M 2023-07-21T22:16:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] boot 2023-07-21T22:16:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: osdmap e47: 7 total, 7 up, 7 in 2023-07-21T22:16:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:28 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:28.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:16:28.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:16:28.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi066 2023-07-21T22:16:28.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi066 to 5755M 2023-07-21T22:16:28.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: osd.6 [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] boot 2023-07-21T22:16:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: osdmap e47: 7 total, 7 up, 7 in 2023-07-21T22:16:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:28 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:16:29.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:29 smithi066 ceph-mon[126379]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:29.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:29 smithi066 ceph-mon[126379]: osdmap e48: 7 total, 7 up, 7 in 2023-07-21T22:16:29.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:29 smithi066 ceph-mon[126379]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:29.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:29.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:29.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:29 smithi104 ceph-mon[129021]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:29.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:29 smithi104 ceph-mon[129021]: osdmap e48: 7 total, 7 up, 7 in 2023-07-21T22:16:29.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:29 smithi104 ceph-mon[129021]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:29.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:29.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:29 smithi149 ceph-mon[129268]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T22:16:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:29 smithi149 ceph-mon[129268]: osdmap e48: 7 total, 7 up, 7 in 2023-07-21T22:16:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:29 smithi149 ceph-mon[129268]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:30.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:30 smithi104 ceph-mon[129021]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:30.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:30 smithi066 ceph-mon[126379]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:30.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:30 smithi149 ceph-mon[129268]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:31.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/101114204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]: dispatch 2023-07-21T22:16:31.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:31 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]: dispatch 2023-07-21T22:16:31.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:31 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]': finished 2023-07-21T22:16:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:31 smithi066 ceph-mon[126379]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:16:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:31 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:31.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/101114204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]: dispatch 2023-07-21T22:16:31.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:31 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]: dispatch 2023-07-21T22:16:31.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:31 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]': finished 2023-07-21T22:16:31.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:31 smithi149 ceph-mon[129268]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:16:31.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:31 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:31.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/101114204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]: dispatch 2023-07-21T22:16:31.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:31 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]: dispatch 2023-07-21T22:16:31.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:31 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8896ebbe-b361-411e-a5fc-f818267ad8e2"}]': finished 2023-07-21T22:16:31.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:31 smithi104 ceph-mon[129021]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T22:16:31.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:31 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:32.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:32 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/3651121135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:32.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:32 smithi066 ceph-mon[126379]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:32.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:32 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/3651121135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:32.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:32 smithi149 ceph-mon[129268]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:32.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:32 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/3651121135' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:32.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:32 smithi104 ceph-mon[129021]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:35.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:35 smithi104 ceph-mon[129021]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:35.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:35 smithi066 ceph-mon[126379]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:35.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:35 smithi149 ceph-mon[129268]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:37.533 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:37 smithi104 ceph-mon[129021]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:37.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:37 smithi066 ceph-mon[126379]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:37.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:37 smithi149 ceph-mon[129268]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:38.367 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:16:38.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:38.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:16:38.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:38.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T22:16:38.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:39.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:39 smithi104 ceph-mon[129021]: Deploying daemon osd.7 on smithi104 2023-07-21T22:16:39.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:39 smithi104 ceph-mon[129021]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:39.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:39 smithi066 ceph-mon[126379]: Deploying daemon osd.7 on smithi104 2023-07-21T22:16:39.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:39 smithi066 ceph-mon[126379]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:39.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:39 smithi149 ceph-mon[129268]: Deploying daemon osd.7 on smithi104 2023-07-21T22:16:39.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:39 smithi149 ceph-mon[129268]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:41.549 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:41 smithi104 ceph-mon[129021]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:41.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:41 smithi066 ceph-mon[126379]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:41.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:41 smithi149 ceph-mon[129268]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:42.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:42.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:42.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:42.955 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:42 smithi104 ceph-mon[129021]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:43.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:43.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:43.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:43.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:42 smithi066 ceph-mon[126379]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:43.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:43.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:43.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:43.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:42 smithi149 ceph-mon[129268]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:44.598 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2023-07-21T22:16:44.599 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.554+0000 7f14ea7fc700 1 -- 172.21.15.104:0/2440092235 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f14f4060950 con 0x7f14e00a1150 2023-07-21T22:16:44.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.556+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14e00a1150 msgr2=0x7f14e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:44.600 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.556+0000 7f14fb721700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14e00a1150 0x7f14e00a3600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14f0007900 tx=0x7f14f0008040 comp rx=0 tx=0).stop 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.556+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 msgr2=0x7f14f40742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.556+0000 7f14fb721700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40742b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f14e4000c00 tx=0x7f14e400f7b0 comp rx=0 tx=0).stop 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 shutdown_connections 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14e00a1150 0x7f14e00a3600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14f40ffbc0 0x7f14f40742b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14f4101750 0x7f14f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:44.601 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 --2- 172.21.15.104:0/2440092235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14f4100dc0 0x7f14f4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:44.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 >> 172.21.15.104:0/2440092235 conn(0x7f14f40fb230 msgr2=0x7f14f41042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:44.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 shutdown_connections 2023-07-21T22:16:44.602 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:16:44.557+0000 7f14fb721700 1 -- 172.21.15.104:0/2440092235 wait complete. 2023-07-21T22:16:45.036 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.7.service 2023-07-21T22:16:45.038 INFO:tasks.cephadm:Deploying osd.8 on smithi149 with /dev/vg_nvme/lv_2... 2023-07-21T22:16:45.039 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T22:16:45.076 INFO:journalctl@ceph.osd.7.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:16:45.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:45.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:45.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:45.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:45.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:45 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.570 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:45.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:45.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:45.571 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:45.572 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.605 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:16:45.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:45.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:16:45.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:45 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:46.043 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:16:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:16:45.738+0000 7f5d6b1f9640 -1 osd.7 0 log_to_monitors true 2023-07-21T22:16:46.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:46 smithi149 ceph-mon[129268]: from='osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:16:46.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:46 smithi149 ceph-mon[129268]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:16:46.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:46 smithi104 ceph-mon[129021]: from='osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:16:46.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:46 smithi104 ceph-mon[129021]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:16:46.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:46 smithi066 ceph-mon[126379]: from='osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:16:46.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:46 smithi066 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T22:16:47.059 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T22:16:47.059 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T22:16:47.059 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-07-21T22:16:47.059 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-07-21T22:16:47.059 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127132 s, 825 MB/s 2023-07-21T22:16:47.060 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2023-07-21T22:16:47.060 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-07-21T22:16:47.447 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:47.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:16:47.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: osdmap e50: 8 total, 7 up, 8 in 2023-07-21T22:16:47.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:47.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:16:47.448 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.449 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:47.450 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2023-07-21T22:16:47.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:16:47.294+0000 7f5d60871700 -1 osd.7 0 waiting for initial osdmap 2023-07-21T22:16:47.618 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:16:47.302+0000 7f5d59e61700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:16:47.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:47.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:16:47.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: osdmap e50: 8 total, 7 up, 8 in 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:47.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:47.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T22:16:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: osdmap e50: 8 total, 7 up, 8 in 2023-07-21T22:16:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:16:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:16:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:16:47.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:47.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:48.008 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:16:48.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi104 2023-07-21T22:16:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi104 to 5722M 2023-07-21T22:16:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:16:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: osdmap e51: 8 total, 7 up, 8 in 2023-07-21T22:16:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] boot 2023-07-21T22:16:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:16:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:48 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi104 2023-07-21T22:16:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi104 to 5722M 2023-07-21T22:16:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:16:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: osdmap e51: 8 total, 7 up, 8 in 2023-07-21T22:16:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:48.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] boot 2023-07-21T22:16:48.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:16:48.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:48 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi104 2023-07-21T22:16:48.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi104 to 5722M 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: osdmap e51: 8 total, 7 up, 8 in 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: osd.7 [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] boot 2023-07-21T22:16:48.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T22:16:48.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:48 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:16:49.169 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 -- 172.21.15.149:0/4036640212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df8100070 msgr2=0x7f2df8100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 --2- 172.21.15.149:0/4036640212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df8100070 0x7f2df8100480 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009990 tx=0x7f2de8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 -- 172.21.15.149:0/4036640212 shutdown_connections 2023-07-21T22:16:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 --2- 172.21.15.149:0/4036640212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df80ff6b0 0x7f2df8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 --2- 172.21.15.149:0/4036640212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df8100070 0x7f2df8100480 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 --2- 172.21.15.149:0/4036640212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 0x7f2df80ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 -- 172.21.15.149:0/4036640212 >> 172.21.15.149:0/4036640212 conn(0x7f2df80f9860 msgr2=0x7f2df80fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:16:49.171 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 -- 172.21.15.149:0/4036640212 shutdown_connections 2023-07-21T22:16:49.171 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.125+0000 7f2dfe606700 1 -- 172.21.15.149:0/4036640212 wait complete. 2023-07-21T22:16:49.172 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 Processor -- start 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 -- start start 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 0x7f2df80fe770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 0x7f2df819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df8100070 0x7f2df819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df819d3e0 con 0x7f2df8100070 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df819d550 con 0x7f2df80fed20 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfe606700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df819d710 con 0x7f2df80ff6b0 2023-07-21T22:16:49.173 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 0x7f2df819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.174 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 0x7f2df819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35602/0 (socket says 172.21.15.149:35602) 2023-07-21T22:16:49.174 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 -- 172.21.15.149:0/1469845145 learned_addr learned my addr 172.21.15.149:0/1469845145 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:16:49.174 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df7fff700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 0x7f2df80fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.174 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfcba3700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df8100070 0x7f2df819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.175 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 -- 172.21.15.149:0/1469845145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 msgr2=0x7f2df80fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.175 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 0x7f2df80fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.175 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 -- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df8100070 msgr2=0x7f2df819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:16:49.175 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df8100070 0x7f2df819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:16:49.175 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df77fe700 1 -- 172.21.15.149:0/1469845145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de80095f0 con 0x7f2df80ff6b0 2023-07-21T22:16:49.176 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2dfcba3700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df8100070 0x7f2df819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:16:49.176 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.126+0000 7f2df7fff700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 0x7f2df80fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:16:49.176 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.127+0000 7f2df77fe700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 0x7f2df819a600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2de000b530 tx=0x7f2de000b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.176 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.127+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0011830 con 0x7f2df80ff6b0 2023-07-21T22:16:49.176 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.127+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de0011e50 con 0x7f2df80ff6b0 2023-07-21T22:16:49.176 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.127+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de000f2e0 con 0x7f2df80ff6b0 2023-07-21T22:16:49.177 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.127+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df819d9f0 con 0x7f2df80ff6b0 2023-07-21T22:16:49.177 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.127+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df819def0 con 0x7f2df80ff6b0 2023-07-21T22:16:49.177 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.128+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df810a2d0 con 0x7f2df80ff6b0 2023-07-21T22:16:49.178 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.129+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2de0010580 con 0x7f2df80ff6b0 2023-07-21T22:16:49.178 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.129+0000 7f2df57fa700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2de40a11d0 0x7f2de40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:16:49.179 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.129+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f2de00cb870 con 0x7f2df80ff6b0 2023-07-21T22:16:49.179 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.130+0000 7f2df7fff700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2de40a11d0 0x7f2de40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:16:49.179 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.130+0000 7f2df7fff700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2de40a11d0 0x7f2de40a3680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2de8006010 tx=0x7f2de800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:16:49.179 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.134+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2de0095b00 con 0x7f2df80ff6b0 2023-07-21T22:16:49.372 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:16:49.327+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2df8060950 con 0x7f2de40a11d0 2023-07-21T22:16:49.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:49 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:16:49.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:49 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:16:49.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:49 smithi104 ceph-mon[129021]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:49.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:49 smithi104 ceph-mon[129021]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:16:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:49 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:16:49.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:49 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:16:49.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:49 smithi066 ceph-mon[126379]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:49.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:49 smithi066 ceph-mon[126379]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:16:49.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:49 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:16:49.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:49 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:16:49.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:49 smithi149 ceph-mon[129268]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T22:16:49.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:49 smithi149 ceph-mon[129268]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T22:16:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:50 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:50 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:50 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:50 smithi149 ceph-mon[129268]: osdmap e54: 8 total, 8 up, 8 in 2023-07-21T22:16:50.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:50 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:50.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:50 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:50.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:50 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:50.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:50 smithi104 ceph-mon[129021]: osdmap e54: 8 total, 8 up, 8 in 2023-07-21T22:16:50.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:50 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:16:50.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:50 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:16:50.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:50 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:50.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:50 smithi066 ceph-mon[126379]: osdmap e54: 8 total, 8 up, 8 in 2023-07-21T22:16:51.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:51 smithi104 ceph-mon[129021]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:51 smithi104 ceph-mon[129021]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:51 smithi104 ceph-mon[129021]: osdmap e55: 8 total, 8 up, 8 in 2023-07-21T22:16:51.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:51 smithi066 ceph-mon[126379]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:51.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:51 smithi066 ceph-mon[126379]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:51.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:51 smithi066 ceph-mon[126379]: osdmap e55: 8 total, 8 up, 8 in 2023-07-21T22:16:51.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:51 smithi149 ceph-mon[129268]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:16:51.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:51 smithi149 ceph-mon[129268]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:51.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:51 smithi149 ceph-mon[129268]: osdmap e55: 8 total, 8 up, 8 in 2023-07-21T22:16:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: osdmap e56: 8 total, 8 up, 8 in 2023-07-21T22:16:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/24061305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]: dispatch 2023-07-21T22:16:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]: dispatch 2023-07-21T22:16:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]': finished 2023-07-21T22:16:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: osdmap e57: 9 total, 8 up, 9 in 2023-07-21T22:16:53.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:16:53.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:53.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:52 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/2274392136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: osdmap e56: 8 total, 8 up, 8 in 2023-07-21T22:16:53.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/24061305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]: dispatch 2023-07-21T22:16:53.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]: dispatch 2023-07-21T22:16:53.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]': finished 2023-07-21T22:16:53.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: osdmap e57: 9 total, 8 up, 9 in 2023-07-21T22:16:53.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:16:53.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:53.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:52 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/2274392136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:53.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: osdmap e56: 8 total, 8 up, 8 in 2023-07-21T22:16:53.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/24061305' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]: dispatch 2023-07-21T22:16:53.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]: dispatch 2023-07-21T22:16:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7e61e60-51c6-484e-a89b-3e079f2bdb15"}]': finished 2023-07-21T22:16:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: osdmap e57: 9 total, 8 up, 9 in 2023-07-21T22:16:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:16:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:53.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:52 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/2274392136' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:16:55.508 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:55 smithi149 ceph-mon[129268]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:55 smithi104 ceph-mon[129021]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:55.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:55 smithi066 ceph-mon[126379]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:57.521 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:57 smithi149 ceph-mon[129268]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:57 smithi104 ceph-mon[129021]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:57.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:57 smithi066 ceph-mon[126379]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:59.002 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:58 smithi149 ceph-mon[129268]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:59.003 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:16:59.003 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:59.003 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:16:58 smithi149 ceph-mon[129268]: Deploying daemon osd.8 on smithi149 2023-07-21T22:16:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:58 smithi104 ceph-mon[129021]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:16:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:59.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:16:58 smithi104 ceph-mon[129021]: Deploying daemon osd.8 on smithi149 2023-07-21T22:16:59.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:58 smithi066 ceph-mon[126379]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:16:59.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T22:16:59.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:16:59.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:16:58 smithi066 ceph-mon[126379]: Deploying daemon osd.8 on smithi149 2023-07-21T22:17:01.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:01 smithi104 ceph-mon[129021]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:01.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:01 smithi066 ceph-mon[126379]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:01.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:01 smithi149 ceph-mon[129268]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:03.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:03 smithi149 ceph-mon[129268]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:03.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:03 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:03.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:03 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:03.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:03 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:03.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:03 smithi104 ceph-mon[129021]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:03.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:03 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:03.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:03 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:03.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:03 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:03.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:03 smithi066 ceph-mon[126379]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:03.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:03 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:03.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:03 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:03.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:03 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:05.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:05 smithi149 ceph-mon[129268]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.590 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.547+0000 7f2df57fa700 1 -- 172.21.15.149:0/1469845145 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2df8060950 con 0x7f2de40a11d0 2023-07-21T22:17:05.591 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 8 on host 'smithi149' 2023-07-21T22:17:05.593 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2de40a11d0 msgr2=0x7f2de40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:05.593 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2de40a11d0 0x7f2de40a3680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2de8006010 tx=0x7f2de800b540 comp rx=0 tx=0).stop 2023-07-21T22:17:05.593 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 msgr2=0x7f2df819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:05.593 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 0x7f2df819a600 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2de000b530 tx=0x7f2de000b8f0 comp rx=0 tx=0).stop 2023-07-21T22:17:05.593 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 shutdown_connections 2023-07-21T22:17:05.593 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2de40a11d0 0x7f2de40a3680 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:05.594 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2df80fed20 0x7f2df80fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:05.594 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2df8100070 0x7f2df819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:05.594 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 --2- 172.21.15.149:0/1469845145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2df80ff6b0 0x7f2df819a600 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:05.594 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 >> 172.21.15.149:0/1469845145 conn(0x7f2df80f9860 msgr2=0x7f2df80fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:05.594 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.549+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 shutdown_connections 2023-07-21T22:17:05.594 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:05.550+0000 7f2dfe606700 1 -- 172.21.15.149:0/1469845145 wait complete. 2023-07-21T22:17:05.617 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:05 smithi104 ceph-mon[129021]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:05 smithi066 ceph-mon[126379]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:05.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:05.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:05.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:05.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.101 DEBUG:teuthology.orchestra.run.smithi149:osd.8> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.8.service 2023-07-21T22:17:06.103 INFO:tasks.cephadm:Deploying osd.9 on smithi066 with /dev/vg_nvme/lv_1... 2023-07-21T22:17:06.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:17:06.135 INFO:journalctl@ceph.osd.8.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:17:06.854 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:17:06.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:06.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:06 smithi104 ceph-mon[129021]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:06.938 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:17:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:17:06.696+0000 7fb2923b6640 -1 osd.8 0 log_to_monitors true 2023-07-21T22:17:06.938 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.939 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.939 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:06.939 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:06 smithi149 ceph-mon[129268]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:06.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:06.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:06.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:06 smithi066 ceph-mon[126379]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:07.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:07 smithi066 ceph-mon[126379]: from='osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:17:07.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:07 smithi066 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:17:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:07 smithi149 ceph-mon[129268]: from='osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:17:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:07 smithi149 ceph-mon[129268]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:17:07.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:07 smithi104 ceph-mon[129021]: from='osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:17:07.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:07 smithi104 ceph-mon[129021]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T22:17:08.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:17:08.535+0000 7fb287a2e700 -1 osd.8 0 waiting for initial osdmap 2023-07-21T22:17:08.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:17:08.546+0000 7fb28101e700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: osdmap e58: 9 total, 8 up, 9 in 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:08.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.837 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:17:08.838 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:17:08.838 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2023-07-21T22:17:08.838 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2023-07-21T22:17:08.838 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.030342 s, 346 MB/s 2023-07-21T22:17:08.838 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2023-07-21T22:17:08.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:17:08.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: osdmap e58: 9 total, 8 up, 9 in 2023-07-21T22:17:08.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.875 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T22:17:08.875 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: osdmap e58: 9 total, 8 up, 9 in 2023-07-21T22:17:08.875 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:08.875 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:17:08.875 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:17:08.875 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:08.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:08.877 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:08.877 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:09.345 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi066:vg_nvme/lv_1 2023-07-21T22:17:09.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi149 2023-07-21T22:17:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi149 to 1959M 2023-07-21T22:17:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:17:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: osdmap e59: 9 total, 8 up, 9 in 2023-07-21T22:17:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] boot 2023-07-21T22:17:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:17:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi149 2023-07-21T22:17:09.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi149 to 1959M 2023-07-21T22:17:09.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:17:09.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: osdmap e59: 9 total, 8 up, 9 in 2023-07-21T22:17:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] boot 2023-07-21T22:17:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:17:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi149 2023-07-21T22:17:09.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi149 to 1959M 2023-07-21T22:17:09.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:17:09.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: osdmap e59: 9 total, 8 up, 9 in 2023-07-21T22:17:09.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:09.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: osd.8 [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] boot 2023-07-21T22:17:09.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T22:17:09.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:17:10.091 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:17:10.639 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:10 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:17:10.639 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:10 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:17:10.639 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:10 smithi066 ceph-mon[126379]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:10.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:10 smithi066 ceph-mon[126379]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T22:17:10.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:10 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:17:10.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:10 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:17:10.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:10 smithi104 ceph-mon[129021]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:10.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:10 smithi104 ceph-mon[129021]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T22:17:10.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:10 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:17:10.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:10 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:17:10.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:10 smithi149 ceph-mon[129268]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:10.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:10 smithi149 ceph-mon[129268]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T22:17:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 -- 172.21.15.66:0/360801946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a40ffb90 msgr2=0x7f50a40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/360801946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a40ffb90 0x7f50a40fffa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009990 tx=0x7f50a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 -- 172.21.15.66:0/360801946 shutdown_connections 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/360801946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4101720 0x7f50a4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/360801946 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a40ffb90 0x7f50a40fffa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/360801946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50a4100d90 0x7f50a41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 -- 172.21.15.66:0/360801946 >> 172.21.15.66:0/360801946 conn(0x7f50a40fb1a0 msgr2=0x7f50a40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:11.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 -- 172.21.15.66:0/360801946 shutdown_connections 2023-07-21T22:17:11.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.414+0000 7f50ac6c3700 1 -- 172.21.15.66:0/360801946 wait complete. 2023-07-21T22:17:11.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.415+0000 7f50ac6c3700 1 Processor -- start 2023-07-21T22:17:11.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.415+0000 7f50ac6c3700 1 -- start start 2023-07-21T22:17:11.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50ac6c3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50a40ffb90 0x7f50a40742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50ac6c3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 0x7f50a4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50ac6c3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4101720 0x7f50a4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50ac6c3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40735c0 con 0x7f50a4100d90 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50ac6c3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4073730 con 0x7f50a40ffb90 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50ac6c3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a419f930 con 0x7f50a4101720 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 0x7f50a4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:11.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 0x7f50a4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48144/0 (socket says 172.21.15.66:48144) 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 -- 172.21.15.66:0/691157191 learned_addr learned my addr 172.21.15.66:0/691157191 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50aa45f700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50a40ffb90 0x7f50a40742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50aac60700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4101720 0x7f50a4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 -- 172.21.15.66:0/691157191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4101720 msgr2=0x7f50a4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4101720 0x7f50a4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 -- 172.21.15.66:0/691157191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50a40ffb90 msgr2=0x7f50a40742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:11.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50a40ffb90 0x7f50a40742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.416+0000 7f50a9c5e700 1 -- 172.21.15.66:0/691157191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a00095f0 con 0x7f50a4100d90 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.417+0000 7f50a9c5e700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 0x7f50a4072900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f509400d980 tx=0x7f509400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.417+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094004220 con 0x7f50a4100d90 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.417+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5094007640 con 0x7f50a4100d90 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.417+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094005690 con 0x7f50a4100d90 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.417+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a419fc10 con 0x7f50a4100d90 2023-07-21T22:17:11.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.417+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41a0110 con 0x7f50a4100d90 2023-07-21T22:17:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.418+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a4107190 con 0x7f50a4100d90 2023-07-21T22:17:11.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.420+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5094004750 con 0x7f50a4100d90 2023-07-21T22:17:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.422+0000 7f509b7fe700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50900a11d0 0x7f50900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.422+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f50940cbdc0 con 0x7f50a4100d90 2023-07-21T22:17:11.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.422+0000 7f50aa45f700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50900a11d0 0x7f50900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:11.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.423+0000 7f50aa45f700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50900a11d0 0x7f50900a3680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f50a000b2e0 tx=0x7f50a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:11.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.426+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5094095e00 con 0x7f50a4100d90 2023-07-21T22:17:11.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:11.643+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a4060950 con 0x7f50900a11d0 2023-07-21T22:17:12.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:12 smithi066 ceph-mon[126379]: osdmap e62: 9 total, 9 up, 9 in 2023-07-21T22:17:12.848 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:12 smithi066 ceph-mon[126379]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:17:12.848 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:12.848 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:12.848 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:12.848 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:12 smithi066 ceph-mon[126379]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:12.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:12 smithi104 ceph-mon[129021]: osdmap e62: 9 total, 9 up, 9 in 2023-07-21T22:17:12.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:12 smithi104 ceph-mon[129021]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:17:12.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:12.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:12.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:12.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:12 smithi104 ceph-mon[129021]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:12.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:12 smithi149 ceph-mon[129268]: osdmap e62: 9 total, 9 up, 9 in 2023-07-21T22:17:12.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:12 smithi149 ceph-mon[129268]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:17:12.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:12.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:12.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:12.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:12 smithi149 ceph-mon[129268]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:13.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:13 smithi104 ceph-mon[129021]: osdmap e63: 9 total, 9 up, 9 in 2023-07-21T22:17:13.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:13 smithi066 ceph-mon[126379]: osdmap e63: 9 total, 9 up, 9 in 2023-07-21T22:17:13.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:13 smithi149 ceph-mon[129268]: osdmap e63: 9 total, 9 up, 9 in 2023-07-21T22:17:14.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:14 smithi104 ceph-mon[129021]: osdmap e64: 9 total, 9 up, 9 in 2023-07-21T22:17:14.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:14 smithi104 ceph-mon[129021]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:17:14.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:14 smithi066 ceph-mon[126379]: osdmap e64: 9 total, 9 up, 9 in 2023-07-21T22:17:14.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:14 smithi066 ceph-mon[126379]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:17:14.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:14 smithi149 ceph-mon[129268]: osdmap e64: 9 total, 9 up, 9 in 2023-07-21T22:17:14.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:14 smithi149 ceph-mon[129268]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T22:17:15.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/895873082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6812986-f4de-4f88-8aec-7f112164c075"}]: dispatch 2023-07-21T22:17:15.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/895873082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6812986-f4de-4f88-8aec-7f112164c075"}]': finished 2023-07-21T22:17:15.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:15 smithi104 ceph-mon[129021]: osdmap e65: 10 total, 9 up, 10 in 2023-07-21T22:17:15.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:15.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/895873082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6812986-f4de-4f88-8aec-7f112164c075"}]: dispatch 2023-07-21T22:17:15.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/895873082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6812986-f4de-4f88-8aec-7f112164c075"}]': finished 2023-07-21T22:17:15.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:15 smithi066 ceph-mon[126379]: osdmap e65: 10 total, 9 up, 10 in 2023-07-21T22:17:15.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:15.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/895873082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6812986-f4de-4f88-8aec-7f112164c075"}]: dispatch 2023-07-21T22:17:15.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/895873082' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6812986-f4de-4f88-8aec-7f112164c075"}]': finished 2023-07-21T22:17:15.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:15 smithi149 ceph-mon[129268]: osdmap e65: 10 total, 9 up, 10 in 2023-07-21T22:17:15.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:16.867 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:16 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/662488892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:17:16.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:16 smithi104 ceph-mon[129021]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-07-21T22:17:16.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:16 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/662488892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:17:16.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:16 smithi066 ceph-mon[126379]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-07-21T22:17:16.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:16 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/662488892' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:17:16.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:16 smithi149 ceph-mon[129268]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-07-21T22:17:19.005 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:18 smithi066 ceph-mon[126379]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:17:19.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:18 smithi104 ceph-mon[129021]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:17:19.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:18 smithi149 ceph-mon[129268]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T22:17:21.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:21 smithi104 ceph-mon[129021]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-21T22:17:21.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:21 smithi066 ceph-mon[126379]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-21T22:17:21.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:21 smithi149 ceph-mon[129268]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-07-21T22:17:23.616 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:23 smithi066 ceph-mon[126379]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:23.616 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:23 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:17:23.616 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:23 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:23.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:23 smithi104 ceph-mon[129021]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:23.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:23 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:17:23.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:23 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:23.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:23 smithi149 ceph-mon[129268]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:23.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:23 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T22:17:23.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:23 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:24.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:24 smithi066 ceph-mon[126379]: Deploying daemon osd.9 on smithi066 2023-07-21T22:17:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:24 smithi104 ceph-mon[129021]: Deploying daemon osd.9 on smithi066 2023-07-21T22:17:24.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:24 smithi149 ceph-mon[129268]: Deploying daemon osd.9 on smithi066 2023-07-21T22:17:25.430 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:25 smithi066 ceph-mon[126379]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:25.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:25 smithi104 ceph-mon[129021]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:25.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:25 smithi149 ceph-mon[129268]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:27 smithi104 ceph-mon[129021]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:27.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:27 smithi066 ceph-mon[126379]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:27 smithi149 ceph-mon[129268]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:28.785 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:28 smithi066 ceph-mon[126379]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:29.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:28 smithi104 ceph-mon[129021]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:29.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:28 smithi149 ceph-mon[129268]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:30.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:30.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:30.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:29 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:30.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:30.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:30.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:29 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:30.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:29 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:31.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:30 smithi066 ceph-mon[126379]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:31.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:30 smithi149 ceph-mon[129268]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:30 smithi104 ceph-mon[129021]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.703+0000 7f509b7fe700 1 -- 172.21.15.66:0/691157191 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f50a4060950 con 0x7f50900a11d0 2023-07-21T22:17:32.749 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 9 on host 'smithi066' 2023-07-21T22:17:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50900a11d0 msgr2=0x7f50900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50900a11d0 0x7f50900a3680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f50a000b2e0 tx=0x7f50a0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:17:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 msgr2=0x7f50a4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 0x7f50a4072900 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f509400d980 tx=0x7f509400dc90 comp rx=0 tx=0).stop 2023-07-21T22:17:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 shutdown_connections 2023-07-21T22:17:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50900a11d0 0x7f50900a3680 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50a40ffb90 0x7f50a40742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f50a4101720 0x7f50a4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 --2- 172.21.15.66:0/691157191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f50a4100d90 0x7f50a4072900 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 >> 172.21.15.66:0/691157191 conn(0x7f50a40fb1a0 msgr2=0x7f50a41042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 shutdown_connections 2023-07-21T22:17:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:17:32.706+0000 7f50ac6c3700 1 -- 172.21.15.66:0/691157191 wait complete. 2023-07-21T22:17:33.317 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.318 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.318 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:33.318 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:33.319 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.319 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:33.319 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.319 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.320 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:33.320 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:32 smithi066 ceph-mon[126379]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:17:33.321 DEBUG:teuthology.orchestra.run.smithi066:osd.9> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.9.service 2023-07-21T22:17:33.328 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2023-07-21T22:17:33.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:17:33.361 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.362 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.362 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:33.362 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:33.362 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.363 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:33.363 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.363 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.363 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:33.363 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:32 smithi104 ceph-mon[129021]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:17:33.406 INFO:journalctl@ceph.osd.9.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:17:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:33.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:33.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:32 smithi149 ceph-mon[129268]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T22:17:33.889 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:17:35.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:34 smithi104 ceph-mon[129021]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:17:35.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:34 smithi104 ceph-mon[129021]: osdmap e66: 10 total, 9 up, 10 in 2023-07-21T22:17:35.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:34 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:35.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:34 smithi104 ceph-mon[129021]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:17:35.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:34 smithi104 ceph-mon[129021]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:35.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-mon[126379]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:17:35.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-mon[126379]: osdmap e66: 10 total, 9 up, 10 in 2023-07-21T22:17:35.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:35.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-mon[126379]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:17:35.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-mon[126379]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:35.196 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:17:34.713+0000 7f5247f53700 -1 osd.9 0 waiting for initial osdmap 2023-07-21T22:17:35.197 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:17:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:17:34.733+0000 7f5241543700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:17:35.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:34 smithi149 ceph-mon[129268]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T22:17:35.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:34 smithi149 ceph-mon[129268]: osdmap e66: 10 total, 9 up, 10 in 2023-07-21T22:17:35.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:34 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:35.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:34 smithi149 ceph-mon[129268]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2023-07-21T22:17:35.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:34 smithi149 ceph-mon[129268]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T22:17:35.413 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:17:35.414 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:17:35.414 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2023-07-21T22:17:35.414 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2023-07-21T22:17:35.414 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131287 s, 799 MB/s 2023-07-21T22:17:35.414 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2023-07-21T22:17:35.831 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2023-07-21T22:17:35.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:35 smithi066 ceph-mon[126379]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:17:35.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:35 smithi066 ceph-mon[126379]: osdmap e67: 10 total, 9 up, 10 in 2023-07-21T22:17:35.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:35.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:36.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:35 smithi104 ceph-mon[129021]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:17:36.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:35 smithi104 ceph-mon[129021]: osdmap e67: 10 total, 9 up, 10 in 2023-07-21T22:17:36.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:36.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:36.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:35 smithi149 ceph-mon[129268]: from='osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2023-07-21T22:17:36.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:35 smithi149 ceph-mon[129268]: osdmap e67: 10 total, 9 up, 10 in 2023-07-21T22:17:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:36.422 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:17:36.910 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] boot 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: osdmap e68: 10 total, 10 up, 10 in 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi066 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:36.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:36.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:36.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:36.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:36.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:36.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi066 to 4316M 2023-07-21T22:17:36.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:36.913 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:36.913 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:36.913 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:36.913 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:36 smithi104 ceph-mon[129021]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:37.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:17:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:17:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] boot 2023-07-21T22:17:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: osdmap e68: 10 total, 10 up, 10 in 2023-07-21T22:17:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi066 2023-07-21T22:17:37.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi066 to 4316M 2023-07-21T22:17:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:37.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:37.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:36 smithi066 ceph-mon[126379]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:37.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: osd.9 [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] boot 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: osdmap e68: 10 total, 10 up, 10 in 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi066 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi066 to 4316M 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:37.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:37.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:37.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:36 smithi149 ceph-mon[129268]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:37.617 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 -- 172.21.15.104:0/3417613219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 msgr2=0x7fd478106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:37.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/3417613219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd478106f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd47000b0a0 tx=0x7fd47000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:17:37.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 -- 172.21.15.104:0/3417613219 shutdown_connections 2023-07-21T22:17:37.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/3417613219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd478106f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:37.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/3417613219 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd478100110 0x7fd478100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:37.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/3417613219 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 0x7fd4780ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:37.618 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 -- 172.21.15.104:0/3417613219 >> 172.21.15.104:0/3417613219 conn(0x7fd4780f9860 msgr2=0x7fd4780fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 -- 172.21.15.104:0/3417613219 shutdown_connections 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.570+0000 7fd47f0a5700 1 -- 172.21.15.104:0/3417613219 wait complete. 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 Processor -- start 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 -- start start 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 0x7fd4780fe730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd47819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:37.619 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd478100110 0x7fd47819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47819f760 con 0x7fd478100110 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47819f8d0 con 0x7fd4780ff690 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47f0a5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47819fa90 con 0x7fd4780fece0 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd47819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd47819cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57154/0 (socket says 172.21.15.104:57154) 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 -- 172.21.15.104:0/4132717244 learned_addr learned my addr 172.21.15.104:0/4132717244 (peer_addr_for_me v2:172.21.15.104:0/0) 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47ce41700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 0x7fd4780fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:37.620 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 -- 172.21.15.104:0/4132717244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 msgr2=0x7fd4780fe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 0x7fd4780fe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 -- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd478100110 msgr2=0x7fd47819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd478100110 0x7fd47819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd477fff700 1 -- 172.21.15.104:0/4132717244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd470009d00 con 0x7fd4780ff690 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.571+0000 7fd47ce41700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 0x7fd4780fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.572+0000 7fd477fff700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd47819cb60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd46400d950 tx=0x7fd46400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:37.621 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.572+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464009b40 con 0x7fd4780ff690 2023-07-21T22:17:37.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.572+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd464009ca0 con 0x7fd4780ff690 2023-07-21T22:17:37.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.572+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47819fd70 con 0x7fd4780ff690 2023-07-21T22:17:37.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.572+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400f620 con 0x7fd4780ff690 2023-07-21T22:17:37.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.572+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4781a0290 con 0x7fd4780ff690 2023-07-21T22:17:37.622 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.573+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd478106050 con 0x7fd4780ff690 2023-07-21T22:17:37.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.574+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd464010460 con 0x7fd4780ff690 2023-07-21T22:17:37.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.575+0000 7fd475ffb700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd4680a1150 0x7fd4680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:37.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.575+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fd4640cc480 con 0x7fd4780ff690 2023-07-21T22:17:37.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.575+0000 7fd47ce41700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd4680a1150 0x7fd4680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:37.623 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.576+0000 7fd47ce41700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd4680a1150 0x7fd4680a3600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0099d0 tx=0x7fd46c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:37.624 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.579+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd4640963d0 con 0x7fd4780ff690 2023-07-21T22:17:37.817 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:37.773+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd478060950 con 0x7fd4680a1150 2023-07-21T22:17:38.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:37 smithi104 ceph-mon[129021]: osdmap e69: 10 total, 10 up, 10 in 2023-07-21T22:17:38.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:37 smithi066 ceph-mon[126379]: osdmap e69: 10 total, 10 up, 10 in 2023-07-21T22:17:38.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:37 smithi149 ceph-mon[129268]: osdmap e69: 10 total, 10 up, 10 in 2023-07-21T22:17:38.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:38 smithi104 ceph-mon[129021]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:17:38.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:38.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:38.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:38.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:38 smithi104 ceph-mon[129021]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:39.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:38 smithi066 ceph-mon[126379]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:17:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:39.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:38 smithi066 ceph-mon[126379]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:39.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:38 smithi149 ceph-mon[129268]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:17:39.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:38 smithi149 ceph-mon[129268]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:40.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:40 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/349616233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]: dispatch 2023-07-21T22:17:40.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:40 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]: dispatch 2023-07-21T22:17:40.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:40 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]': finished 2023-07-21T22:17:40.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:40 smithi066 ceph-mon[126379]: osdmap e70: 11 total, 10 up, 11 in 2023-07-21T22:17:40.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:40 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:40.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:40 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/349616233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]: dispatch 2023-07-21T22:17:40.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:40 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]: dispatch 2023-07-21T22:17:40.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:40 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]': finished 2023-07-21T22:17:40.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:40 smithi149 ceph-mon[129268]: osdmap e70: 11 total, 10 up, 11 in 2023-07-21T22:17:40.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:40 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:40.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:40 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/349616233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]: dispatch 2023-07-21T22:17:40.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:40 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]: dispatch 2023-07-21T22:17:40.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:40 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24a4c4d2-8441-4081-a35b-b758c5753b8b"}]': finished 2023-07-21T22:17:40.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:40 smithi104 ceph-mon[129021]: osdmap e70: 11 total, 10 up, 11 in 2023-07-21T22:17:40.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:40 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:41.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:41 smithi066 ceph-mon[126379]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:41.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:41 smithi066 ceph-mon[126379]: from='client.? 172.21.15.104:0/966075778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:17:41.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:41 smithi149 ceph-mon[129268]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:41.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:41 smithi149 ceph-mon[129268]: from='client.? 172.21.15.104:0/966075778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:17:41.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:41 smithi104 ceph-mon[129021]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:41.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:41 smithi104 ceph-mon[129021]: from='client.? 172.21.15.104:0/966075778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:17:43.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:43 smithi104 ceph-mon[129021]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:43.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:43 smithi066 ceph-mon[126379]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:43.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:43 smithi149 ceph-mon[129268]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:45.558 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:45 smithi104 ceph-mon[129021]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:45 smithi066 ceph-mon[126379]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:45.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:45 smithi149 ceph-mon[129268]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:47.595 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:47 smithi104 ceph-mon[129021]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:47.595 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:17:47.595 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:47 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:47.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:47 smithi066 ceph-mon[126379]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:17:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:47 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:47.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:47 smithi149 ceph-mon[129268]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:47.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T22:17:47.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:47 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:48.581 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:48 smithi104 ceph-mon[129021]: Deploying daemon osd.10 on smithi104 2023-07-21T22:17:48.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:48 smithi066 ceph-mon[126379]: Deploying daemon osd.10 on smithi104 2023-07-21T22:17:48.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:48 smithi149 ceph-mon[129268]: Deploying daemon osd.10 on smithi104 2023-07-21T22:17:49.600 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:49 smithi104 ceph-mon[129021]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:49.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:49 smithi066 ceph-mon[126379]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:49.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:49 smithi149 ceph-mon[129268]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:51 smithi104 ceph-mon[129021]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:51.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:51 smithi066 ceph-mon[126379]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:51.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:51.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:51.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:51 smithi149 ceph-mon[129268]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:51.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:51.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:52.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:52 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:52.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:52 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:52.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:52 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:53.542 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:53 smithi104 ceph-mon[129021]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:53.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:53 smithi066 ceph-mon[126379]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:53.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:53 smithi149 ceph-mon[129268]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:53.983 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.939+0000 7fd475ffb700 1 -- 172.21.15.104:0/4132717244 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd478060950 con 0x7fd4680a1150 2023-07-21T22:17:53.984 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2023-07-21T22:17:53.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd4680a1150 msgr2=0x7fd4680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:53.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd4680a1150 0x7fd4680a3600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0099d0 tx=0x7fd46c008040 comp rx=0 tx=0).stop 2023-07-21T22:17:53.985 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 msgr2=0x7fd47819cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd47819cb60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd46400d950 tx=0x7fd46400dd10 comp rx=0 tx=0).stop 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 shutdown_connections 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd4680a1150 0x7fd4680a3600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd478100110 0x7fd47819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4780fece0 0x7fd4780fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 --2- 172.21.15.104:0/4132717244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4780ff690 0x7fd47819cb60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 >> 172.21.15.104:0/4132717244 conn(0x7fd4780f9860 msgr2=0x7fd478103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:53.986 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 shutdown_connections 2023-07-21T22:17:53.987 INFO:teuthology.orchestra.run.smithi104.stderr:2023-07-21T22:17:53.941+0000 7fd47f0a5700 1 -- 172.21.15.104:0/4132717244 wait complete. 2023-07-21T22:17:54.410 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.10.service 2023-07-21T22:17:54.413 INFO:tasks.cephadm:Deploying osd.11 on smithi149 with /dev/vg_nvme/lv_1... 2023-07-21T22:17:54.413 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T22:17:54.443 INFO:journalctl@ceph.osd.10.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:17:54.976 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:17:55.179 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:54 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.179 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:54 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.179 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:54 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.179 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:54 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.180 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:54 smithi149 ceph-mon[129268]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:55.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:54 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:54 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:54 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:54 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:54 smithi066 ceph-mon[126379]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:55.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:54 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:54 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.231 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:54 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.232 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:54 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:55.232 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:54 smithi104 ceph-mon[129021]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:55.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:17:55 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:17:55.233+0000 7fef8ee2e640 -1 osd.10 0 log_to_monitors true 2023-07-21T22:17:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:55 smithi149 ceph-mon[129268]: from='osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:17:56.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:55 smithi149 ceph-mon[129268]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:17:56.245 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:55 smithi104 ceph-mon[129021]: from='osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:17:56.245 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:55 smithi104 ceph-mon[129021]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:17:56.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:55 smithi066 ceph-mon[126379]: from='osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:17:56.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:55 smithi066 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T22:17:56.497 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T22:17:56.498 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T22:17:56.498 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2023-07-21T22:17:56.498 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2023-07-21T22:17:56.498 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011066 s, 948 MB/s 2023-07-21T22:17:56.498 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2023-07-21T22:17:56.934 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2023-07-21T22:17:57.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: osdmap e71: 11 total, 10 up, 11 in 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:57.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:57.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:57.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:57.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:17:56.955+0000 7fef844a6700 -1 osd.10 0 waiting for initial osdmap 2023-07-21T22:17:57.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:17:56.967+0000 7fef8029b700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:17:57.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: osdmap e71: 11 total, 10 up, 11 in 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:57.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:57.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:57.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:57.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: osdmap e71: 11 total, 10 up, 11 in 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2023-07-21T22:17:57.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:17:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:17:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:17:57.489 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:17:58.254 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi104 2023-07-21T22:17:58.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi104 to 4292M 2023-07-21T22:17:58.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:58.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:17:58.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: osdmap e72: 11 total, 10 up, 11 in 2023-07-21T22:17:58.255 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] boot 2023-07-21T22:17:58.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:17:58.256 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:57 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi104 2023-07-21T22:17:58.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi104 to 4292M 2023-07-21T22:17:58.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: osdmap e72: 11 total, 10 up, 11 in 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] boot 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:17:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:57 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi104 2023-07-21T22:17:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi104 to 4292M 2023-07-21T22:17:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:17:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2023-07-21T22:17:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: osdmap e72: 11 total, 10 up, 11 in 2023-07-21T22:17:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: osd.10 [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] boot 2023-07-21T22:17:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T22:17:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:57 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:17:58.723 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 -- 172.21.15.149:0/3712741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 msgr2=0x7fad080fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:58.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/3712741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad080fe260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7facf8009990 tx=0x7facf8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:17:58.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 -- 172.21.15.149:0/3712741634 shutdown_connections 2023-07-21T22:17:58.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/3712741634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad081017c0 0x7fad08108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:58.724 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/3712741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad080fe260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:58.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/3712741634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad08100e30 0x7fad08101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:58.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 -- 172.21.15.149:0/3712741634 >> 172.21.15.149:0/3712741634 conn(0x7fad080f9860 msgr2=0x7fad080fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:17:58.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 -- 172.21.15.149:0/3712741634 shutdown_connections 2023-07-21T22:17:58.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.678+0000 7fad0f3d5700 1 -- 172.21.15.149:0/3712741634 wait complete. 2023-07-21T22:17:58.725 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 Processor -- start 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 -- start start 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad08100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08100e30 0x7fad080fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad081017c0 0x7fad080ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad080ffaa0 con 0x7fad08100e30 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad080ffc10 con 0x7fad081017c0 2023-07-21T22:17:58.726 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0f3d5700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0819f990 con 0x7fad080fde50 2023-07-21T22:17:58.727 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad08100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:58.727 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad08100790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35750/0 (socket says 172.21.15.149:35750) 2023-07-21T22:17:58.727 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 -- 172.21.15.149:0/2687370271 learned_addr learned my addr 172.21.15.149:0/2687370271 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:17:58.727 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d972700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad081017c0 0x7fad080ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:58.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 -- 172.21.15.149:0/2687370271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad081017c0 msgr2=0x7fad080ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:58.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad081017c0 0x7fad080ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:58.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 -- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08100e30 msgr2=0x7fad080fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:17:58.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0c970700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08100e30 0x7fad080fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:58.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08100e30 0x7fad080fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:17:58.728 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 -- 172.21.15.149:0/2687370271 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf80095f0 con 0x7fad080fde50 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0c970700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08100e30 0x7fad080fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.679+0000 7fad0d171700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad08100790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7facf80098e0 tx=0x7facf800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.680+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf801d070 con 0x7fad080fde50 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.680+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facf800fb70 con 0x7fad080fde50 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.680+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0819fc10 con 0x7fad080fde50 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.680+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf8017650 con 0x7fad080fde50 2023-07-21T22:17:58.729 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.680+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad081a0020 con 0x7fad080fde50 2023-07-21T22:17:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.681+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad08107280 con 0x7fad080fde50 2023-07-21T22:17:58.730 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.682+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7facf8022470 con 0x7fad080fde50 2023-07-21T22:17:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.682+0000 7facfe7fc700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7facf40a1150 0x7facf40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:17:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.682+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6727+0+0 (secure 0 0 0) 0x7facf80cd7e0 con 0x7fad080fde50 2023-07-21T22:17:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.683+0000 7fad0c970700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7facf40a1150 0x7facf40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:17:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.683+0000 7fad0c970700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7facf40a1150 0x7facf40a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fad04005fd0 tx=0x7fad04005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:17:58.731 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.687+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7facf80975f0 con 0x7fad080fde50 2023-07-21T22:17:58.926 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:17:58.883+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fad08060950 con 0x7facf40a1150 2023-07-21T22:17:59.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:17:59.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:17:59.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:17:59.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: osdmap e74: 11 total, 11 up, 11 in 2023-07-21T22:17:59.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:59.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:59.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:59.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:17:59.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:17:59.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:17:59.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: osdmap e74: 11 total, 11 up, 11 in 2023-07-21T22:17:59.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:59.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:59.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:17:59.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:17:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:17:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:17:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: osdmap e74: 11 total, 11 up, 11 in 2023-07-21T22:17:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T22:17:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T22:17:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:00.316 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:59 smithi149 ceph-mon[129268]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:18:00.317 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:17:59 smithi149 ceph-mon[129268]: osdmap e75: 11 total, 11 up, 11 in 2023-07-21T22:18:00.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:59 smithi104 ceph-mon[129021]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:18:00.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:17:59 smithi104 ceph-mon[129021]: osdmap e75: 11 total, 11 up, 11 in 2023-07-21T22:18:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:59 smithi066 ceph-mon[126379]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:18:00.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:17:59 smithi066 ceph-mon[126379]: osdmap e75: 11 total, 11 up, 11 in 2023-07-21T22:18:01.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:00 smithi104 ceph-mon[129021]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:01.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:00 smithi104 ceph-mon[129021]: osdmap e76: 11 total, 11 up, 11 in 2023-07-21T22:18:01.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:00 smithi066 ceph-mon[126379]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:01.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:00 smithi066 ceph-mon[126379]: osdmap e76: 11 total, 11 up, 11 in 2023-07-21T22:18:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:00 smithi149 ceph-mon[129268]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:00 smithi149 ceph-mon[129268]: osdmap e76: 11 total, 11 up, 11 in 2023-07-21T22:18:02.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:01 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/2807840248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]: dispatch 2023-07-21T22:18:02.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:01 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]: dispatch 2023-07-21T22:18:02.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:01 smithi104 ceph-mon[129021]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]': finished 2023-07-21T22:18:02.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:01 smithi104 ceph-mon[129021]: osdmap e77: 12 total, 11 up, 12 in 2023-07-21T22:18:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:01 smithi104 ceph-mon[129021]: from='client.? 172.21.15.149:0/2471621603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:18:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/2807840248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]: dispatch 2023-07-21T22:18:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:01 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]: dispatch 2023-07-21T22:18:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:01 smithi066 ceph-mon[126379]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]': finished 2023-07-21T22:18:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:01 smithi066 ceph-mon[126379]: osdmap e77: 12 total, 11 up, 12 in 2023-07-21T22:18:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:02.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.149:0/2471621603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:18:02.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:01 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/2807840248' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]: dispatch 2023-07-21T22:18:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:01 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]: dispatch 2023-07-21T22:18:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:01 smithi149 ceph-mon[129268]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3e27952-c49b-4512-a879-f0b53e0aea22"}]': finished 2023-07-21T22:18:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:01 smithi149 ceph-mon[129268]: osdmap e77: 12 total, 11 up, 12 in 2023-07-21T22:18:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:01 smithi149 ceph-mon[129268]: from='client.? 172.21.15.149:0/2471621603' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T22:18:03.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:02 smithi104 ceph-mon[129021]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:03.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:02 smithi066 ceph-mon[126379]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:03.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:02 smithi149 ceph-mon[129268]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:05 smithi104 ceph-mon[129021]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-07-21T22:18:05.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:05 smithi066 ceph-mon[126379]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-07-21T22:18:05.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:05 smithi149 ceph-mon[129268]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 106 KiB/s, 0 objects/s recovering 2023-07-21T22:18:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:07 smithi104 ceph-mon[129021]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-07-21T22:18:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:07 smithi066 ceph-mon[126379]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-07-21T22:18:07.730 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:07 smithi149 ceph-mon[129268]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-07-21T22:18:08.529 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:18:08.529 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:08.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:18:08.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:08.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T22:18:08.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:09.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:09 smithi149 ceph-mon[129268]: Deploying daemon osd.11 on smithi149 2023-07-21T22:18:09.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:09 smithi149 ceph-mon[129268]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:18:09.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:09 smithi104 ceph-mon[129021]: Deploying daemon osd.11 on smithi149 2023-07-21T22:18:09.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:09 smithi104 ceph-mon[129021]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:18:09.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:09 smithi066 ceph-mon[126379]: Deploying daemon osd.11 on smithi149 2023-07-21T22:18:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:09 smithi066 ceph-mon[126379]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T22:18:11.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:11 smithi149 ceph-mon[129268]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-21T22:18:11.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:11 smithi104 ceph-mon[129021]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-21T22:18:11.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:11 smithi066 ceph-mon[126379]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-07-21T22:18:13.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:13 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:13 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:13 smithi149 ceph-mon[129268]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-21T22:18:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:13 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:13.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:13 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:13.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:13 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:13.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:13 smithi104 ceph-mon[129021]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-21T22:18:13.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:13 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:13.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:13.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:13.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:13 smithi066 ceph-mon[126379]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2023-07-21T22:18:13.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:13 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:15.145 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.100+0000 7facfe7fc700 1 -- 172.21.15.149:0/2687370271 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fad08060950 con 0x7facf40a1150 2023-07-21T22:18:15.145 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 11 on host 'smithi149' 2023-07-21T22:18:15.147 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7facf40a1150 msgr2=0x7facf40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:15.147 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7facf40a1150 0x7facf40a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fad04005fd0 tx=0x7fad04005ee0 comp rx=0 tx=0).stop 2023-07-21T22:18:15.147 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 msgr2=0x7fad08100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:15.148 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad08100790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7facf80098e0 tx=0x7facf800f740 comp rx=0 tx=0).stop 2023-07-21T22:18:15.148 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 shutdown_connections 2023-07-21T22:18:15.148 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7facf40a1150 0x7facf40a3600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:15.148 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad081017c0 0x7fad080ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:15.148 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad080fde50 0x7fad08100790 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:15.149 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 --2- 172.21.15.149:0/2687370271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad08100e30 0x7fad080fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:15.149 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.102+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 >> 172.21.15.149:0/2687370271 conn(0x7fad080f9860 msgr2=0x7fad081043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:15.149 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.103+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 shutdown_connections 2023-07-21T22:18:15.149 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:15.103+0000 7fad0f3d5700 1 -- 172.21.15.149:0/2687370271 wait complete. 2023-07-21T22:18:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:18:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:15.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:15.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:15 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.585 DEBUG:teuthology.orchestra.run.smithi149:osd.11> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.11.service 2023-07-21T22:18:15.588 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-21T22:18:15.588 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T22:18:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:18:15.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:15.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:15 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.621 INFO:journalctl@ceph.osd.11.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:18:15.660 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-07-21T22:18:15.660 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.660 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.661 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:15.661 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:15.661 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.661 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:15.662 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:15.662 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:15 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:16.340 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:18:16.369 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:16 smithi149 ceph-mon[129268]: from='osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:18:16.369 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:16 smithi149 ceph-mon[129268]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:18:16.370 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:18:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:18:16.120+0000 7ffaceb3d640 -1 osd.11 0 log_to_monitors true 2023-07-21T22:18:16.572 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:16 smithi066 ceph-mon[126379]: from='osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:18:16.572 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:16 smithi066 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:18:16.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:16 smithi104 ceph-mon[129021]: from='osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:18:16.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:16 smithi104 ceph-mon[129021]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T22:18:17.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:17 smithi066 ceph-mon[126379]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:17 smithi066 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:18:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:17 smithi066 ceph-mon[126379]: osdmap e78: 12 total, 11 up, 12 in 2023-07-21T22:18:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:17 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:17 smithi066 ceph-mon[126379]: from='osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:18:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:17 smithi066 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:18:17.585 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:18:17.238+0000 7ffac49b6700 -1 osd.11 0 waiting for initial osdmap 2023-07-21T22:18:17.586 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:18:17.256+0000 7ffabd7a5700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T22:18:17.586 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-mon[129268]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:17.586 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-mon[129268]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:18:17.587 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-mon[129268]: osdmap e78: 12 total, 11 up, 12 in 2023-07-21T22:18:17.587 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:17.587 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-mon[129268]: from='osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:18:17.587 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:17 smithi149 ceph-mon[129268]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:18:17.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:17 smithi104 ceph-mon[129021]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:17.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:17 smithi104 ceph-mon[129021]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T22:18:17.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:17 smithi104 ceph-mon[129021]: osdmap e78: 12 total, 11 up, 12 in 2023-07-21T22:18:17.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:17 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:17.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:17 smithi104 ceph-mon[129021]: from='osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:18:17.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:17 smithi104 ceph-mon[129021]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2023-07-21T22:18:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 -- 172.21.15.66:0/1533926298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab40ffb90 msgr2=0x7f1ab40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:17.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 --2- 172.21.15.66:0/1533926298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab40ffb90 0x7f1ab40fffa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009990 tx=0x7f1aa4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 -- 172.21.15.66:0/1533926298 shutdown_connections 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 --2- 172.21.15.66:0/1533926298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab4101720 0x7f1ab4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 --2- 172.21.15.66:0/1533926298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab40ffb90 0x7f1ab40fffa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 --2- 172.21.15.66:0/1533926298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab4100d90 0x7f1ab41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 -- 172.21.15.66:0/1533926298 >> 172.21.15.66:0/1533926298 conn(0x7f1ab40fb1a0 msgr2=0x7f1ab40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 -- 172.21.15.66:0/1533926298 shutdown_connections 2023-07-21T22:18:17.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.677+0000 7f1aba013700 1 -- 172.21.15.66:0/1533926298 wait complete. 2023-07-21T22:18:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.678+0000 7f1aba013700 1 Processor -- start 2023-07-21T22:18:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 -- start start 2023-07-21T22:18:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 0x7f1ab4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 0x7f1ab4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:17.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab4101720 0x7f1ab419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419f640 con 0x7f1ab4100d90 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419f780 con 0x7f1ab40ffb90 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1aba013700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab419f910 con 0x7f1ab4101720 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1ab2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 0x7f1ab4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 0x7f1ab4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1ab2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 0x7f1ab4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39372/0 (socket says 172.21.15.66:39372) 2023-07-21T22:18:17.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 0x7f1ab4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:52284/0 (socket says 172.21.15.66:52284) 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.679+0000 7f1ab2ffd700 1 -- 172.21.15.66:0/145323076 learned_addr learned my addr 172.21.15.66:0/145323076 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab3fff700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab4101720 0x7f1ab419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab37fe700 1 -- 172.21.15.66:0/145323076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab4101720 msgr2=0x7f1ab419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab37fe700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab4101720 0x7f1ab419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab37fe700 1 -- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 msgr2=0x7f1ab4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab37fe700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 0x7f1ab4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab37fe700 1 -- 172.21.15.66:0/145323076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40095f0 con 0x7f1ab40ffb90 2023-07-21T22:18:17.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab2ffd700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 0x7f1ab4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:17.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab37fe700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 0x7f1ab4197ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40098e0 tx=0x7f1aa4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:17.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401d070 con 0x7f1ab40ffb90 2023-07-21T22:18:17.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa4021410 con 0x7f1ab40ffb90 2023-07-21T22:18:17.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab419fb90 con 0x7f1ab40ffb90 2023-07-21T22:18:17.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.680+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4017650 con 0x7f1ab40ffb90 2023-07-21T22:18:17.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.681+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab419ffa0 con 0x7f1ab40ffb90 2023-07-21T22:18:17.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.682+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab4192210 con 0x7f1ab40ffb90 2023-07-21T22:18:17.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.684+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1aa4021810 con 0x7f1ab40ffb90 2023-07-21T22:18:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.685+0000 7f1ab0ff9700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.685+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f1aa40cddf0 con 0x7f1ab40ffb90 2023-07-21T22:18:17.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.686+0000 7f1ab2ffd700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:17.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.686+0000 7f1ab2ffd700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8011550 tx=0x7f1aa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:17.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.687+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1aa4097af0 con 0x7f1ab40ffb90 2023-07-21T22:18:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.910+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f1ab402caa0 con 0x7f1ab40ffb90 2023-07-21T22:18:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.910+0000 7f1ab0ff9700 1 -- 172.21.15.66:0/145323076 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f1aa409b110 con 0x7f1ab40ffb90 2023-07-21T22:18:17.954 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:17.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1a9c0a1150 msgr2=0x7f1a9c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8011550 tx=0x7f1aa8009260 comp rx=0 tx=0).stop 2023-07-21T22:18:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 msgr2=0x7f1ab4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 0x7f1ab4197ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40098e0 tx=0x7f1aa4009580 comp rx=0 tx=0).stop 2023-07-21T22:18:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 shutdown_connections 2023-07-21T22:18:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ab40ffb90 0x7f1ab4197ab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab4101720 0x7f1ab419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 --2- 172.21.15.66:0/145323076 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1ab4100d90 0x7f1ab4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 >> 172.21.15.66:0/145323076 conn(0x7f1ab40fb1a0 msgr2=0x7f1ab41066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 shutdown_connections 2023-07-21T22:18:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:17.913+0000 7f1aba013700 1 -- 172.21.15.66:0/145323076 wait complete. 2023-07-21T22:18:18.330 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1689977877,"num_in_osds":12,"osd_in_since":1689977881,"num_remapped_pgs":0} 2023-07-21T22:18:18.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:18:18.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: osdmap e79: 12 total, 11 up, 12 in 2023-07-21T22:18:18.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:18.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:18.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.623 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.623 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.623 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:18.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:18.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:18 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/145323076' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:18:18.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:18:18.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: osdmap e79: 12 total, 11 up, 12 in 2023-07-21T22:18:18.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:18.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:18.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:18 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/145323076' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:18:18.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2023-07-21T22:18:18.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: osdmap e79: 12 total, 11 up, 12 in 2023-07-21T22:18:18.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:18.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:18.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:18:18.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:18.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:18.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:18.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:18 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/145323076' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:18:19.331 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T22:18:19.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: purged_snaps scrub starts 2023-07-21T22:18:19.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: purged_snaps scrub ok 2023-07-21T22:18:19.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: Detected new or changed devices on smithi149 2023-07-21T22:18:19.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi149 to 1469M 2023-07-21T22:18:19.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:19.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] boot 2023-07-21T22:18:19.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:18:19.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:19.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: purged_snaps scrub starts 2023-07-21T22:18:19.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: purged_snaps scrub ok 2023-07-21T22:18:19.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: Detected new or changed devices on smithi149 2023-07-21T22:18:19.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi149 to 1469M 2023-07-21T22:18:19.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:19.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] boot 2023-07-21T22:18:19.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:18:19.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:19.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: purged_snaps scrub starts 2023-07-21T22:18:19.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: purged_snaps scrub ok 2023-07-21T22:18:19.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: Detected new or changed devices on smithi149 2023-07-21T22:18:19.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi149 to 1469M 2023-07-21T22:18:19.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T22:18:19.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: osd.11 [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] boot 2023-07-21T22:18:19.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T22:18:19.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:18:20.080 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:18:20.938 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:20 smithi066 ceph-mon[126379]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:18:20.938 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:20 smithi066 ceph-mon[126379]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:21.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:20 smithi149 ceph-mon[129268]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:18:21.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:20 smithi149 ceph-mon[129268]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:21.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:20 smithi104 ceph-mon[129021]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T22:18:21.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:20 smithi104 ceph-mon[129021]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 -- 172.21.15.66:0/197865472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40ffb90 msgr2=0x7f0da40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 --2- 172.21.15.66:0/197865472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40ffb90 0x7f0da40fffa0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d94009990 tx=0x7f0d94009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 -- 172.21.15.66:0/197865472 shutdown_connections 2023-07-21T22:18:21.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 --2- 172.21.15.66:0/197865472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da4101720 0x7f0da4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 --2- 172.21.15.66:0/197865472 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da40ffb90 0x7f0da40fffa0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 --2- 172.21.15.66:0/197865472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0da4100d90 0x7f0da41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 -- 172.21.15.66:0/197865472 >> 172.21.15.66:0/197865472 conn(0x7f0da40fb1a0 msgr2=0x7f0da40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:21.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 -- 172.21.15.66:0/197865472 shutdown_connections 2023-07-21T22:18:21.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.519+0000 7f0daa458700 1 -- 172.21.15.66:0/197865472 wait complete. 2023-07-21T22:18:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 Processor -- start 2023-07-21T22:18:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 -- start start 2023-07-21T22:18:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 0x7f0da4195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:21.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4100d90 0x7f0da4195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0da4101720 0x7f0da419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419d420 con 0x7f0da4100d90 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419d560 con 0x7f0da4101720 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.520+0000 7f0daa458700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da419d6f0 con 0x7f0da40ffb90 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4100d90 0x7f0da4195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 0x7f0da4195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:21.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 0x7f0da4195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:60414/0 (socket says 172.21.15.66:60414) 2023-07-21T22:18:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da3fff700 1 -- 172.21.15.66:0/2980175800 learned_addr learned my addr 172.21.15.66:0/2980175800 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da89f5700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0da4101720 0x7f0da419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 -- 172.21.15.66:0/2980175800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 msgr2=0x7f0da4195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 0x7f0da4195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 -- 172.21.15.66:0/2980175800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0da4101720 msgr2=0x7f0da419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:21.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0da4101720 0x7f0da419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 -- 172.21.15.66:0/2980175800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d940095f0 con 0x7f0da4100d90 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da3fff700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 0x7f0da4195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.521+0000 7f0da37fe700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4100d90 0x7f0da4195dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800ea20 tx=0x7f0d9800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.522+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800cbd0 con 0x7f0da4100d90 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.522+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da419d9d0 con 0x7f0da4100d90 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.522+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d9800ee60 con 0x7f0da4100d90 2023-07-21T22:18:21.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.522+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98018680 con 0x7f0da4100d90 2023-07-21T22:18:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.522+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da419df20 con 0x7f0da4100d90 2023-07-21T22:18:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.523+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da41071d0 con 0x7f0da4100d90 2023-07-21T22:18:21.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.525+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0d98010430 con 0x7f0da4100d90 2023-07-21T22:18:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.528+0000 7f0da17fa700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0d8c0a11d0 0x7f0d8c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:21.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.528+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0d98014070 con 0x7f0da4100d90 2023-07-21T22:18:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.528+0000 7f0da3fff700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0d8c0a11d0 0x7f0d8c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:21.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.528+0000 7f0da3fff700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0d8c0a11d0 0x7f0d8c0a3680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b680 tx=0x7f0d94005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:21.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.532+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0d98051770 con 0x7f0da4100d90 2023-07-21T22:18:21.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.740+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0da402c850 con 0x7f0da4100d90 2023-07-21T22:18:21.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.740+0000 7f0da17fa700 1 -- 172.21.15.66:0/2980175800 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7f0d98096100 con 0x7f0da4100d90 2023-07-21T22:18:21.785 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0d8c0a11d0 msgr2=0x7f0d8c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0d8c0a11d0 0x7f0d8c0a3680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400b680 tx=0x7f0d94005fd0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4100d90 msgr2=0x7f0da4195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4100d90 0x7f0da4195dd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800ea20 tx=0x7f0d9800ed30 comp rx=0 tx=0).stop 2023-07-21T22:18:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 shutdown_connections 2023-07-21T22:18:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0d8c0a11d0 0x7f0d8c0a3680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0da4101720 0x7f0da419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0da40ffb90 0x7f0da4195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 --2- 172.21.15.66:0/2980175800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0da4100d90 0x7f0da4195dd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 >> 172.21.15.66:0/2980175800 conn(0x7f0da40fb1a0 msgr2=0x7f0da4104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 shutdown_connections 2023-07-21T22:18:21.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:21.743+0000 7f0daa458700 1 -- 172.21.15.66:0/2980175800 wait complete. 2023-07-21T22:18:22.158 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:21 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2980175800' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:18:22.161 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1689977898,"num_in_osds":12,"osd_in_since":1689977881,"num_remapped_pgs":0} 2023-07-21T22:18:22.162 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:18:22.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:21 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2980175800' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:18:22.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:21 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2980175800' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T22:18:22.904 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:18:23.160 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:22 smithi066 ceph-mon[126379]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:23.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:22 smithi149 ceph-mon[129268]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:23.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:22 smithi104 ceph-mon[129021]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:24.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 -- 172.21.15.66:0/431714268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64141007a0 msgr2=0x7f6414104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:24.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 --2- 172.21.15.66:0/431714268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64141007a0 0x7f6414104770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f64080099c0 tx=0x7f6408009cd0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 -- 172.21.15.66:0/431714268 shutdown_connections 2023-07-21T22:18:24.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 --2- 172.21.15.66:0/431714268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6414104ea0 0x7f641410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 --2- 172.21.15.66:0/431714268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64140ffe00 0x7f64141001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 --2- 172.21.15.66:0/431714268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64141007a0 0x7f6414104770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 -- 172.21.15.66:0/431714268 >> 172.21.15.66:0/431714268 conn(0x7f64140fb660 msgr2=0x7f64140fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:24.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 -- 172.21.15.66:0/431714268 shutdown_connections 2023-07-21T22:18:24.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.328+0000 7f641a9a9700 1 -- 172.21.15.66:0/431714268 wait complete. 2023-07-21T22:18:24.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.329+0000 7f641a9a9700 1 Processor -- start 2023-07-21T22:18:24.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 -- start start 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 0x7f6414195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64141007a0 0x7f6414196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 0x7f641419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419acf0 con 0x7f6414104ea0 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419ae60 con 0x7f64140ffe00 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f641a9a9700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641419b020 con 0x7f64141007a0 2023-07-21T22:18:24.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f6418f46700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 0x7f641419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f6413fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 0x7f6414195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f6413fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 0x7f6414195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48114/0 (socket says 172.21.15.66:48114) 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f6413fff700 1 -- 172.21.15.66:0/3844653539 learned_addr learned my addr 172.21.15.66:0/3844653539 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.330+0000 7f6418f46700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 0x7f641419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55254/0 (socket says 172.21.15.66:55254) 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f64137fe700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64141007a0 0x7f6414196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6413fff700 1 -- 172.21.15.66:0/3844653539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64141007a0 msgr2=0x7f6414196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:24.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6413fff700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64141007a0 0x7f6414196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6413fff700 1 -- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 msgr2=0x7f641419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6413fff700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 0x7f641419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6413fff700 1 -- 172.21.15.66:0/3844653539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080095f0 con 0x7f64140ffe00 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6418f46700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 0x7f641419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f6413fff700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 0x7f6414195d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f640400e9a0 tx=0x7f640400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404009790 con 0x7f64140ffe00 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641419b300 con 0x7f64140ffe00 2023-07-21T22:18:24.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6404010430 con 0x7f64140ffe00 2023-07-21T22:18:24.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.331+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404018680 con 0x7f64140ffe00 2023-07-21T22:18:24.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.332+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64141a2b60 con 0x7f64140ffe00 2023-07-21T22:18:24.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.333+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641404e210 con 0x7f64140ffe00 2023-07-21T22:18:24.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.335+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f64040105a0 con 0x7f64140ffe00 2023-07-21T22:18:24.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.336+0000 7f64117fa700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f63fc0a1150 0x7f63fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:24.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.336+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6404014070 con 0x7f64140ffe00 2023-07-21T22:18:24.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.336+0000 7f64137fe700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f63fc0a1150 0x7f63fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:24.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.337+0000 7f64137fe700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f63fc0a1150 0x7f63fc0a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f64080098e0 tx=0x7f6408000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:24.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.340+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6404051770 con 0x7f64140ffe00 2023-07-21T22:18:24.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.548+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6414197090 con 0x7f64140ffe00 2023-07-21T22:18:24.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.550+0000 7f64117fa700 1 -- 172.21.15.66:0/3844653539 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18432 (secure 0 0 0) 0x7f6404096220 con 0x7f64140ffe00 2023-07-21T22:18:24.594 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:24.595 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:18:19.827969+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:18:24.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f63fc0a1150 msgr2=0x7f63fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:24.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f63fc0a1150 0x7f63fc0a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f64080098e0 tx=0x7f6408000bc0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 msgr2=0x7f6414195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:24.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 0x7f6414195d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f640400e9a0 tx=0x7f640400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 shutdown_connections 2023-07-21T22:18:24.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f63fc0a1150 0x7f63fc0a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64140ffe00 0x7f6414195d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6414104ea0 0x7f641419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 --2- 172.21.15.66:0/3844653539 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64141007a0 0x7f6414196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:24.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 >> 172.21.15.66:0/3844653539 conn(0x7f64140fb660 msgr2=0x7f64140fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:24.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 shutdown_connections 2023-07-21T22:18:24.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:24.553+0000 7f641a9a9700 1 -- 172.21.15.66:0/3844653539 wait complete. 2023-07-21T22:18:25.094 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:18:25.095 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-21T22:18:25.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:25 smithi066 ceph-mon[126379]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:25.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:25 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3844653539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:18:25.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:25 smithi104 ceph-mon[129021]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:25.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:25 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3844653539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:18:25.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:25 smithi149 ceph-mon[129268]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:25.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:25 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3844653539' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:18:25.838 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:18:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 -- 172.21.15.66:0/3562350157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82180ffe00 msgr2=0x7f82181001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 --2- 172.21.15.66:0/3562350157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82180ffe00 0x7f82181001d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f82080099c0 tx=0x7f8208009cd0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 -- 172.21.15.66:0/3562350157 shutdown_connections 2023-07-21T22:18:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 --2- 172.21.15.66:0/3562350157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8218104ea0 0x7f821810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 --2- 172.21.15.66:0/3562350157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f82180ffe00 0x7f82181001d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 --2- 172.21.15.66:0/3562350157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82181007a0 0x7f8218104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 -- 172.21.15.66:0/3562350157 >> 172.21.15.66:0/3562350157 conn(0x7f82180fb660 msgr2=0x7f82180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 -- 172.21.15.66:0/3562350157 shutdown_connections 2023-07-21T22:18:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.173+0000 7f821d404700 1 -- 172.21.15.66:0/3562350157 wait complete. 2023-07-21T22:18:27.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.174+0000 7f821d404700 1 Processor -- start 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.174+0000 7f821d404700 1 -- start start 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f821d404700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82180ffe00 0x7f8218072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f821d404700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82181007a0 0x7f8218072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f821d404700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 0x7f82180732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f82177fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 0x7f82180732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f82177fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 0x7f82180732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55276/0 (socket says 172.21.15.66:55276) 2023-07-21T22:18:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f82177fe700 1 -- 172.21.15.66:0/2384408306 learned_addr learned my addr 172.21.15.66:0/2384408306 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f8216ffd700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82180ffe00 0x7f8218072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181a40a0 con 0x7f8218104ea0 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181a4210 con 0x7f82181007a0 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82181a43d0 con 0x7f82180ffe00 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.175+0000 7f82167fc700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82181007a0 0x7f8218072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f8216ffd700 1 -- 172.21.15.66:0/2384408306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82181007a0 msgr2=0x7f8218072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f8216ffd700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82181007a0 0x7f8218072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f8216ffd700 1 -- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 msgr2=0x7f82180732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f8216ffd700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 0x7f82180732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f8216ffd700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080095f0 con 0x7f82180ffe00 2023-07-21T22:18:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f82177fe700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 0x7f82180732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f8216ffd700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82180ffe00 0x7f8218072810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82080098e0 tx=0x7f820800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820801d070 con 0x7f82180ffe00 2023-07-21T22:18:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8208022470 con 0x7f82180ffe00 2023-07-21T22:18:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82181a4650 con 0x7f82180ffe00 2023-07-21T22:18:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.176+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208017650 con 0x7f82180ffe00 2023-07-21T22:18:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.177+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82181a4a60 con 0x7f82180ffe00 2023-07-21T22:18:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.177+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821804e210 con 0x7f82180ffe00 2023-07-21T22:18:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.180+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8208022890 con 0x7f82180ffe00 2023-07-21T22:18:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.181+0000 7f81fffff700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f82000a1150 0x7f82000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.181+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82080cd520 con 0x7f82180ffe00 2023-07-21T22:18:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.181+0000 7f82167fc700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f82000a1150 0x7f82000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.182+0000 7f82167fc700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f82000a1150 0x7f82000a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8218074000 tx=0x7f820c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:27.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.183+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8208097100 con 0x7f82180ffe00 2023-07-21T22:18:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.389+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82181a4c80 con 0x7f82180ffe00 2023-07-21T22:18:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.390+0000 7f81fffff700 1 -- 172.21.15.66:0/2384408306 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f820809a720 con 0x7f82180ffe00 2023-07-21T22:18:27.434 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:18:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f82000a1150 msgr2=0x7f82000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f82000a1150 0x7f82000a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8218074000 tx=0x7f820c009260 comp rx=0 tx=0).stop 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82180ffe00 msgr2=0x7f8218072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82180ffe00 0x7f8218072810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82080098e0 tx=0x7f820800f690 comp rx=0 tx=0).stop 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 shutdown_connections 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f82000a1150 0x7f82000a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8218104ea0 0x7f82180732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82180ffe00 0x7f8218072810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 --2- 172.21.15.66:0/2384408306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82181007a0 0x7f8218072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 >> 172.21.15.66:0/2384408306 conn(0x7f82180fb660 msgr2=0x7f8218103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 shutdown_connections 2023-07-21T22:18:27.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:27.393+0000 7f821d404700 1 -- 172.21.15.66:0/2384408306 wait complete. 2023-07-21T22:18:27.563 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:27 smithi066 ceph-mon[126379]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:27 smithi104 ceph-mon[129021]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:27 smithi149 ceph-mon[129268]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:27.922 INFO:tasks.cephadm:Adding mds.a on smithi066 2023-07-21T22:18:27.923 INFO:tasks.cephadm:Adding mds.d on smithi066 2023-07-21T22:18:27.923 INFO:tasks.cephadm:Adding mds.g on smithi066 2023-07-21T22:18:27.924 INFO:tasks.cephadm:Adding mds.j on smithi066 2023-07-21T22:18:27.925 INFO:tasks.cephadm:Adding mds.b on smithi104 2023-07-21T22:18:27.925 INFO:tasks.cephadm:Adding mds.e on smithi104 2023-07-21T22:18:27.926 INFO:tasks.cephadm:Adding mds.h on smithi104 2023-07-21T22:18:27.926 INFO:tasks.cephadm:Adding mds.k on smithi104 2023-07-21T22:18:27.927 INFO:tasks.cephadm:Adding mds.c on smithi149 2023-07-21T22:18:27.927 INFO:tasks.cephadm:Adding mds.f on smithi149 2023-07-21T22:18:27.927 INFO:tasks.cephadm:Adding mds.i on smithi149 2023-07-21T22:18:27.928 INFO:tasks.cephadm:Adding mds.l on smithi149 2023-07-21T22:18:27.928 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l' 2023-07-21T22:18:28.496 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:18:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:28 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2384408306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:18:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:28 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2384408306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:18:28.707 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:28 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2384408306' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:18:29.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:29 smithi149 ceph-mon[129268]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:29.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:29 smithi104 ceph-mon[129021]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:29.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:29 smithi066 ceph-mon[126379]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:29.707 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 -- 172.21.15.149:0/3654371492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580ffc80 msgr2=0x7fa8581000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:29.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 --2- 172.21.15.149:0/3654371492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580ffc80 0x7fa8581000d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa848009990 tx=0x7fa848009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 -- 172.21.15.149:0/3654371492 shutdown_connections 2023-07-21T22:18:29.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 --2- 172.21.15.149:0/3654371492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8580fea80 0x7fa8580fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 --2- 172.21.15.149:0/3654371492 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa858100610 0x7fa858106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 --2- 172.21.15.149:0/3654371492 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580ffc80 0x7fa8581000d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.708 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 -- 172.21.15.149:0/3654371492 >> 172.21.15.149:0/3654371492 conn(0x7fa8580fa0b0 msgr2=0x7fa8580fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:29.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 -- 172.21.15.149:0/3654371492 shutdown_connections 2023-07-21T22:18:29.709 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.663+0000 7fa85d158700 1 -- 172.21.15.149:0/3654371492 wait complete. 2023-07-21T22:18:29.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 Processor -- start 2023-07-21T22:18:29.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 -- start start 2023-07-21T22:18:29.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 0x7fa858195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:29.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8580ffc80 0x7fa858195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:29.710 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa858100610 0x7fa85819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85819d420 con 0x7fa858100610 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85819d560 con 0x7fa8580ffc80 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85d158700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85819d6f0 con 0x7fa8580fea80 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 0x7fa858195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 0x7fa858195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:35874/0 (socket says 172.21.15.149:35874) 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 -- 172.21.15.149:0/2885717738 learned_addr learned my addr 172.21.15.149:0/2885717738 (peer_addr_for_me v2:172.21.15.149:0/0) 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa85659c700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8580ffc80 0x7fa858195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:29.711 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 -- 172.21.15.149:0/2885717738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8580ffc80 msgr2=0x7fa858195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8580ffc80 0x7fa858195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 -- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa858100610 msgr2=0x7fa85819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa858100610 0x7fa85819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 -- 172.21.15.149:0/2885717738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8480095f0 con 0x7fa8580fea80 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.664+0000 7fa856d9d700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 0x7fa858195890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e9a0 tx=0x7fa84c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.665+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c009790 con 0x7fa8580fea80 2023-07-21T22:18:29.712 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.665+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa84c0044e0 con 0x7fa8580fea80 2023-07-21T22:18:29.713 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.665+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85819d9d0 con 0x7fa8580fea80 2023-07-21T22:18:29.713 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.665+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c003d20 con 0x7fa8580fea80 2023-07-21T22:18:29.713 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.665+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85819ded0 con 0x7fa8580fea80 2023-07-21T22:18:29.714 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.666+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa858106b90 con 0x7fa8580fea80 2023-07-21T22:18:29.714 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.667+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa84c003e80 con 0x7fa8580fea80 2023-07-21T22:18:29.714 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.667+0000 7fa83ffff700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8400a1150 0x7fa8400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:29.714 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.667+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa84c014070 con 0x7fa8580fea80 2023-07-21T22:18:29.714 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.668+0000 7fa85659c700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8400a1150 0x7fa8400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:29.714 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.668+0000 7fa85659c700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8400a1150 0x7fa8400a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b2e0 tx=0x7fa848005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:29.715 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.671+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa84c0963b0 con 0x7fa8580fea80 2023-07-21T22:18:29.911 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.866+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fa858060a50 con 0x7fa8400a1150 2023-07-21T22:18:29.920 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.875+0000 7fa83ffff700 1 -- 172.21.15.149:0/2885717738 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fa858060a50 con 0x7fa8400a1150 2023-07-21T22:18:29.920 INFO:teuthology.orchestra.run.smithi149.stdout:Scheduled mds.all update... 2023-07-21T22:18:29.922 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.877+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8400a1150 msgr2=0x7fa8400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8400a1150 0x7fa8400a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa84800b2e0 tx=0x7fa848005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 msgr2=0x7fa858195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 0x7fa858195890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e9a0 tx=0x7fa84c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 shutdown_connections 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8400a1150 0x7fa8400a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa858100610 0x7fa85819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.923 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8580fea80 0x7fa858195890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.924 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 --2- 172.21.15.149:0/2885717738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8580ffc80 0x7fa858195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:29.924 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 >> 172.21.15.149:0/2885717738 conn(0x7fa8580fa0b0 msgr2=0x7fa8580fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:29.924 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 shutdown_connections 2023-07-21T22:18:29.924 INFO:teuthology.orchestra.run.smithi149.stderr:2023-07-21T22:18:29.878+0000 7fa85d158700 1 -- 172.21.15.149:0/2885717738 wait complete. 2023-07-21T22:18:30.338 DEBUG:teuthology.orchestra.run.smithi066:mds.a> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.a.service 2023-07-21T22:18:30.344 DEBUG:teuthology.orchestra.run.smithi066:mds.d> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.d.service 2023-07-21T22:18:30.390 DEBUG:teuthology.orchestra.run.smithi066:mds.g> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.g.service 2023-07-21T22:18:30.430 INFO:journalctl@ceph.mds.a.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:18:30.434 DEBUG:teuthology.orchestra.run.smithi066:mds.j> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.j.service 2023-07-21T22:18:30.474 INFO:journalctl@ceph.mds.d.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:18:30.477 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.b.service 2023-07-21T22:18:30.479 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.e.service 2023-07-21T22:18:30.508 INFO:journalctl@ceph.mds.g.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:18:30.509 INFO:journalctl@ceph.mds.b.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:18:30.510 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.h.service 2023-07-21T22:18:30.535 INFO:journalctl@ceph.mds.e.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:18:30.536 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.k.service 2023-07-21T22:18:30.552 INFO:journalctl@ceph.mds.j.smithi066.stdout:-- Logs begin at Fri 2023-07-21 21:48:02 UTC. -- 2023-07-21T22:18:30.562 INFO:journalctl@ceph.mds.h.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:18:30.563 DEBUG:teuthology.orchestra.run.smithi149:mds.c> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.c.service 2023-07-21T22:18:30.565 DEBUG:teuthology.orchestra.run.smithi149:mds.f> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.f.service 2023-07-21T22:18:30.588 INFO:journalctl@ceph.mds.k.smithi104.stdout:-- Logs begin at Fri 2023-07-21 21:41:18 UTC. -- 2023-07-21T22:18:30.594 INFO:journalctl@ceph.mds.c.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:18:30.595 DEBUG:teuthology.orchestra.run.smithi149:mds.i> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.i.service 2023-07-21T22:18:30.621 INFO:journalctl@ceph.mds.f.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:18:30.622 DEBUG:teuthology.orchestra.run.smithi149:mds.l> sudo journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.l.service 2023-07-21T22:18:30.649 INFO:journalctl@ceph.mds.i.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:18:30.650 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-21T22:18:30.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:18:30.677 INFO:journalctl@ceph.mds.l.smithi149.stdout:-- Logs begin at Fri 2023-07-21 21:41:32 UTC. -- 2023-07-21T22:18:30.868 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 -- 172.21.15.66:0/4226514354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 msgr2=0x7fc548107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:30.868 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/4226514354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 0x7fc548107f40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc54000b0a0 tx=0x7fc54000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 -- 172.21.15.66:0/4226514354 shutdown_connections 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/4226514354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5480ffb00 0x7fc5480fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/4226514354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 0x7fc548107f40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/4226514354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc548100cf0 0x7fc548101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 -- 172.21.15.66:0/4226514354 >> 172.21.15.66:0/4226514354 conn(0x7fc5480fb0e0 msgr2=0x7fc5480fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 -- 172.21.15.66:0/4226514354 shutdown_connections 2023-07-21T22:18:30.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.824+0000 7fc54d9ff700 1 -- 172.21.15.66:0/4226514354 wait complete. 2023-07-21T22:18:30.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 Processor -- start 2023-07-21T22:18:30.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 -- start start 2023-07-21T22:18:30.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5480ffb00 0x7fc548195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:30.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc548100cf0 0x7fc548195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 0x7fc54819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54819d480 con 0x7fc548101680 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54819d5c0 con 0x7fc548100cf0 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.826+0000 7fc54d9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54819d750 con 0x7fc5480ffb00 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc546ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5480ffb00 0x7fc548195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc548100cf0 0x7fc548195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:30.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc546ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5480ffb00 0x7fc548195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:42778/0 (socket says 172.21.15.66:42778) 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc546ffd700 1 -- 172.21.15.66:0/2038577799 learned_addr learned my addr 172.21.15.66:0/2038577799 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5477fe700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 0x7fc54819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 -- 172.21.15.66:0/2038577799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5480ffb00 msgr2=0x7fc548195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5480ffb00 0x7fc548195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 -- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 msgr2=0x7fc54819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 0x7fc54819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:30.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 -- 172.21.15.66:0/2038577799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc540009d00 con 0x7fc548100cf0 2023-07-21T22:18:30.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc5467fc700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc548100cf0 0x7fc548195e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00e9f0 tx=0x7fc53c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:30.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00c9c0 con 0x7fc548100cf0 2023-07-21T22:18:30.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53c010430 con 0x7fc548100cf0 2023-07-21T22:18:30.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54819da30 con 0x7fc548100cf0 2023-07-21T22:18:30.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.827+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c018680 con 0x7fc548100cf0 2023-07-21T22:18:30.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.828+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54819df90 con 0x7fc548100cf0 2023-07-21T22:18:30.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.829+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc548106fa0 con 0x7fc548100cf0 2023-07-21T22:18:30.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.831+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc53c010860 con 0x7fc548100cf0 2023-07-21T22:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.832+0000 7fc54c9fd700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc5300a1220 0x7fc5300a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.833+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc53c014070 con 0x7fc548100cf0 2023-07-21T22:18:30.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.833+0000 7fc546ffd700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc5300a1220 0x7fc5300a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.833+0000 7fc546ffd700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc5300a1220 0x7fc5300a36d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc538005950 tx=0x7fc538009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:30.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:30.834+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc53c0960b0 con 0x7fc548100cf0 2023-07-21T22:18:31.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.047+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc54802c850 con 0x7fc548100cf0 2023-07-21T22:18:31.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.048+0000 7fc54c9fd700 1 -- 172.21.15.66:0/2038577799 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18432 (secure 0 0 0) 0x7fc53c0996d0 con 0x7fc548100cf0 2023-07-21T22:18:31.092 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:31.093 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:18:19.827969+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:18:31.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc5300a1220 msgr2=0x7fc5300a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc5300a1220 0x7fc5300a36d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc538005950 tx=0x7fc538009260 comp rx=0 tx=0).stop 2023-07-21T22:18:31.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc548100cf0 msgr2=0x7fc548195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc548100cf0 0x7fc548195e40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00e9f0 tx=0x7fc53c00ed00 comp rx=0 tx=0).stop 2023-07-21T22:18:31.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 shutdown_connections 2023-07-21T22:18:31.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc5300a1220 0x7fc5300a36d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc548101680 0x7fc54819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5480ffb00 0x7fc548195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 --2- 172.21.15.66:0/2038577799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc548100cf0 0x7fc548195e40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 >> 172.21.15.66:0/2038577799 conn(0x7fc5480fb0e0 msgr2=0x7fc5481040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:31.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 shutdown_connections 2023-07-21T22:18:31.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.051+0000 7fc54d9ff700 1 -- 172.21.15.66:0/2038577799 wait complete. 2023-07-21T22:18:31.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:18:31.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2023-07-21T22:18:31.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:31.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:31.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:31.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:31.113 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:31.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:31.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:31.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:31.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:30 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:31.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:18:31.117 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:18:31.181 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:18:31.181 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2023-07-21T22:18:31.182 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:31.182 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:31.182 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:31.183 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:31.183 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:31.183 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:31.183 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:31.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:31.184 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:30 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 -- 172.21.15.66:0/1714923888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c100cd0 msgr2=0x7f1d3c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 --2- 172.21.15.66:0/1714923888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c101120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009990 tx=0x7f1d2c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 -- 172.21.15.66:0/1714923888 shutdown_connections 2023-07-21T22:18:31.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 --2- 172.21.15.66:0/1714923888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c101660 0x7f1d3c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 --2- 172.21.15.66:0/1714923888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c0ffae0 0x7f1d3c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 --2- 172.21.15.66:0/1714923888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c101120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 -- 172.21.15.66:0/1714923888 >> 172.21.15.66:0/1714923888 conn(0x7f1d3c0fb0e0 msgr2=0x7f1d3c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 -- 172.21.15.66:0/1714923888 shutdown_connections 2023-07-21T22:18:31.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.284+0000 7f1d445ff700 1 -- 172.21.15.66:0/1714923888 wait complete. 2023-07-21T22:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.285+0000 7f1d445ff700 1 Processor -- start 2023-07-21T22:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.285+0000 7f1d445ff700 1 -- start start 2023-07-21T22:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d445ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0ffae0 0x7f1d3c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d445ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d445ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 0x7f1d3c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d445ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c19f5f0 con 0x7f1d3c100cd0 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d445ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c19f730 con 0x7f1d3c0ffae0 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d445ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c19f8c0 con 0x7f1d3c101660 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 0x7f1d3c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 0x7f1d3c19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:42788/0 (socket says 172.21.15.66:42788) 2023-07-21T22:18:31.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d41b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 -- 172.21.15.66:0/2955250724 learned_addr learned my addr 172.21.15.66:0/2955250724 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:31.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d4239b700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0ffae0 0x7f1d3c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 -- 172.21.15.66:0/2955250724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0ffae0 msgr2=0x7f1d3c197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0ffae0 0x7f1d3c197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 -- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c100cd0 msgr2=0x7f1d3c197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d42b9c700 1 -- 172.21.15.66:0/2955250724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0095f0 con 0x7f1d3c101660 2023-07-21T22:18:31.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.286+0000 7f1d41b9a700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:31.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.287+0000 7f1d42b9c700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 0x7f1d3c19f020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400ef30 tx=0x7f1d3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:31.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.287+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34018070 con 0x7f1d3c101660 2023-07-21T22:18:31.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.287+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d3400f040 con 0x7f1d3c101660 2023-07-21T22:18:31.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.287+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c19fb40 con 0x7f1d3c101660 2023-07-21T22:18:31.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.287+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400cd00 con 0x7f1d3c101660 2023-07-21T22:18:31.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.287+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1a0050 con 0x7f1d3c101660 2023-07-21T22:18:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.288+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c1921c0 con 0x7f1d3c101660 2023-07-21T22:18:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.292+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1d34003a00 con 0x7f1d3c101660 2023-07-21T22:18:31.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.294+0000 7f1d337fe700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1d280a11d0 0x7f1d280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.294+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1d340cc090 con 0x7f1d3c101660 2023-07-21T22:18:31.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.294+0000 7f1d4239b700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1d280a11d0 0x7f1d280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.295+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1d34099150 con 0x7f1d3c101660 2023-07-21T22:18:31.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.295+0000 7f1d4239b700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1d280a11d0 0x7f1d280a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c100b30 tx=0x7f1d38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:18:31.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: Saving service mds.all spec with placement smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2023-07-21T22:18:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:18:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:18:31.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:31.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:31.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:31.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:30 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:31.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.507+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d3c02c9e0 con 0x7f1d3c101660 2023-07-21T22:18:31.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.507+0000 7f1d337fe700 1 -- 172.21.15.66:0/2955250724 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d34099330 con 0x7f1d3c101660 2023-07-21T22:18:31.551 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:18:31.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1d280a11d0 msgr2=0x7f1d280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1d280a11d0 0x7f1d280a3680 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c100b30 tx=0x7f1d38009260 comp rx=0 tx=0).stop 2023-07-21T22:18:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 msgr2=0x7f1d3c19f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 0x7f1d3c19f020 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400ef30 tx=0x7f1d3400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 shutdown_connections 2023-07-21T22:18:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1d280a11d0 0x7f1d280a3680 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3c0ffae0 0x7f1d3c197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d3c101660 0x7f1d3c19f020 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 --2- 172.21.15.66:0/2955250724 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d3c100cd0 0x7f1d3c197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 >> 172.21.15.66:0/2955250724 conn(0x7f1d3c0fb0e0 msgr2=0x7f1d3c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:31.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 shutdown_connections 2023-07-21T22:18:31.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.510+0000 7f1d445ff700 1 -- 172.21.15.66:0/2955250724 wait complete. 2023-07-21T22:18:31.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 -- 172.21.15.66:0/3941769087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644074550 msgr2=0x7f06440729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3941769087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644074550 0x7f06440729a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0640009990 tx=0x7f0640009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 -- 172.21.15.66:0/3941769087 shutdown_connections 2023-07-21T22:18:31.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3941769087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06441079c0 0x7f0644109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3941769087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644074550 0x7f06440729a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3941769087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0644072f70 0x7f06440733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 -- 172.21.15.66:0/3941769087 >> 172.21.15.66:0/3941769087 conn(0x7f06440fb0e0 msgr2=0x7f06440fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 -- 172.21.15.66:0/3941769087 shutdown_connections 2023-07-21T22:18:31.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.736+0000 7f064b5ff700 1 -- 172.21.15.66:0/3941769087 wait complete. 2023-07-21T22:18:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.738+0000 7f064b5ff700 1 Processor -- start 2023-07-21T22:18:31.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.738+0000 7f064b5ff700 1 -- start start 2023-07-21T22:18:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064b5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 0x7f0644197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064b5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0644074550 0x7f0644197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064b5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06441079c0 0x7f064419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064b5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419f610 con 0x7f0644072f70 2023-07-21T22:18:31.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064b5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419f750 con 0x7f0644074550 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064b5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419f8e0 con 0x7f06441079c0 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 0x7f0644197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 0x7f0644197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55318/0 (socket says 172.21.15.66:55318) 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 -- 172.21.15.66:0/3676104834 learned_addr learned my addr 172.21.15.66:0/3676104834 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f0648b9a700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0644074550 0x7f0644197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f0649b9c700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06441079c0 0x7f064419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 -- 172.21.15.66:0/3676104834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06441079c0 msgr2=0x7f064419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06441079c0 0x7f064419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 -- 172.21.15.66:0/3676104834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0644074550 msgr2=0x7f0644197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0644074550 0x7f0644197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.739+0000 7f064939b700 1 -- 172.21.15.66:0/3676104834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400095f0 con 0x7f0644072f70 2023-07-21T22:18:31.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.740+0000 7f064939b700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 0x7f0644197a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0640004710 tx=0x7f0640009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.740+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064001d070 con 0x7f0644072f70 2023-07-21T22:18:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.740+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0640021410 con 0x7f0644072f70 2023-07-21T22:18:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.740+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640017650 con 0x7f0644072f70 2023-07-21T22:18:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.740+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064419fb60 con 0x7f0644072f70 2023-07-21T22:18:31.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.740+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064419ff30 con 0x7f0644072f70 2023-07-21T22:18:31.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.741+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644106070 con 0x7f0644072f70 2023-07-21T22:18:31.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.743+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0640021580 con 0x7f0644072f70 2023-07-21T22:18:31.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.744+0000 7f063a7fc700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f06300a11a0 0x7f06300a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:31.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.745+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06400ce000 con 0x7f0644072f70 2023-07-21T22:18:31.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.745+0000 7f0648b9a700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f06300a11a0 0x7f06300a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:31.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.745+0000 7f0648b9a700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f06300a11a0 0x7f06300a3650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0644074270 tx=0x7f0634009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:31.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:31.747+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0640097be0 con 0x7f0644072f70 2023-07-21T22:18:32.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.001+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f064402cbf0 con 0x7f0644072f70 2023-07-21T22:18:32.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.002+0000 7f063a7fc700 1 -- 172.21.15.66:0/3676104834 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f064009b200 con 0x7f0644072f70 2023-07-21T22:18:32.046 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:32.046 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-21T22:18:32.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f06300a11a0 msgr2=0x7f06300a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:32.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f06300a11a0 0x7f06300a3650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0644074270 tx=0x7f0634009260 comp rx=0 tx=0).stop 2023-07-21T22:18:32.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 msgr2=0x7f0644197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:32.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 0x7f0644197a90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0640004710 tx=0x7f0640009580 comp rx=0 tx=0).stop 2023-07-21T22:18:32.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 shutdown_connections 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f06300a11a0 0x7f06300a3650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06441079c0 0x7f064419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0644072f70 0x7f0644197a90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 --2- 172.21.15.66:0/3676104834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0644074550 0x7f0644197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 >> 172.21.15.66:0/3676104834 conn(0x7f06440fb0e0 msgr2=0x7f06440fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 shutdown_connections 2023-07-21T22:18:32.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:32.004+0000 7f064b5ff700 1 -- 172.21.15.66:0/3676104834 wait complete. 2023-07-21T22:18:32.050 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 1 2023-07-21T22:18:32.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:31 smithi066 ceph-mon[126379]: Deploying daemon mds.l on smithi149 2023-07-21T22:18:32.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2038577799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:18:32.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2955250724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:18:32.238 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:18:32 smithi149 systemd[1]: Starting Ceph mds.l for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:31 smithi149 ceph-mon[129268]: Deploying daemon mds.l on smithi149 2023-07-21T22:18:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2038577799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:18:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2955250724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:18:32.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:31 smithi104 ceph-mon[129021]: Deploying daemon mds.l on smithi149 2023-07-21T22:18:32.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2038577799' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:18:32.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2955250724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:18:32.715 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:18:32 smithi149 podman[154393]: 2023-07-21T22:18:32.989 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mds[154434]: -- 172.21.15.149:0/1854289531 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d8051ef1e0 con 0x55d8051c4c00 2023-07-21T22:18:32.989 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:18:32 smithi149 bash[154393]: 91ff1ef556ebd3f9c76ef549d121219f00ee39529aa55d3a87736059c6d45d82 2023-07-21T22:18:32.989 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-l[154410]: starting mds.l at 2023-07-21T22:18:32.990 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:18:32 smithi149 systemd[1]: Started Ceph mds.l for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:32.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3676104834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:32.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:32.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6832/2009309286,v1:172.21.15.149:6833/2009309286] up:boot 2023-07-21T22:18:32.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: fsmap 1 up:standby 2023-07-21T22:18:32.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:18:32.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:32.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:32.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:32.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:32.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:32.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:32 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3676104834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6832/2009309286,v1:172.21.15.149:6833/2009309286] up:boot 2023-07-21T22:18:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: fsmap 1 up:standby 2023-07-21T22:18:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:18:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:32 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:33.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3676104834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:33.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6832/2009309286,v1:172.21.15.149:6833/2009309286] up:boot 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: fsmap 1 up:standby 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:33.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:33.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:33.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:32 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:34.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:34.083 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:33 smithi104 ceph-mon[129021]: Deploying daemon mds.k on smithi104 2023-07-21T22:18:34.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:33 smithi066 ceph-mon[126379]: Deploying daemon mds.k on smithi104 2023-07-21T22:18:34.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:33 smithi149 ceph-mon[129268]: Deploying daemon mds.k on smithi104 2023-07-21T22:18:34.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/3394109033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 msgr2=0x7f9b38101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:34.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/3394109033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38101140 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0098f0 tx=0x7f9b2c009c00 comp rx=0 tx=0).stop 2023-07-21T22:18:34.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/3394109033 shutdown_connections 2023-07-21T22:18:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/3394109033 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b38101680 0x7f9b38107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/3394109033 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b380ffb00 0x7f9b380fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/3394109033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38101140 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/3394109033 >> 172.21.15.66:0/3394109033 conn(0x7f9b380fb0e0 msgr2=0x7f9b380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/3394109033 shutdown_connections 2023-07-21T22:18:34.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.242+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/3394109033 wait complete. 2023-07-21T22:18:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.243+0000 7f9b3d5ff700 1 Processor -- start 2023-07-21T22:18:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.243+0000 7f9b3d5ff700 1 -- start start 2023-07-21T22:18:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.243+0000 7f9b3d5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b380ffb00 0x7f9b38195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b3d5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:34.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b3d5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b38101680 0x7f9b3819cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b3d5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3819d490 con 0x7f9b380ffb00 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b3d5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3819d5d0 con 0x7f9b38100cf0 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b3d5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3819d760 con 0x7f9b38101680 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b36ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b380ffb00 0x7f9b38195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:58602/0 (socket says 172.21.15.66:58602) 2023-07-21T22:18:34.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 -- 172.21.15.66:0/2482062479 learned_addr learned my addr 172.21.15.66:0/2482062479 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b377fe700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b38101680 0x7f9b3819cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 -- 172.21.15.66:0/2482062479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b38101680 msgr2=0x7f9b3819cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b38101680 0x7f9b3819cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 -- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b380ffb00 msgr2=0x7f9b38195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b380ffb00 0x7f9b38195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 -- 172.21.15.66:0/2482062479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c009520 con 0x7f9b38100cf0 2023-07-21T22:18:34.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.244+0000 7f9b367fc700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38195e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c005e90 tx=0x7f9b2c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.245+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c01c070 con 0x7f9b38100cf0 2023-07-21T22:18:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.245+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b2c00f7f0 con 0x7f9b38100cf0 2023-07-21T22:18:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.245+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3819d9e0 con 0x7f9b38100cf0 2023-07-21T22:18:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.245+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c004b70 con 0x7f9b38100cf0 2023-07-21T22:18:34.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.245+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3819de60 con 0x7f9b38100cf0 2023-07-21T22:18:34.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.246+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b381070e0 con 0x7f9b38100cf0 2023-07-21T22:18:34.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.248+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9b2c020070 con 0x7f9b38100cf0 2023-07-21T22:18:34.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.249+0000 7f9b1ffff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b200a1220 0x7f9b200a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:34.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.249+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9b2c0cd410 con 0x7f9b38100cf0 2023-07-21T22:18:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.249+0000 7f9b36ffd700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b200a1220 0x7f9b200a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.250+0000 7f9b36ffd700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b200a1220 0x7f9b200a36d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b38100b50 tx=0x7f9b28009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:34.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.251+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b2c096ff0 con 0x7f9b38100cf0 2023-07-21T22:18:34.368 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:18:34 smithi104 systemd[1]: Starting Ceph mds.k for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:34.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.508+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b3802c850 con 0x7f9b38100cf0 2023-07-21T22:18:34.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.509+0000 7f9b1ffff700 1 -- 172.21.15.66:0/2482062479 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f9b2c09a610 con 0x7f9b38100cf0 2023-07-21T22:18:34.552 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:34.553 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T22:18:34.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b200a1220 msgr2=0x7f9b200a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:34.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b200a1220 0x7f9b200a36d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9b38100b50 tx=0x7f9b28009310 comp rx=0 tx=0).stop 2023-07-21T22:18:34.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 msgr2=0x7f9b38195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38195e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c005e90 tx=0x7f9b2c00b7e0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 shutdown_connections 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b200a1220 0x7f9b200a36d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b38101680 0x7f9b3819cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b380ffb00 0x7f9b38195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 --2- 172.21.15.66:0/2482062479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38100cf0 0x7f9b38195e50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:34.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 >> 172.21.15.66:0/2482062479 conn(0x7f9b380fb0e0 msgr2=0x7f9b38104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:34.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 shutdown_connections 2023-07-21T22:18:34.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:34.511+0000 7f9b3d5ff700 1 -- 172.21.15.66:0/2482062479 wait complete. 2023-07-21T22:18:34.557 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 2 2023-07-21T22:18:34.904 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:18:34 smithi104 podman[153331]: 2023-07-21T22:18:34.905 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:18:34 smithi104 ceph-mds[153371]: -- 172.21.15.104:0/946693102 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5572682451e0 con 0x55726821ac00 2023-07-21T22:18:34.905 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:18:34 smithi104 bash[153331]: d7a1a1904531cfd5582701b2daed40202b4c156f5fae27774692d80602896f48 2023-07-21T22:18:34.905 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:18:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-k[153347]: starting mds.k at 2023-07-21T22:18:35.194 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:34 smithi066 ceph-mon[126379]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:35.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2482062479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:35.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:34 smithi149 ceph-mon[129268]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:35.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2482062479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:35.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:34 smithi104 ceph-mon[129021]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:35.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2482062479' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:35.369 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:18:34 smithi104 systemd[1]: Started Ceph mds.k for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:36.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6832/3341651114,v1:172.21.15.104:6833/3341651114] up:boot 2023-07-21T22:18:36.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: fsmap 2 up:standby 2023-07-21T22:18:36.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:18:36.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:36.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:36.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:36.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:35 smithi066 ceph-mon[126379]: Deploying daemon mds.j on smithi066 2023-07-21T22:18:36.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6832/3341651114,v1:172.21.15.104:6833/3341651114] up:boot 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: fsmap 2 up:standby 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:36.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:36.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:35 smithi149 ceph-mon[129268]: Deploying daemon mds.j on smithi066 2023-07-21T22:18:36.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6832/3341651114,v1:172.21.15.104:6833/3341651114] up:boot 2023-07-21T22:18:36.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: fsmap 2 up:standby 2023-07-21T22:18:36.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:35 smithi104 ceph-mon[129021]: Deploying daemon mds.j on smithi066 2023-07-21T22:18:36.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:36.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.923+0000 7f34549ff700 1 -- 172.21.15.66:0/1522867018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3450100cf0 msgr2=0x7f3450101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:36.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.923+0000 7f34549ff700 1 --2- 172.21.15.66:0/1522867018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3450100cf0 0x7f3450101140 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3444009990 tx=0x7f3444009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:36.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 -- 172.21.15.66:0/1522867018 shutdown_connections 2023-07-21T22:18:36.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 --2- 172.21.15.66:0/1522867018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450101680 0x7f3450107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:36.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 --2- 172.21.15.66:0/1522867018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34500ffb00 0x7f34500fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:36.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 --2- 172.21.15.66:0/1522867018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3450100cf0 0x7f3450101140 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:36.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 -- 172.21.15.66:0/1522867018 >> 172.21.15.66:0/1522867018 conn(0x7f34500fb0e0 msgr2=0x7f34500fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:36.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 -- 172.21.15.66:0/1522867018 shutdown_connections 2023-07-21T22:18:36.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.924+0000 7f34549ff700 1 -- 172.21.15.66:0/1522867018 wait complete. 2023-07-21T22:18:36.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.925+0000 7f34549ff700 1 Processor -- start 2023-07-21T22:18:36.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.925+0000 7f34549ff700 1 -- start start 2023-07-21T22:18:36.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.925+0000 7f34549ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 0x7f34500742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:36.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.925+0000 7f34549ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450100cf0 0x7f3450072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:36.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.925+0000 7f34549ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450101680 0x7f3450072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:36.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f34549ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500735e0 con 0x7f34500ffb00 2023-07-21T22:18:36.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f34549ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3450073750 con 0x7f3450100cf0 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f34549ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345019f880 con 0x7f3450101680 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 0x7f34500742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 0x7f34500742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54986/0 (socket says 172.21.15.66:54986) 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 -- 172.21.15.66:0/1040286017 learned_addr learned my addr 172.21.15.66:0/1040286017 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344ed9d700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450101680 0x7f3450072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344dd9b700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450100cf0 0x7f3450072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:36.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 -- 172.21.15.66:0/1040286017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450101680 msgr2=0x7f3450072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450101680 0x7f3450072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 -- 172.21.15.66:0/1040286017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450100cf0 msgr2=0x7f3450072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450100cf0 0x7f3450072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 -- 172.21.15.66:0/1040286017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34440095f0 con 0x7f34500ffb00 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344dd9b700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450100cf0 0x7f3450072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.926+0000 7f344e59c700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 0x7f34500742c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f344000e970 tx=0x7f344000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.927+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440009790 con 0x7f34500ffb00 2023-07-21T22:18:36.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.927+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344000eec0 con 0x7f34500ffb00 2023-07-21T22:18:36.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.927+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345019fb60 con 0x7f34500ffb00 2023-07-21T22:18:36.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.927+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440003cf0 con 0x7f34500ffb00 2023-07-21T22:18:36.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.927+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34501a00c0 con 0x7f34500ffb00 2023-07-21T22:18:36.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.928+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3450106ff0 con 0x7f34500ffb00 2023-07-21T22:18:36.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.930+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f344000ccd0 con 0x7f34500ffb00 2023-07-21T22:18:36.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.931+0000 7f343f7fe700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34380a1220 0x7f34380a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:36.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.932+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3440014070 con 0x7f34500ffb00 2023-07-21T22:18:36.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.932+0000 7f344dd9b700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34380a1220 0x7f34380a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:36.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.932+0000 7f344dd9b700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34380a1220 0x7f34380a36d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f3444005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:36.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:36.936+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3440096210 con 0x7f34500ffb00 2023-07-21T22:18:37.147 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:36 smithi066 ceph-mon[126379]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:37.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.195+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f345002c850 con 0x7f34500ffb00 2023-07-21T22:18:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:36 smithi149 ceph-mon[129268]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:37.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.195+0000 7f343f7fe700 1 -- 172.21.15.66:0/1040286017 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f345002c850 con 0x7f34500ffb00 2023-07-21T22:18:37.240 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:37.240 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T22:18:37.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.197+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34380a1220 msgr2=0x7f34380a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:37.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.197+0000 7f34549ff700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34380a1220 0x7f34380a36d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3444000c00 tx=0x7f3444005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:37.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 msgr2=0x7f34500742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:37.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 0x7f34500742c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f344000e970 tx=0x7f344000ec80 comp rx=0 tx=0).stop 2023-07-21T22:18:37.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 shutdown_connections 2023-07-21T22:18:37.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34380a1220 0x7f34380a36d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:37.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3450101680 0x7f3450072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:37.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34500ffb00 0x7f34500742c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:37.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 --2- 172.21.15.66:0/1040286017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3450100cf0 0x7f3450072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:37.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 >> 172.21.15.66:0/1040286017 conn(0x7f34500fb0e0 msgr2=0x7f3450104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:37.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 shutdown_connections 2023-07-21T22:18:37.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:37.198+0000 7f34549ff700 1 -- 172.21.15.66:0/1040286017 wait complete. 2023-07-21T22:18:37.244 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 3 2023-07-21T22:18:37.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:36 smithi104 ceph-mon[129021]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:37.445 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:18:37 smithi066 systemd[1]: Starting Ceph mds.j for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:37.930 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:18:37 smithi066 podman[159097]: 2023-07-21T22:18:37.930 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:18:37 smithi066 bash[159097]: 7e707ab90db155f60af4f257b8dfb8d17e78f965c0bcb58f1179ab212ca79586 2023-07-21T22:18:37.931 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:18:37 smithi066 ceph-mds[159137]: -- 172.21.15.66:0/3185823350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c719f91e0 con 0x564c719cec00 2023-07-21T22:18:37.931 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:18:37 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-j[159113]: starting mds.j at 2023-07-21T22:18:38.195 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:18:37 smithi066 systemd[1]: Started Ceph mds.j for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:38.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:37 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1040286017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:38.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:37 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1040286017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:38.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:37 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1040286017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:39.237 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.237 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.237 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.237 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:39.237 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:39.237 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:39.238 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: Deploying daemon mds.d on smithi066 2023-07-21T22:18:39.238 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:39.238 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.66:6834/1071874116,v1:172.21.15.66:6835/1071874116] up:boot 2023-07-21T22:18:39.238 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: fsmap 3 up:standby 2023-07-21T22:18:39.238 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:18:39.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:39.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: Deploying daemon mds.d on smithi066 2023-07-21T22:18:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:39.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.66:6834/1071874116,v1:172.21.15.66:6835/1071874116] up:boot 2023-07-21T22:18:39.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: fsmap 3 up:standby 2023-07-21T22:18:39.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:18:39.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 -- 172.21.15.66:0/1915203660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38074550 msgr2=0x7fdc380729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:39.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/1915203660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38074550 0x7fdc380729a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009990 tx=0x7fdc28009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 -- 172.21.15.66:0/1915203660 shutdown_connections 2023-07-21T22:18:39.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/1915203660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc381079c0 0x7fdc38109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/1915203660 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38074550 0x7fdc380729a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/1915203660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc38072f70 0x7fdc380733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 -- 172.21.15.66:0/1915203660 >> 172.21.15.66:0/1915203660 conn(0x7fdc380fb0e0 msgr2=0x7fdc380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:39.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 -- 172.21.15.66:0/1915203660 shutdown_connections 2023-07-21T22:18:39.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.434+0000 7fdc3cdff700 1 -- 172.21.15.66:0/1915203660 wait complete. 2023-07-21T22:18:39.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 Processor -- start 2023-07-21T22:18:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 -- start start 2023-07-21T22:18:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 0x7fdc38197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc38074550 0x7fdc38197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc381079c0 0x7fdc3819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3819f5d0 con 0x7fdc38072f70 2023-07-21T22:18:39.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3819f710 con 0x7fdc381079c0 2023-07-21T22:18:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3cdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3819f8a0 con 0x7fdc38074550 2023-07-21T22:18:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.436+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 0x7fdc38197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 0x7fdc38197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55014/0 (socket says 172.21.15.66:55014) 2023-07-21T22:18:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 -- 172.21.15.66:0/2049431575 learned_addr learned my addr 172.21.15.66:0/2049431575 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc36d9d700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc381079c0 0x7fdc3819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:39.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc35d9b700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc38074550 0x7fdc38197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 -- 172.21.15.66:0/2049431575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc38074550 msgr2=0x7fdc38197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc38074550 0x7fdc38197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 -- 172.21.15.66:0/2049431575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc381079c0 msgr2=0x7fdc3819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc381079c0 0x7fdc3819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 -- 172.21.15.66:0/2049431575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280095f0 con 0x7fdc38072f70 2023-07-21T22:18:39.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc3659c700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 0x7fdc38197a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b680 tx=0x7fdc28005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.437+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2801d070 con 0x7fdc38072f70 2023-07-21T22:18:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.438+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3819fb20 con 0x7fdc38072f70 2023-07-21T22:18:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.438+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc28021410 con 0x7fdc38072f70 2023-07-21T22:18:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.438+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28017650 con 0x7fdc38072f70 2023-07-21T22:18:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.438+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3819ffa0 con 0x7fdc38072f70 2023-07-21T22:18:39.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.439+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc38106070 con 0x7fdc38072f70 2023-07-21T22:18:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.441+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdc28021580 con 0x7fdc38072f70 2023-07-21T22:18:39.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.443+0000 7fdc277fe700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdc200a11a0 0x7fdc200a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.443+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdc280cd990 con 0x7fdc38072f70 2023-07-21T22:18:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.444+0000 7fdc35d9b700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdc200a11a0 0x7fdc200a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:39.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.444+0000 7fdc35d9b700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdc200a11a0 0x7fdc200a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc38074270 tx=0x7fdc2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:39.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:39.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:39.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:39.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:39.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: Deploying daemon mds.d on smithi066 2023-07-21T22:18:39.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:39.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.66:6834/1071874116,v1:172.21.15.66:6835/1071874116] up:boot 2023-07-21T22:18:39.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: fsmap 3 up:standby 2023-07-21T22:18:39.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:18:39.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.448+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdc28097570 con 0x7fdc38072f70 2023-07-21T22:18:39.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.728+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc3802c850 con 0x7fdc38072f70 2023-07-21T22:18:39.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.729+0000 7fdc277fe700 1 -- 172.21.15.66:0/2049431575 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2777 (secure 0 0 0) 0x7fdc2809ab90 con 0x7fdc38072f70 2023-07-21T22:18:39.773 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:39.773 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T22:18:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdc200a11a0 msgr2=0x7fdc200a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdc200a11a0 0x7fdc200a3650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc38074270 tx=0x7fdc2c009260 comp rx=0 tx=0).stop 2023-07-21T22:18:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 msgr2=0x7fdc38197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:39.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 0x7fdc38197a50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b680 tx=0x7fdc28005740 comp rx=0 tx=0).stop 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 shutdown_connections 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdc200a11a0 0x7fdc200a3650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc381079c0 0x7fdc3819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc38072f70 0x7fdc38197a50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 --2- 172.21.15.66:0/2049431575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc38074550 0x7fdc38197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 >> 172.21.15.66:0/2049431575 conn(0x7fdc380fb0e0 msgr2=0x7fdc380fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:39.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 shutdown_connections 2023-07-21T22:18:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:39.731+0000 7fdc3cdff700 1 -- 172.21.15.66:0/2049431575 wait complete. 2023-07-21T22:18:39.777 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 4 2023-07-21T22:18:40.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:39 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2049431575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:40.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:39 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2049431575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:40.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:39 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2049431575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:40.692 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:18:40 smithi066 systemd[1]: Starting Ceph mds.d for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:40.945 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:18:40 smithi066 podman[159487]: 2023-07-21T22:18:40.945 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:18:40 smithi066 bash[159487]: ed25230da3ce99c4b8bf71af2f6d5cfebed8a26764bf22f96907edf28c0ea1b0 2023-07-21T22:18:40.945 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:18:40 smithi066 ceph-mds[159527]: -- 172.21.15.66:0/2546742492 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584e8ced1e0 con 0x5584e8cc2800 2023-07-21T22:18:40.946 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:18:40 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-d[159503]: starting mds.d at 2023-07-21T22:18:41.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:41 smithi104 ceph-mon[129021]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:41.445 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:18:40 smithi066 systemd[1]: Started Ceph mds.d for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:41.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:41 smithi066 ceph-mon[126379]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:41.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:41 smithi149 ceph-mon[129268]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:41.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:42.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2003588969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4100550 msgr2=0x7f25e4102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:42.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2003588969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4100550 0x7f25e4102930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009990 tx=0x7f25e0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2003588969 shutdown_connections 2023-07-21T22:18:42.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2003588969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4100550 0x7f25e4102930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2003588969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25e4105790 0x7f25e4107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2003588969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25e4102e70 0x7f25e4105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.974+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2003588969 >> 172.21.15.66:0/2003588969 conn(0x7f25e40f9fd0 msgr2=0x7f25e40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.975+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2003588969 shutdown_connections 2023-07-21T22:18:42.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.975+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2003588969 wait complete. 2023-07-21T22:18:42.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 Processor -- start 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 -- start start 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25e4100550 0x7f25e41979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 0x7f25e4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25e4105790 0x7f25e419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e419f570 con 0x7f25e4105790 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e419f6b0 con 0x7f25e4102e70 2023-07-21T22:18:42.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ec1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e419f840 con 0x7f25e4100550 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25e979a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 0x7f25e4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25e979a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 0x7f25e4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:52804/0 (socket says 172.21.15.66:52804) 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25e979a700 1 -- 172.21.15.66:0/2482493023 learned_addr learned my addr 172.21.15.66:0/2482493023 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25ea79c700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25e4105790 0x7f25e419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.976+0000 7f25e9f9b700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25e4100550 0x7f25e41979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25e979a700 1 -- 172.21.15.66:0/2482493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25e4100550 msgr2=0x7f25e41979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:42.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25e979a700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25e4100550 0x7f25e41979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25e979a700 1 -- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25e4105790 msgr2=0x7f25e419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25e979a700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25e4105790 0x7f25e419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25e979a700 1 -- 172.21.15.66:0/2482493023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00095f0 con 0x7f25e4102e70 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25e979a700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 0x7f25e4197f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25d400e9a0 tx=0x7f25d400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4009790 con 0x7f25e4102e70 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e419fb20 con 0x7f25e4102e70 2023-07-21T22:18:42.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d4010430 con 0x7f25e4102e70 2023-07-21T22:18:42.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4018680 con 0x7f25e4102e70 2023-07-21T22:18:42.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.977+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e41a0080 con 0x7f25e4102e70 2023-07-21T22:18:42.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.978+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e4107350 con 0x7f25e4102e70 2023-07-21T22:18:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.980+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25d40105a0 con 0x7f25e4102e70 2023-07-21T22:18:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.982+0000 7f25daffd700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f25d00a11a0 0x7f25d00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.982+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25d4014070 con 0x7f25e4102e70 2023-07-21T22:18:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.982+0000 7f25e9f9b700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f25d00a11a0 0x7f25d00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:42.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.983+0000 7f25e9f9b700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f25d00a11a0 0x7f25d00a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f25e000b2e0 tx=0x7f25e0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:42.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:41.986+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f25d4096190 con 0x7f25e4102e70 2023-07-21T22:18:42.104 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.66:6836/1892710488,v1:172.21.15.66:6837/1892710488] up:boot 2023-07-21T22:18:42.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: fsmap 4 up:standby 2023-07-21T22:18:42.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:18:42.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.105 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:42.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:42.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:42.106 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:42 smithi104 ceph-mon[129021]: Deploying daemon mds.e on smithi104 2023-07-21T22:18:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.66:6836/1892710488,v1:172.21.15.66:6837/1892710488] up:boot 2023-07-21T22:18:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: fsmap 4 up:standby 2023-07-21T22:18:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:18:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:42 smithi066 ceph-mon[126379]: Deploying daemon mds.e on smithi104 2023-07-21T22:18:42.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.240+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e402c850 con 0x7f25e4102e70 2023-07-21T22:18:42.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.241+0000 7f25daffd700 1 -- 172.21.15.66:0/2482493023 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3534 (secure 0 0 0) 0x7f25d40997b0 con 0x7f25e4102e70 2023-07-21T22:18:42.285 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:42.285 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[]} 2023-07-21T22:18:42.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f25d00a11a0 msgr2=0x7f25d00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:42.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f25d00a11a0 0x7f25d00a3650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f25e000b2e0 tx=0x7f25e0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 msgr2=0x7f25e4197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:42.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 0x7f25e4197f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25d400e9a0 tx=0x7f25d400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 shutdown_connections 2023-07-21T22:18:42.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f25d00a11a0 0x7f25d00a3650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f25e4105790 0x7f25e419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f25e4100550 0x7f25e41979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 --2- 172.21.15.66:0/2482493023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25e4102e70 0x7f25e4197f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 >> 172.21.15.66:0/2482493023 conn(0x7f25e40f9fd0 msgr2=0x7f25e40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 shutdown_connections 2023-07-21T22:18:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:42.243+0000 7f25ec1ff700 1 -- 172.21.15.66:0/2482493023 wait complete. 2023-07-21T22:18:42.291 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 5 2023-07-21T22:18:42.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.66:6836/1892710488,v1:172.21.15.66:6837/1892710488] up:boot 2023-07-21T22:18:42.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: fsmap 4 up:standby 2023-07-21T22:18:42.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:18:42.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:42 smithi149 ceph-mon[129268]: Deploying daemon mds.e on smithi104 2023-07-21T22:18:42.876 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:18:42 smithi104 systemd[1]: Starting Ceph mds.e for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:43.177 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:18:42 smithi104 podman[153672]: 2023-07-21T22:18:43.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:43 smithi104 ceph-mon[129021]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:43.178 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:43 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2482493023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:43.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:43 smithi066 ceph-mon[126379]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:43.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:43 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2482493023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:43.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:43 smithi149 ceph-mon[129268]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:43.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:43 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2482493023' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:43.618 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:18:43 smithi104 ceph-mds[153713]: -- 172.21.15.104:0/277626935 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557b633111e0 con 0x557b632e7000 2023-07-21T22:18:43.618 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:18:43 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e[153688]: starting mds.e at 2023-07-21T22:18:43.618 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:18:43 smithi104 bash[153672]: 1930cab105f5cd141164ffdea7ca3fb78cb88f504bb6b325b9d5c9866302028d 2023-07-21T22:18:43.619 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:18:43 smithi104 systemd[1]: Started Ceph mds.e for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:44.306 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:44.523 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:44.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:44.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:44.524 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: Deploying daemon mds.f on smithi149 2023-07-21T22:18:44.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6834/1273148910,v1:172.21.15.104:6835/1273148910] up:boot 2023-07-21T22:18:44.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: fsmap 5 up:standby 2023-07-21T22:18:44.525 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:44 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:18:44.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 -- 172.21.15.66:0/3323436345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 msgr2=0x7fed000fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:44.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 --2- 172.21.15.66:0/3323436345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed000fff10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009990 tx=0x7fecf0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 -- 172.21.15.66:0/3323436345 shutdown_connections 2023-07-21T22:18:44.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 --2- 172.21.15.66:0/3323436345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed00101680 0x7fed00107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 --2- 172.21.15.66:0/3323436345 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed000fff10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 --2- 172.21.15.66:0/3323436345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed00100cf0 0x7fed00101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 -- 172.21.15.66:0/3323436345 >> 172.21.15.66:0/3323436345 conn(0x7fed000fb0e0 msgr2=0x7fed000fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:44.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 -- 172.21.15.66:0/3323436345 shutdown_connections 2023-07-21T22:18:44.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.479+0000 7fed065ff700 1 -- 172.21.15.66:0/3323436345 wait complete. 2023-07-21T22:18:44.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.480+0000 7fed065ff700 1 Processor -- start 2023-07-21T22:18:44.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.480+0000 7fed065ff700 1 -- start start 2023-07-21T22:18:44.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed065ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed00197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:44.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed065ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed00100cf0 0x7fed00197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed065ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed00101680 0x7fed0019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed065ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019f600 con 0x7fed000ffb00 2023-07-21T22:18:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed065ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019f740 con 0x7fed00101680 2023-07-21T22:18:44.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed065ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019f8d0 con 0x7fed00100cf0 2023-07-21T22:18:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed00197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:44.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed00197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:45852/0 (socket says 172.21.15.66:45852) 2023-07-21T22:18:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 -- 172.21.15.66:0/362330852 learned_addr learned my addr 172.21.15.66:0/362330852 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecff7fe700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed00100cf0 0x7fed00197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fed04b9c700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed00101680 0x7fed0019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:44.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 -- 172.21.15.66:0/362330852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed00100cf0 msgr2=0x7fed00197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed00100cf0 0x7fed00197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 -- 172.21.15.66:0/362330852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed00101680 msgr2=0x7fed0019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:44.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed00101680 0x7fed0019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.481+0000 7fecfffff700 1 -- 172.21.15.66:0/362330852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf00095f0 con 0x7fed000ffb00 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.482+0000 7fecfffff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed00197a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fecf000b620 tx=0x7fecf0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.482+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf001d070 con 0x7fed000ffb00 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.482+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0019fb50 con 0x7fed000ffb00 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.482+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecf0021410 con 0x7fed000ffb00 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.482+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0017650 con 0x7fed000ffb00 2023-07-21T22:18:44.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.482+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0019ff80 con 0x7fed000ffb00 2023-07-21T22:18:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.483+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed001921e0 con 0x7fed000ffb00 2023-07-21T22:18:44.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.485+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fecf0021580 con 0x7fed000ffb00 2023-07-21T22:18:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.487+0000 7fecfd7fa700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fece80a11a0 0x7fece80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.487+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fecf00ce0e0 con 0x7fed000ffb00 2023-07-21T22:18:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.487+0000 7fecff7fe700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fece80a11a0 0x7fece80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.488+0000 7fecff7fe700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fece80a11a0 0x7fece80a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009c60 tx=0x7fecf4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:44.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.489+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fecf0097cc0 con 0x7fed000ffb00 2023-07-21T22:18:44.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:44.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:44.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:44.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: Deploying daemon mds.f on smithi149 2023-07-21T22:18:44.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6834/1273148910,v1:172.21.15.104:6835/1273148910] up:boot 2023-07-21T22:18:44.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: fsmap 5 up:standby 2023-07-21T22:18:44.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:44 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:18:44.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:44.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:44.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:44.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:44.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: Deploying daemon mds.f on smithi149 2023-07-21T22:18:44.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6834/1273148910,v1:172.21.15.104:6835/1273148910] up:boot 2023-07-21T22:18:44.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: fsmap 5 up:standby 2023-07-21T22:18:44.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:44 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:18:44.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.748+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed0002c850 con 0x7fed000ffb00 2023-07-21T22:18:44.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.748+0000 7fecfd7fa700 1 -- 172.21.15.66:0/362330852 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4294 (secure 0 0 0) 0x7fecf009b2e0 con 0x7fed000ffb00 2023-07-21T22:18:44.793 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:44.794 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T22:18:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.750+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fece80a11a0 msgr2=0x7fece80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:44.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.750+0000 7fed065ff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fece80a11a0 0x7fece80a3650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009c60 tx=0x7fecf4009260 comp rx=0 tx=0).stop 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.750+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 msgr2=0x7fed00197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.750+0000 7fed065ff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed00197a80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fecf000b620 tx=0x7fecf0005740 comp rx=0 tx=0).stop 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 shutdown_connections 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fece80a11a0 0x7fece80a3650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed00101680 0x7fed0019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fed000ffb00 0x7fed00197a80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 --2- 172.21.15.66:0/362330852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed00100cf0 0x7fed00197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:44.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 >> 172.21.15.66:0/362330852 conn(0x7fed000fb0e0 msgr2=0x7fed001065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 shutdown_connections 2023-07-21T22:18:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:44.751+0000 7fed065ff700 1 -- 172.21.15.66:0/362330852 wait complete. 2023-07-21T22:18:44.797 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 6 2023-07-21T22:18:45.239 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:18:44 smithi149 systemd[1]: Starting Ceph mds.f for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:45.530 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:18:45 smithi149 podman[154741]: 2023-07-21T22:18:45.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:45 smithi149 ceph-mon[129268]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:45.530 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:45 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/362330852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:45.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:45 smithi104 ceph-mon[129021]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:45.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:45 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/362330852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:45 smithi066 ceph-mon[126379]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:45.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:45 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/362330852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:45.988 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:18:45 smithi149 ceph-mds[154781]: -- 172.21.15.149:0/3558070019 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562287c911e0 con 0x562287c66c00 2023-07-21T22:18:45.989 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:18:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f[154757]: starting mds.f at 2023-07-21T22:18:45.989 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:18:45 smithi149 bash[154741]: dd9c9d461778a17197d6c0603f142602301a1be17df17b87a6583da28520e014 2023-07-21T22:18:45.989 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:18:45 smithi149 systemd[1]: Started Ceph mds.f for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:46.815 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:47.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 -- 172.21.15.66:0/4045683010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 msgr2=0x7fe4540fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:47.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 --2- 172.21.15.66:0/4045683010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540fff10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe444009990 tx=0x7fe444009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 -- 172.21.15.66:0/4045683010 shutdown_connections 2023-07-21T22:18:47.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 --2- 172.21.15.66:0/4045683010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 0x7fe454107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 --2- 172.21.15.66:0/4045683010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540fff10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 --2- 172.21.15.66:0/4045683010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe454100cf0 0x7fe454101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 -- 172.21.15.66:0/4045683010 >> 172.21.15.66:0/4045683010 conn(0x7fe4540fb0e0 msgr2=0x7fe4540fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:47.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 -- 172.21.15.66:0/4045683010 shutdown_connections 2023-07-21T22:18:47.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.992+0000 7fe4595ff700 1 -- 172.21.15.66:0/4045683010 wait complete. 2023-07-21T22:18:47.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 Processor -- start 2023-07-21T22:18:47.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 -- start start 2023-07-21T22:18:47.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:47.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe454100cf0 0x7fe454072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:47.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 0x7fe454072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:47.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540735e0 con 0x7fe454101680 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe454073750 con 0x7fe454100cf0 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.993+0000 7fe4595ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45419f850 con 0x7fe4540ffb00 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe4537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 0x7fe454072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35680/0 (socket says 172.21.15.66:35680) 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 -- 172.21.15.66:0/2679201783 learned_addr learned my addr 172.21.15.66:0/2679201783 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:47.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe44bfff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe454100cf0 0x7fe454072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 -- 172.21.15.66:0/2679201783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe454100cf0 msgr2=0x7fe454072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe454100cf0 0x7fe454072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 -- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 msgr2=0x7fe454072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 0x7fe454072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 -- 172.21.15.66:0/2679201783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440095f0 con 0x7fe4540ffb00 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe4537fe700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 0x7fe454072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.994+0000 7fe452ffd700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540742c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe44400b2e0 tx=0x7fe444004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:47.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.995+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44401d070 con 0x7fe4540ffb00 2023-07-21T22:18:47.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.995+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe444005030 con 0x7fe4540ffb00 2023-07-21T22:18:47.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.995+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45419fad0 con 0x7fe4540ffb00 2023-07-21T22:18:47.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.995+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444004c20 con 0x7fe4540ffb00 2023-07-21T22:18:47.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.995+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45419fef0 con 0x7fe4540ffb00 2023-07-21T22:18:47.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.996+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45410b2c0 con 0x7fe4540ffb00 2023-07-21T22:18:47.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.998+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe444004d80 con 0x7fe4540ffb00 2023-07-21T22:18:47.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:46.999+0000 7fe450ff9700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe4400a11a0 0x7fe4400a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:47.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.000+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4440cdad0 con 0x7fe4540ffb00 2023-07-21T22:18:47.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.000+0000 7fe44bfff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe4400a11a0 0x7fe4400a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:47.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.000+0000 7fe44bfff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe4400a11a0 0x7fe4400a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009c60 tx=0x7fe43c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:47.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.002+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe4440976b0 con 0x7fe4540ffb00 2023-07-21T22:18:47.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: Deploying daemon mds.c on smithi149 2023-07-21T22:18:47.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:47.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6834/1642617037,v1:172.21.15.149:6835/1642617037] up:boot 2023-07-21T22:18:47.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: fsmap 6 up:standby 2023-07-21T22:18:47.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:46 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:18:47.123 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.124 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.124 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.124 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:47.124 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:47.124 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:47.124 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: Deploying daemon mds.c on smithi149 2023-07-21T22:18:47.125 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:47.125 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6834/1642617037,v1:172.21.15.149:6835/1642617037] up:boot 2023-07-21T22:18:47.125 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: fsmap 6 up:standby 2023-07-21T22:18:47.125 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:46 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:18:47.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:47.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:47.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: Deploying daemon mds.c on smithi149 2023-07-21T22:18:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6834/1642617037,v1:172.21.15.149:6835/1642617037] up:boot 2023-07-21T22:18:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: fsmap 6 up:standby 2023-07-21T22:18:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:46 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:18:47.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.268+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4541069b0 con 0x7fe4540ffb00 2023-07-21T22:18:47.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.268+0000 7fe450ff9700 1 -- 172.21.15.66:0/2679201783 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5054 (secure 0 0 0) 0x7fe44409acd0 con 0x7fe4540ffb00 2023-07-21T22:18:47.313 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:47.314 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T22:18:47.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe4400a11a0 msgr2=0x7fe4400a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:47.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe4400a11a0 0x7fe4400a3650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009c60 tx=0x7fe43c009260 comp rx=0 tx=0).stop 2023-07-21T22:18:47.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 msgr2=0x7fe4540742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:47.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540742c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe44400b2e0 tx=0x7fe444004730 comp rx=0 tx=0).stop 2023-07-21T22:18:47.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 shutdown_connections 2023-07-21T22:18:47.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe4400a11a0 0x7fe4400a3650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe454101680 0x7fe454072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4540ffb00 0x7fe4540742c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 --2- 172.21.15.66:0/2679201783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe454100cf0 0x7fe454072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:47.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 >> 172.21.15.66:0/2679201783 conn(0x7fe4540fb0e0 msgr2=0x7fe4540fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:47.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 shutdown_connections 2023-07-21T22:18:47.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:47.271+0000 7fe4595ff700 1 -- 172.21.15.66:0/2679201783 wait complete. 2023-07-21T22:18:47.320 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 7 2023-07-21T22:18:47.489 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:18:47 smithi149 systemd[1]: Starting Ceph mds.c for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:47.918 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:47 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2679201783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:47.918 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:18:47 smithi149 podman[155083]: 2023-07-21T22:18:48.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:47 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2679201783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:48.238 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:18:47 smithi149 ceph-mds[155123]: -- 172.21.15.149:0/2519436290 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd0fa951e0 con 0x55fd0fa6a800 2023-07-21T22:18:48.239 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:18:47 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c[155099]: starting mds.c at 2023-07-21T22:18:48.239 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:18:47 smithi149 bash[155083]: 8df0d8a1501746d18c2a7f770144b600648c000c584eed46312a0f8a42acaf1f 2023-07-21T22:18:48.239 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:18:48 smithi149 systemd[1]: Started Ceph mds.c for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:48.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:47 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2679201783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:49.336 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: Deploying daemon mds.h on smithi104 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6836/1037928792,v1:172.21.15.149:6837/1037928792] up:boot 2023-07-21T22:18:49.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: fsmap 7 up:standby 2023-07-21T22:18:49.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:49 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:18:49.405 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:49.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:49.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:49.406 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: Deploying daemon mds.h on smithi104 2023-07-21T22:18:49.407 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:49.407 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6836/1037928792,v1:172.21.15.149:6837/1037928792] up:boot 2023-07-21T22:18:49.407 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: fsmap 7 up:standby 2023-07-21T22:18:49.407 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:49 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:18:49.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:49.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:49.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:49.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:49.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: Deploying daemon mds.h on smithi104 2023-07-21T22:18:49.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:49.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6836/1037928792,v1:172.21.15.149:6837/1037928792] up:boot 2023-07-21T22:18:49.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: fsmap 7 up:standby 2023-07-21T22:18:49.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:49 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:18:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 -- 172.21.15.66:0/2103451155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118100cf0 msgr2=0x7f1118101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 --2- 172.21.15.66:0/2103451155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118100cf0 0x7f1118101140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1104009990 tx=0x7f1104009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 -- 172.21.15.66:0/2103451155 shutdown_connections 2023-07-21T22:18:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 --2- 172.21.15.66:0/2103451155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1118101680 0x7f1118107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 --2- 172.21.15.66:0/2103451155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f11180ffb00 0x7f11180fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 --2- 172.21.15.66:0/2103451155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1118100cf0 0x7f1118101140 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 -- 172.21.15.66:0/2103451155 >> 172.21.15.66:0/2103451155 conn(0x7f11180fb0e0 msgr2=0x7f11180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 -- 172.21.15.66:0/2103451155 shutdown_connections 2023-07-21T22:18:49.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.509+0000 7f111efff700 1 -- 172.21.15.66:0/2103451155 wait complete. 2023-07-21T22:18:49.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 Processor -- start 2023-07-21T22:18:49.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 -- start start 2023-07-21T22:18:49.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 0x7f1118105880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:49.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1118100cf0 0x7f1118105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1118101680 0x7f1118102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118102bb0 con 0x7f1118100cf0 2023-07-21T22:18:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118102d20 con 0x7f11180ffb00 2023-07-21T22:18:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111efff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118102ee0 con 0x7f1118101680 2023-07-21T22:18:49.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.511+0000 7f111cd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 0x7f1118105880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 0x7f1118105880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:52840/0 (socket says 172.21.15.66:52840) 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 -- 172.21.15.66:0/3067624160 learned_addr learned my addr 172.21.15.66:0/3067624160 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f1117fff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1118100cf0 0x7f1118105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111d59c700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1118101680 0x7f1118102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 -- 172.21.15.66:0/3067624160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1118101680 msgr2=0x7f1118102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1118101680 0x7f1118102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 -- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1118100cf0 msgr2=0x7f1118105dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1118100cf0 0x7f1118105dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 -- 172.21.15.66:0/3067624160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040095f0 con 0x7f11180ffb00 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111d59c700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1118101680 0x7f1118102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f111cd9b700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 0x7f1118105880 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f110c00e970 tx=0x7f110c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c009790 con 0x7f11180ffb00 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.512+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110c00eec0 con 0x7f11180ffb00 2023-07-21T22:18:49.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.513+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11181a3fe0 con 0x7f11180ffb00 2023-07-21T22:18:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.513+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c003cf0 con 0x7f11180ffb00 2023-07-21T22:18:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.513+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11181a44b0 con 0x7f11180ffb00 2023-07-21T22:18:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.514+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11181921e0 con 0x7f11180ffb00 2023-07-21T22:18:49.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.516+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f110c00ccd0 con 0x7f11180ffb00 2023-07-21T22:18:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.517+0000 7f1115ffb700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f11080a1220 0x7f11080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.517+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f110c014070 con 0x7f11180ffb00 2023-07-21T22:18:49.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.518+0000 7f1117fff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f11080a1220 0x7f11080a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:49.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.518+0000 7f1117fff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f11080a1220 0x7f11080a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f110400b2e0 tx=0x7f1104005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:49.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.519+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f110c0967e0 con 0x7f11180ffb00 2023-07-21T22:18:49.619 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:18:49 smithi104 systemd[1]: Starting Ceph mds.h for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:49.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.776+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f111802c850 con 0x7f11180ffb00 2023-07-21T22:18:49.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.777+0000 7f1115ffb700 1 -- 172.21.15.66:0/3067624160 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5814 (secure 0 0 0) 0x7f110c099e00 con 0x7f11180ffb00 2023-07-21T22:18:49.822 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:49.822 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-07-21T22:18:49.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.779+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f11080a1220 msgr2=0x7f11080a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.779+0000 7f111efff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f11080a1220 0x7f11080a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f110400b2e0 tx=0x7f1104005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.779+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 msgr2=0x7f1118105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.779+0000 7f111efff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 0x7f1118105880 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f110c00e970 tx=0x7f110c00ec80 comp rx=0 tx=0).stop 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 shutdown_connections 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f11080a1220 0x7f11080a36d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11180ffb00 0x7f1118105880 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1118101680 0x7f1118102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 --2- 172.21.15.66:0/3067624160 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1118100cf0 0x7f1118105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:49.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 >> 172.21.15.66:0/3067624160 conn(0x7f11180fb0e0 msgr2=0x7f11181065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:49.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 shutdown_connections 2023-07-21T22:18:49.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:49.780+0000 7f111efff700 1 -- 172.21.15.66:0/3067624160 wait complete. 2023-07-21T22:18:49.826 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 8 2023-07-21T22:18:50.151 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:50 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3067624160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:50.151 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:18:49 smithi104 podman[154015]: 2023-07-21T22:18:50.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:50 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3067624160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:50.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:50 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3067624160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:50.618 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:18:50 smithi104 ceph-mds[154057]: -- 172.21.15.104:0/1817478020 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f57bbff1e0 con 0x55f57bbd5000 2023-07-21T22:18:50.618 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:18:50 smithi104 bash[154015]: 4ac07dcbef39dc5a960f3a1223837ce8b5fb355d125531a855b44db4251c7537 2023-07-21T22:18:50.619 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:18:50 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h[154032]: starting mds.h at 2023-07-21T22:18:50.619 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:18:50 smithi104 systemd[1]: Started Ceph mds.h for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:51.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:51.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:51.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:51.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:51.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: Deploying daemon mds.i on smithi149 2023-07-21T22:18:51.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] up:boot 2023-07-21T22:18:51.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: fsmap 8 up:standby 2023-07-21T22:18:51.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:51 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:18:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:51.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: Deploying daemon mds.i on smithi149 2023-07-21T22:18:51.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] up:boot 2023-07-21T22:18:51.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: fsmap 8 up:standby 2023-07-21T22:18:51.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:51 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:18:51.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:51.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:51.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:51.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:51.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:51.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: Deploying daemon mds.i on smithi149 2023-07-21T22:18:51.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] up:boot 2023-07-21T22:18:51.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: fsmap 8 up:standby 2023-07-21T22:18:51.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:51 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:18:51.844 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:52.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3896568685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e80ffb00 msgr2=0x7f80e80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:52.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3896568685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e80ffb00 0x7f80e80fff10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009990 tx=0x7f80d8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3896568685 shutdown_connections 2023-07-21T22:18:52.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3896568685 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e8101680 0x7f80e8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3896568685 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e80ffb00 0x7f80e80fff10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3896568685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e8100cf0 0x7f80e8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3896568685 >> 172.21.15.66:0/3896568685 conn(0x7f80e80fb0e0 msgr2=0x7f80e80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:52.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3896568685 shutdown_connections 2023-07-21T22:18:52.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.017+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3896568685 wait complete. 2023-07-21T22:18:52.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.018+0000 7f80ed1ff700 1 Processor -- start 2023-07-21T22:18:52.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.018+0000 7f80ed1ff700 1 -- start start 2023-07-21T22:18:52.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80ed1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e80ffb00 0x7f80e8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80ed1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 0x7f80e8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80ed1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e8101680 0x7f80e819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80ed1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819f5d0 con 0x7f80e8100cf0 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80ed1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819f710 con 0x7f80e80ffb00 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80ed1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e819f8a0 con 0x7f80e8101680 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 0x7f80e8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:52.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 0x7f80e8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49644/0 (socket says 172.21.15.66:49644) 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 -- 172.21.15.66:0/3521141666 learned_addr learned my addr 172.21.15.66:0/3521141666 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80e6d9d700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e80ffb00 0x7f80e8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80e759e700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e8101680 0x7f80e819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 -- 172.21.15.66:0/3521141666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e8101680 msgr2=0x7f80e819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e8101680 0x7f80e819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 -- 172.21.15.66:0/3521141666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e80ffb00 msgr2=0x7f80e8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:52.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e80ffb00 0x7f80e8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.019+0000 7f80dffff700 1 -- 172.21.15.66:0/3521141666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80095f0 con 0x7f80e8100cf0 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.020+0000 7f80dffff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 0x7f80e8197f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f80d000d950 tx=0x7f80d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.020+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000f890 con 0x7f80e8100cf0 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.020+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80d0010460 con 0x7f80e8100cf0 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.020+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d000e610 con 0x7f80e8100cf0 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.020+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e819fb80 con 0x7f80e8100cf0 2023-07-21T22:18:52.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.020+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e81a0040 con 0x7f80e8100cf0 2023-07-21T22:18:52.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.021+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e81921e0 con 0x7f80e8100cf0 2023-07-21T22:18:52.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.023+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f80d00108d0 con 0x7f80e8100cf0 2023-07-21T22:18:52.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.024+0000 7f80e4d99700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80d40a1220 0x7f80d40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:52.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.025+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f80d00cc260 con 0x7f80e8100cf0 2023-07-21T22:18:52.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.025+0000 7f80e6d9d700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80d40a1220 0x7f80d40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:52.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.025+0000 7f80e6d9d700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80d40a1220 0x7f80d40a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b2e0 tx=0x7f80d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:52.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.029+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f80d0095db0 con 0x7f80e8100cf0 2023-07-21T22:18:52.226 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:18:51 smithi149 systemd[1]: Starting Ceph mds.i for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:52.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.290+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80e802c850 con 0x7f80e8100cf0 2023-07-21T22:18:52.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.291+0000 7f80e4d99700 1 -- 172.21.15.66:0/3521141666 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6574 (secure 0 0 0) 0x7f80d00993d0 con 0x7f80e8100cf0 2023-07-21T22:18:52.336 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:52.336 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T22:18:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80d40a1220 msgr2=0x7f80d40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80d40a1220 0x7f80d40a36d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f80d800b2e0 tx=0x7f80d8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 msgr2=0x7f80e8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 0x7f80e8197f90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f80d000d950 tx=0x7f80d000dc60 comp rx=0 tx=0).stop 2023-07-21T22:18:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 shutdown_connections 2023-07-21T22:18:52.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80d40a1220 0x7f80d40a36d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80e80ffb00 0x7f80e8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80e8101680 0x7f80e819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 --2- 172.21.15.66:0/3521141666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80e8100cf0 0x7f80e8197f90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 >> 172.21.15.66:0/3521141666 conn(0x7f80e80fb0e0 msgr2=0x7f80e81065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 shutdown_connections 2023-07-21T22:18:52.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:52.293+0000 7f80ed1ff700 1 -- 172.21.15.66:0/3521141666 wait complete. 2023-07-21T22:18:52.340 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 9 2023-07-21T22:18:52.489 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:18:52 smithi149 podman[155423]: 2023-07-21T22:18:52.989 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:18:52 smithi149 ceph-mds[155463]: -- 172.21.15.149:0/2270425742 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5584233e91e0 con 0x5584233be800 2023-07-21T22:18:52.989 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:18:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: starting mds.i at 2023-07-21T22:18:52.990 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:18:52 smithi149 bash[155423]: 0107abbd26669db146cebbca4c8ff8285a6cf04522c766482dc93b4dbe91fb6a 2023-07-21T22:18:52.990 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:18:52 smithi149 systemd[1]: Started Ceph mds.i for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3521141666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:53.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:53.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:53 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:53.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:53.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3521141666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:53.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.542 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.543 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:53.543 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:53.543 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:53 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:53.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3521141666' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:53 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:54.358 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:54.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:54 smithi066 ceph-mon[126379]: Deploying daemon mds.g on smithi066 2023-07-21T22:18:54.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:54 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:boot 2023-07-21T22:18:54.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:54 smithi066 ceph-mon[126379]: fsmap 9 up:standby 2023-07-21T22:18:54.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:54 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:18:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:54 smithi149 ceph-mon[129268]: Deploying daemon mds.g on smithi066 2023-07-21T22:18:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:54 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:boot 2023-07-21T22:18:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:54 smithi149 ceph-mon[129268]: fsmap 9 up:standby 2023-07-21T22:18:54.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:54 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:18:54.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:54 smithi104 ceph-mon[129021]: Deploying daemon mds.g on smithi066 2023-07-21T22:18:54.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:54 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:boot 2023-07-21T22:18:54.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:54 smithi104 ceph-mon[129021]: fsmap 9 up:standby 2023-07-21T22:18:54.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:54 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:18:54.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.589+0000 7f6e177ff700 1 -- 172.21.15.66:0/283129916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10100cf0 msgr2=0x7f6e10101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:54.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.589+0000 7f6e177ff700 1 --2- 172.21.15.66:0/283129916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10100cf0 0x7f6e10101140 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009990 tx=0x7f6e00009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 -- 172.21.15.66:0/283129916 shutdown_connections 2023-07-21T22:18:54.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 --2- 172.21.15.66:0/283129916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e10101680 0x7f6e10107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 --2- 172.21.15.66:0/283129916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e100fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 --2- 172.21.15.66:0/283129916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10100cf0 0x7f6e10101140 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 -- 172.21.15.66:0/283129916 >> 172.21.15.66:0/283129916 conn(0x7f6e100fb0e0 msgr2=0x7f6e100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 -- 172.21.15.66:0/283129916 shutdown_connections 2023-07-21T22:18:54.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.590+0000 7f6e177ff700 1 -- 172.21.15.66:0/283129916 wait complete. 2023-07-21T22:18:54.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 Processor -- start 2023-07-21T22:18:54.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 -- start start 2023-07-21T22:18:54.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e10197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:54.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e10100cf0 0x7f6e10197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:54.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10101680 0x7f6e1019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:54.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1019f5b0 con 0x7f6e100ffb00 2023-07-21T22:18:54.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1019f6f0 con 0x7f6e10101680 2023-07-21T22:18:54.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.591+0000 7f6e177ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1019f880 con 0x7f6e10100cf0 2023-07-21T22:18:54.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e10197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:54.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e10197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49662/0 (socket says 172.21.15.66:49662) 2023-07-21T22:18:54.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 -- 172.21.15.66:0/1340472361 learned_addr learned my addr 172.21.15.66:0/1340472361 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:54.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e15d9c700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10101680 0x7f6e1019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:54.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e14d9a700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e10100cf0 0x7f6e10197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:54.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 -- 172.21.15.66:0/1340472361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e10100cf0 msgr2=0x7f6e10197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:54.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e10100cf0 0x7f6e10197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 -- 172.21.15.66:0/1340472361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10101680 msgr2=0x7f6e1019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10101680 0x7f6e1019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 -- 172.21.15.66:0/1340472361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e000095f0 con 0x7f6e100ffb00 2023-07-21T22:18:54.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.592+0000 7f6e1559b700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e10197a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00e970 tx=0x7f6e0c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.593+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c009790 con 0x7f6e100ffb00 2023-07-21T22:18:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.593+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1019fb60 con 0x7f6e100ffb00 2023-07-21T22:18:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.593+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0c00c7e0 con 0x7f6e100ffb00 2023-07-21T22:18:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.593+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0c003c70 con 0x7f6e100ffb00 2023-07-21T22:18:54.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.593+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e101a00c0 con 0x7f6e100ffb00 2023-07-21T22:18:54.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.594+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1010b320 con 0x7f6e100ffb00 2023-07-21T22:18:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.596+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6e0c00cc50 con 0x7f6e100ffb00 2023-07-21T22:18:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.597+0000 7f6e067fc700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6dfc0a11a0 0x7f6dfc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.597+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6e0c014070 con 0x7f6e100ffb00 2023-07-21T22:18:54.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.598+0000 7f6e14d9a700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6dfc0a11a0 0x7f6dfc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:54.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.598+0000 7f6e14d9a700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6dfc0a11a0 0x7f6dfc0a3650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000b2e0 tx=0x7f6e00005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:54.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.600+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6e0c096350 con 0x7f6e100ffb00 2023-07-21T22:18:54.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.859+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e1010b4e0 con 0x7f6e100ffb00 2023-07-21T22:18:54.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.859+0000 7f6e067fc700 1 -- 172.21.15.66:0/1340472361 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7336 (secure 0 0 0) 0x7f6e0c003dd0 con 0x7f6e100ffb00 2023-07-21T22:18:54.904 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:54.904 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T22:18:54.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6dfc0a11a0 msgr2=0x7f6dfc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:54.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6dfc0a11a0 0x7f6dfc0a3650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000b2e0 tx=0x7f6e00005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 msgr2=0x7f6e10197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:54.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e10197a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00e970 tx=0x7f6e0c00ec80 comp rx=0 tx=0).stop 2023-07-21T22:18:54.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 shutdown_connections 2023-07-21T22:18:54.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6dfc0a11a0 0x7f6dfc0a3650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e10101680 0x7f6e1019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6e100ffb00 0x7f6e10197a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 --2- 172.21.15.66:0/1340472361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e10100cf0 0x7f6e10197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 >> 172.21.15.66:0/1340472361 conn(0x7f6e100fb0e0 msgr2=0x7f6e10106580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 shutdown_connections 2023-07-21T22:18:54.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:54.862+0000 7f6e177ff700 1 -- 172.21.15.66:0/1340472361 wait complete. 2023-07-21T22:18:54.909 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 10 2023-07-21T22:18:55.195 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:18:54 smithi066 systemd[1]: Starting Ceph mds.g for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:55.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:55 smithi066 ceph-mon[126379]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:55.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:55 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1340472361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:55.485 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:18:55 smithi066 podman[160100]: 2023-07-21T22:18:55.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:55 smithi149 ceph-mon[129268]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:55.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:55 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1340472361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:55 smithi104 ceph-mon[129021]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:55 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1340472361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:55.945 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:18:55 smithi066 bash[160100]: fb49a61277a929be5bb15a21e67afecb5ef1c8f17d52f2c2bf9eb195a90180fb 2023-07-21T22:18:55.945 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:18:55 smithi066 systemd[1]: Started Ceph mds.g for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:55.946 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:18:55 smithi066 ceph-mds[160140]: -- 172.21.15.66:0/2501601669 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56110b4371e0 con 0x56110b40cc00 2023-07-21T22:18:55.946 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:18:55 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-g[160116]: starting mds.g at 2023-07-21T22:18:56.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:56.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:56.893 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: Deploying daemon mds.a on smithi066 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.66:6838/1234257491,v1:172.21.15.66:6839/1234257491] up:boot 2023-07-21T22:18:56.894 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: fsmap 10 up:standby 2023-07-21T22:18:56.895 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:56 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:18:56.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:56.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:56.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:56.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:56.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:56.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:56.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:56.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: Deploying daemon mds.a on smithi066 2023-07-21T22:18:56.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:56.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.66:6838/1234257491,v1:172.21.15.66:6839/1234257491] up:boot 2023-07-21T22:18:56.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: fsmap 10 up:standby 2023-07-21T22:18:56.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:56 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:18:57.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: Deploying daemon mds.a on smithi066 2023-07-21T22:18:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:57.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.66:6838/1234257491,v1:172.21.15.66:6839/1234257491] up:boot 2023-07-21T22:18:57.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: fsmap 10 up:standby 2023-07-21T22:18:57.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:56 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:18:57.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 -- 172.21.15.66:0/514399670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac101680 msgr2=0x7f7fac107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:57.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/514399670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac101680 0x7f7fac107f40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400b0a0 tx=0x7f7fa400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 -- 172.21.15.66:0/514399670 shutdown_connections 2023-07-21T22:18:57.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/514399670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fac0ffb00 0x7f7fac0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/514399670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac101680 0x7f7fac107f40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/514399670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fac100cf0 0x7f7fac101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 -- 172.21.15.66:0/514399670 >> 172.21.15.66:0/514399670 conn(0x7f7fac0fb0e0 msgr2=0x7f7fac0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:57.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 -- 172.21.15.66:0/514399670 shutdown_connections 2023-07-21T22:18:57.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.095+0000 7f7fb0fff700 1 -- 172.21.15.66:0/514399670 wait complete. 2023-07-21T22:18:57.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.096+0000 7f7fb0fff700 1 Processor -- start 2023-07-21T22:18:57.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 -- start start 2023-07-21T22:18:57.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fac0ffb00 0x7f7fac0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:57.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 0x7f7fac072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fac101680 0x7f7fac072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0735e0 con 0x7f7fac100cf0 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac073750 con 0x7f7fac101680 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fb0fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac19f930 con 0x7f7fac0ffb00 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fa9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 0x7f7fac072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fa9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 0x7f7fac072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49702/0 (socket says 172.21.15.66:49702) 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7fa9d9b700 1 -- 172.21.15.66:0/3504185327 learned_addr learned my addr 172.21.15.66:0/3504185327 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:57.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7faa59c700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fac0ffb00 0x7f7fac0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.097+0000 7f7faad9d700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fac101680 0x7f7fac072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fa9d9b700 1 -- 172.21.15.66:0/3504185327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fac0ffb00 msgr2=0x7f7fac0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fa9d9b700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fac0ffb00 0x7f7fac0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fa9d9b700 1 -- 172.21.15.66:0/3504185327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fac101680 msgr2=0x7f7fac072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fa9d9b700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fac101680 0x7f7fac072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fa9d9b700 1 -- 172.21.15.66:0/3504185327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c009520 con 0x7f7fac100cf0 2023-07-21T22:18:57.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fa9d9b700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 0x7f7fac072900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e9a0 tx=0x7f7fa000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0009790 con 0x7f7fac100cf0 2023-07-21T22:18:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fa000cbd0 con 0x7f7fac100cf0 2023-07-21T22:18:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa4009d00 con 0x7f7fac100cf0 2023-07-21T22:18:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0018680 con 0x7f7fac100cf0 2023-07-21T22:18:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.098+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac19ffb0 con 0x7f7fac100cf0 2023-07-21T22:18:57.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.099+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac106fa0 con 0x7f7fac100cf0 2023-07-21T22:18:57.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.102+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7fa000cd40 con 0x7f7fac100cf0 2023-07-21T22:18:57.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.103+0000 7f7f9b7fe700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7f940a1220 0x7f7f940a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.103+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7fa0014070 con 0x7f7fac100cf0 2023-07-21T22:18:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.103+0000 7f7faa59c700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7f940a1220 0x7f7f940a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:57.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.104+0000 7f7faa59c700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7f940a1220 0x7f7f940a36d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fac100b50 tx=0x7f7f9c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:57.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.107+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7fa0051770 con 0x7f7fac100cf0 2023-07-21T22:18:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.388+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fac0611c0 con 0x7f7fac100cf0 2023-07-21T22:18:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.389+0000 7f7f9b7fe700 1 -- 172.21.15.66:0/3504185327 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8094 (secure 0 0 0) 0x7f7fa0096230 con 0x7f7fac100cf0 2023-07-21T22:18:57.434 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:57.435 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T22:18:57.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7f940a1220 msgr2=0x7f7f940a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7f940a1220 0x7f7f940a36d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7fac100b50 tx=0x7f7f9c009260 comp rx=0 tx=0).stop 2023-07-21T22:18:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 msgr2=0x7f7fac072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 0x7f7fac072900 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000e9a0 tx=0x7f7fa000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 shutdown_connections 2023-07-21T22:18:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7f940a1220 0x7f7f940a36d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fac101680 0x7f7fac072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7fac0ffb00 0x7f7fac0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 --2- 172.21.15.66:0/3504185327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7fac100cf0 0x7f7fac072900 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:57.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 >> 172.21.15.66:0/3504185327 conn(0x7f7fac0fb0e0 msgr2=0x7f7fac1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:57.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 shutdown_connections 2023-07-21T22:18:57.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:57.392+0000 7f7fb0fff700 1 -- 172.21.15.66:0/3504185327 wait complete. 2023-07-21T22:18:57.438 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 11 2023-07-21T22:18:57.988 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:57 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3504185327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:58.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:57 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3504185327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:58.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:57 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3504185327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:18:58.196 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:18:57 smithi066 systemd[1]: Starting Ceph mds.a for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:18:58.677 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:18:58 smithi066 podman[160493]: 2023-07-21T22:18:58.677 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:18:58 smithi066 bash[160493]: 037fdd3da291f0f2f40fe2a1b97627e9c984d2e9f94915cc573d7b46c2be4b02 2023-07-21T22:18:58.677 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:18:58 smithi066 ceph-mds[160533]: -- 172.21.15.66:0/3913614047 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5620ecb791e0 con 0x5620ecb4ec00 2023-07-21T22:18:58.677 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:18:58 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-a[160509]: starting mds.a at 2023-07-21T22:18:58.945 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:18:58 smithi066 systemd[1]: Started Ceph mds.a for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:18:58.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:58 smithi066 ceph-mon[126379]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:58.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:58.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:58 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:58 smithi104 ceph-mon[129021]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:58 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:59.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:58 smithi149 ceph-mon[129268]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:18:59.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:59.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:58 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:59.457 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:18:59.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3445761810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8074550 msgr2=0x7fa3d80729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:59.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3445761810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8074550 0x7fa3d80729a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009990 tx=0x7fa3c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3445761810 shutdown_connections 2023-07-21T22:18:59.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3445761810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d81079c0 0x7fa3d8109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3445761810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d8074550 0x7fa3d80729a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3445761810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3d8072f70 0x7fa3d80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3445761810 >> 172.21.15.66:0/3445761810 conn(0x7fa3d80fb0e0 msgr2=0x7fa3d80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:59.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3445761810 shutdown_connections 2023-07-21T22:18:59.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.639+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3445761810 wait complete. 2023-07-21T22:18:59.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 Processor -- start 2023-07-21T22:18:59.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 -- start start 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3d8072f70 0x7fa3d81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 0x7fa3d8197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81079c0 0x7fa3d819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f560 con 0x7fa3d81079c0 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f6a0 con 0x7fa3d8074550 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.640+0000 7fa3dd9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f830 con 0x7fa3d8072f70 2023-07-21T22:18:59.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 0x7fa3d8197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81079c0 0x7fa3d819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 0x7fa3d8197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50174/0 (socket says 172.21.15.66:50174) 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d67fc700 1 -- 172.21.15.66:0/3355884973 learned_addr learned my addr 172.21.15.66:0/3355884973 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d6ffd700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3d8072f70 0x7fa3d81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 -- 172.21.15.66:0/3355884973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3d8072f70 msgr2=0x7fa3d81979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3d8072f70 0x7fa3d81979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 -- 172.21.15.66:0/3355884973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 msgr2=0x7fa3d8197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:59.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 0x7fa3d8197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 -- 172.21.15.66:0/3355884973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80095f0 con 0x7fa3d81079c0 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d67fc700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 0x7fa3d8197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.641+0000 7fa3d77fe700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81079c0 0x7fa3d819ef90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000cc50 tx=0x7fa3d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.642+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000aa90 con 0x7fa3d81079c0 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.642+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d819fb10 con 0x7fa3d81079c0 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.642+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3d000abf0 con 0x7fa3d81079c0 2023-07-21T22:18:59.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.642+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00115e0 con 0x7fa3d81079c0 2023-07-21T22:18:59.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.642+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d81a0070 con 0x7fa3d81079c0 2023-07-21T22:18:59.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.643+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d8106070 con 0x7fa3d81079c0 2023-07-21T22:18:59.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.645+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa3d0012420 con 0x7fa3d81079c0 2023-07-21T22:18:59.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.646+0000 7fa3dc9fd700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa3c00a11a0 0x7fa3c00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:18:59.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.646+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3d00cc600 con 0x7fa3d81079c0 2023-07-21T22:18:59.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.647+0000 7fa3d6ffd700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa3c00a11a0 0x7fa3c00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:18:59.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.647+0000 7fa3d6ffd700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa3c00a11a0 0x7fa3c00a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800b2e0 tx=0x7fa3c8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:18:59.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.651+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa3d0096150 con 0x7fa3d81079c0 2023-07-21T22:18:59.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.66:6840/3476312972,v1:172.21.15.66:6841/3476312972] up:boot 2023-07-21T22:18:59.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: fsmap 11 up:standby 2023-07-21T22:18:59.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:18:59.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:18:59.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:18:59.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:18:59.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:18:59.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:18:59 smithi066 ceph-mon[126379]: Deploying daemon mds.b on smithi104 2023-07-21T22:18:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.910+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3d80611c0 con 0x7fa3d81079c0 2023-07-21T22:18:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.910+0000 7fa3dc9fd700 1 -- 172.21.15.66:0/3355884973 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8852 (secure 0 0 0) 0x7fa3d0099770 con 0x7fa3d81079c0 2023-07-21T22:18:59.955 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:18:59.955 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T22:18:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa3c00a11a0 msgr2=0x7fa3c00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa3c00a11a0 0x7fa3c00a3650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800b2e0 tx=0x7fa3c8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81079c0 msgr2=0x7fa3d819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:18:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81079c0 0x7fa3d819ef90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa3d000cc50 tx=0x7fa3d000cf60 comp rx=0 tx=0).stop 2023-07-21T22:18:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 shutdown_connections 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa3c00a11a0 0x7fa3c00a3650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa3d81079c0 0x7fa3d819ef90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3d8072f70 0x7fa3d81979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 --2- 172.21.15.66:0/3355884973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3d8074550 0x7fa3d8197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 >> 172.21.15.66:0/3355884973 conn(0x7fa3d80fb0e0 msgr2=0x7fa3d80fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 shutdown_connections 2023-07-21T22:18:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:18:59.913+0000 7fa3dd9ff700 1 -- 172.21.15.66:0/3355884973 wait complete. 2023-07-21T22:18:59.960 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 12 2023-07-21T22:19:00.033 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.66:6840/3476312972,v1:172.21.15.66:6841/3476312972] up:boot 2023-07-21T22:19:00.034 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: fsmap 11 up:standby 2023-07-21T22:19:00.034 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:19:00.034 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:00.034 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:19:00.034 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:19:00.034 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:00.035 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:18:59 smithi104 ceph-mon[129021]: Deploying daemon mds.b on smithi104 2023-07-21T22:19:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.66:6840/3476312972,v1:172.21.15.66:6841/3476312972] up:boot 2023-07-21T22:19:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: fsmap 11 up:standby 2023-07-21T22:19:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:19:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T22:19:00.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T22:19:00.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:00.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:18:59 smithi149 ceph-mon[129268]: Deploying daemon mds.b on smithi104 2023-07-21T22:19:00.368 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:00 smithi104 systemd[1]: Starting Ceph mds.b for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:19:00.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3355884973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:00.825 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:00 smithi104 ceph-mon[129021]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:00.825 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:00 smithi104 podman[154357]: 2023-07-21T22:19:00.826 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:00 smithi104 ceph-mds[154397]: -- 172.21.15.104:0/3564873114 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555ad966f1e0 con 0x555ad9644800 2023-07-21T22:19:00.826 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: starting mds.b at 2023-07-21T22:19:01.118 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:00 smithi104 bash[154357]: ebcdcf4440184076831bf5069ad864703fe73580eb17f8dc46801516b456a4df 2023-07-21T22:19:01.118 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:00 smithi104 systemd[1]: Started Ceph mds.b for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:19:01.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3355884973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:01.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:00 smithi066 ceph-mon[126379]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:01.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3355884973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:01.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:00 smithi149 ceph-mon[129268]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:01.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:19:02.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:02.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] up:boot 2023-07-21T22:19:02.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: fsmap 12 up:standby 2023-07-21T22:19:02.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:01 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:19:02.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 -- 172.21.15.66:0/3188009849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814072f10 msgr2=0x7fd814073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 --2- 172.21.15.66:0/3188009849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814072f10 0x7fd814073380 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc009990 tx=0x7fd7fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 -- 172.21.15.66:0/3188009849 shutdown_connections 2023-07-21T22:19:02.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 --2- 172.21.15.66:0/3188009849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814107970 0x7fd814109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 --2- 172.21.15.66:0/3188009849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8140744f0 0x7fd814072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 --2- 172.21.15.66:0/3188009849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814072f10 0x7fd814073380 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 -- 172.21.15.66:0/3188009849 >> 172.21.15.66:0/3188009849 conn(0x7fd8140fb100 msgr2=0x7fd8140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:02.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 -- 172.21.15.66:0/3188009849 shutdown_connections 2023-07-21T22:19:02.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.160+0000 7fd8195ff700 1 -- 172.21.15.66:0/3188009849 wait complete. 2023-07-21T22:19:02.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.161+0000 7fd8195ff700 1 Processor -- start 2023-07-21T22:19:02.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 -- start start 2023-07-21T22:19:02.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 0x7fd8141979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8140744f0 0x7fd814197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814107970 0x7fd81419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419f570 con 0x7fd8140744f0 2023-07-21T22:19:02.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419f6b0 con 0x7fd814072f10 2023-07-21T22:19:02.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd8195ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81419f840 con 0x7fd814107970 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd812ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 0x7fd8141979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd812ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 0x7fd8141979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:46340/0 (socket says 172.21.15.66:46340) 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.162+0000 7fd812ffd700 1 -- 172.21.15.66:0/197918405 learned_addr learned my addr 172.21.15.66:0/197918405 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd80bfff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8140744f0 0x7fd814197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd8137fe700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814107970 0x7fd81419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd812ffd700 1 -- 172.21.15.66:0/197918405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814107970 msgr2=0x7fd81419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd812ffd700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814107970 0x7fd81419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd812ffd700 1 -- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8140744f0 msgr2=0x7fd814197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd812ffd700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8140744f0 0x7fd814197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd812ffd700 1 -- 172.21.15.66:0/197918405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc0095f0 con 0x7fd814072f10 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd812ffd700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 0x7fd8141979f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9a0 tx=0x7fd80400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804009790 con 0x7fd814072f10 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd80400cbd0 con 0x7fd814072f10 2023-07-21T22:19:02.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd804018680 con 0x7fd814072f10 2023-07-21T22:19:02.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81419fb20 con 0x7fd814072f10 2023-07-21T22:19:02.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.163+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81419ffe0 con 0x7fd814072f10 2023-07-21T22:19:02.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.164+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8141060b0 con 0x7fd814072f10 2023-07-21T22:19:02.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.167+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd80400cd40 con 0x7fd814072f10 2023-07-21T22:19:02.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.168+0000 7fd810ff9700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8000a11a0 0x7fd8000a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.169+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd804014070 con 0x7fd814072f10 2023-07-21T22:19:02.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.169+0000 7fd80bfff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8000a11a0 0x7fd8000a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.169+0000 7fd80bfff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8000a11a0 0x7fd8000a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b2e0 tx=0x7fd7fc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:02.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.170+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd804096850 con 0x7fd814072f10 2023-07-21T22:19:02.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] up:boot 2023-07-21T22:19:02.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: fsmap 12 up:standby 2023-07-21T22:19:02.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:01 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:19:02.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] up:boot 2023-07-21T22:19:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: fsmap 12 up:standby 2023-07-21T22:19:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:01 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:19:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.438+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd81402caa0 con 0x7fd814072f10 2023-07-21T22:19:02.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.439+0000 7fd810ff9700 1 -- 172.21.15.66:0/197918405 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9613 (secure 0 0 0) 0x7fd804099e70 con 0x7fd814072f10 2023-07-21T22:19:02.484 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:02.484 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24572,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-21T22:19:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.441+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8000a11a0 msgr2=0x7fd8000a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.441+0000 7fd8195ff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8000a11a0 0x7fd8000a3650 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc00b2e0 tx=0x7fd7fc005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.441+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 msgr2=0x7fd8141979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.441+0000 7fd8195ff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 0x7fd8141979f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd80400e9a0 tx=0x7fd80400ed60 comp rx=0 tx=0).stop 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 shutdown_connections 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8000a11a0 0x7fd8000a3650 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd814072f10 0x7fd8141979f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd814107970 0x7fd81419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 --2- 172.21.15.66:0/197918405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8140744f0 0x7fd814197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 >> 172.21.15.66:0/197918405 conn(0x7fd8140fb100 msgr2=0x7fd814109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 shutdown_connections 2023-07-21T22:19:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.442+0000 7fd8195ff700 1 -- 172.21.15.66:0/197918405 wait complete. 2023-07-21T22:19:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 13 2023-07-21T22:19:02.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:02.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 -- 172.21.15.66:0/1771413894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff014074550 msgr2=0x7ff0140729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 --2- 172.21.15.66:0/1771413894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff014074550 0x7ff0140729a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff010009990 tx=0x7ff010009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 -- 172.21.15.66:0/1771413894 shutdown_connections 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 --2- 172.21.15.66:0/1771413894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0141079c0 0x7ff014109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 --2- 172.21.15.66:0/1771413894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff014074550 0x7ff0140729a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 --2- 172.21.15.66:0/1771413894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff0140733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 -- 172.21.15.66:0/1771413894 >> 172.21.15.66:0/1771413894 conn(0x7ff0140fb0e0 msgr2=0x7ff0140fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 -- 172.21.15.66:0/1771413894 shutdown_connections 2023-07-21T22:19:02.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.678+0000 7ff01bdff700 1 -- 172.21.15.66:0/1771413894 wait complete. 2023-07-21T22:19:02.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.679+0000 7ff01bdff700 1 Processor -- start 2023-07-21T22:19:02.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 -- start start 2023-07-21T22:19:02.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff014197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff014074550 0x7ff014197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0141079c0 0x7ff01419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419f5b0 con 0x7ff0141079c0 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419f6f0 con 0x7ff014074550 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01bdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01419f880 con 0x7ff014072f70 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff019b9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff014197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01939a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff014074550 0x7ff014197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff019b9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff014197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:59040/0 (socket says 172.21.15.66:59040) 2023-07-21T22:19:02.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff019b9b700 1 -- 172.21.15.66:0/3924292693 learned_addr learned my addr 172.21.15.66:0/3924292693 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.680+0000 7ff01a39c700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0141079c0 0x7ff01419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff019b9b700 1 -- 172.21.15.66:0/3924292693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff014074550 msgr2=0x7ff014197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff019b9b700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff014074550 0x7ff014197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff019b9b700 1 -- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0141079c0 msgr2=0x7ff01419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff019b9b700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0141079c0 0x7ff01419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff019b9b700 1 -- 172.21.15.66:0/3924292693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c009c40 con 0x7ff014072f70 2023-07-21T22:19:02.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff01939a700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff014074550 0x7ff014197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff019b9b700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff014197a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff010004710 tx=0x7ff010009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01001d070 con 0x7ff014072f70 2023-07-21T22:19:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff010021410 con 0x7ff014072f70 2023-07-21T22:19:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff010017650 con 0x7ff014072f70 2023-07-21T22:19:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0100095f0 con 0x7ff014072f70 2023-07-21T22:19:02.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.681+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01419fd90 con 0x7ff014072f70 2023-07-21T22:19:02.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.683+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff014106070 con 0x7ff014072f70 2023-07-21T22:19:02.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.685+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff010021580 con 0x7ff014072f70 2023-07-21T22:19:02.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.686+0000 7ff00affd700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff0000a1220 0x7ff0000a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:02.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.686+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0100cda10 con 0x7ff014072f70 2023-07-21T22:19:02.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.686+0000 7ff01939a700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff0000a1220 0x7ff0000a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:02.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.687+0000 7ff01939a700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff0000a1220 0x7ff0000a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff014074270 tx=0x7ff004009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:02.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.689+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff010097710 con 0x7ff014072f70 2023-07-21T22:19:02.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.921+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff01402caa0 con 0x7ff014072f70 2023-07-21T22:19:02.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.922+0000 7ff00affd700 1 -- 172.21.15.66:0/3924292693 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18432 (secure 0 0 0) 0x7ff01009ad30 con 0x7ff014072f70 2023-07-21T22:19:02.967 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:02.967 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":81,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:18:19.827969+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff0000a1220 msgr2=0x7ff0000a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff0000a1220 0x7ff0000a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff014074270 tx=0x7ff004009260 comp rx=0 tx=0).stop 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 msgr2=0x7ff014197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff014197a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff010004710 tx=0x7ff010009580 comp rx=0 tx=0).stop 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 shutdown_connections 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff0000a1220 0x7ff0000a36d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0141079c0 0x7ff01419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.925+0000 7ff01bdff700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff014072f70 0x7ff014197a30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.926+0000 7ff01bdff700 1 --2- 172.21.15.66:0/3924292693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff014074550 0x7ff014197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.926+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 >> 172.21.15.66:0/3924292693 conn(0x7ff0140fb0e0 msgr2=0x7ff0140fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.926+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 shutdown_connections 2023-07-21T22:19:02.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:02.926+0000 7ff01bdff700 1 -- 172.21.15.66:0/3924292693 wait complete. 2023-07-21T22:19:02.991 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T22:19:02.992 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:19:03.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:02 smithi066 ceph-mon[126379]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:03.206 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/197918405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:03.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 -- 172.21.15.66:0/4124612389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c105860 msgr2=0x7f227c107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 --2- 172.21.15.66:0/4124612389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c105860 0x7f227c107c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f227400b0a0 tx=0x7f227400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 -- 172.21.15.66:0/4124612389 shutdown_connections 2023-07-21T22:19:03.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 --2- 172.21.15.66:0/4124612389 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c105860 0x7f227c107c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 --2- 172.21.15.66:0/4124612389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c100620 0x7f227c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 --2- 172.21.15.66:0/4124612389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f227c102f40 0x7f227c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.170+0000 7f2281fff700 1 -- 172.21.15.66:0/4124612389 >> 172.21.15.66:0/4124612389 conn(0x7f227c0fa000 msgr2=0x7f227c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:03.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.171+0000 7f2281fff700 1 -- 172.21.15.66:0/4124612389 shutdown_connections 2023-07-21T22:19:03.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.171+0000 7f2281fff700 1 -- 172.21.15.66:0/4124612389 wait complete. 2023-07-21T22:19:03.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 Processor -- start 2023-07-21T22:19:03.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 -- start start 2023-07-21T22:19:03.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c100620 0x7f227c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f227c102f40 0x7f227c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c105860 0x7f227c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19f650 con 0x7f227c100620 2023-07-21T22:19:03.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19f790 con 0x7f227c102f40 2023-07-21T22:19:03.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.172+0000 7f2281fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c19f920 con 0x7f227c105860 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c100620 0x7f227c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c100620 0x7f227c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43790/0 (socket says 172.21.15.66:43790) 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c105860 0x7f227c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227b7fe700 1 -- 172.21.15.66:0/331070693 learned_addr learned my addr 172.21.15.66:0/331070693 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227affd700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f227c102f40 0x7f227c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227bfff700 1 -- 172.21.15.66:0/331070693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f227c102f40 msgr2=0x7f227c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227bfff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f227c102f40 0x7f227c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227bfff700 1 -- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c100620 msgr2=0x7f227c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227bfff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c100620 0x7f227c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.173+0000 7f227bfff700 1 -- 172.21.15.66:0/331070693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2270009520 con 0x7f227c105860 2023-07-21T22:19:03.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.174+0000 7f227bfff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c105860 0x7f227c19f080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2274009fd0 tx=0x7f2274007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:03.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.174+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227400e040 con 0x7f227c105860 2023-07-21T22:19:03.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.174+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2274004c50 con 0x7f227c105860 2023-07-21T22:19:03.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.174+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2274009d00 con 0x7f227c105860 2023-07-21T22:19:03.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.174+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227401b6c0 con 0x7f227c105860 2023-07-21T22:19:03.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.174+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c19fe30 con 0x7f227c105860 2023-07-21T22:19:03.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.175+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c1073d0 con 0x7f227c105860 2023-07-21T22:19:03.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.177+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f227401d470 con 0x7f227c105860 2023-07-21T22:19:03.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.178+0000 7f2278ff9700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22640a11d0 0x7f22640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.179+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f22740cd6d0 con 0x7f227c105860 2023-07-21T22:19:03.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.179+0000 7f227b7fe700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22640a11d0 0x7f22640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.179+0000 7f227b7fe700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22640a11d0 0x7f22640a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f226c009d20 tx=0x7f226c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:03.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.181+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f22740973d0 con 0x7f227c105860 2023-07-21T22:19:03.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:02 smithi149 ceph-mon[129268]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:03.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:02 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/197918405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:03.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:02 smithi104 ceph-mon[129021]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:03.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:02 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/197918405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:03.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.408+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f227c10b350 con 0x7f227c105860 2023-07-21T22:19:03.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.409+0000 7f2278ff9700 1 -- 172.21.15.66:0/331070693 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f227409a9f0 con 0x7f227c105860 2023-07-21T22:19:03.454 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:19:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22640a11d0 msgr2=0x7f22640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22640a11d0 0x7f22640a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f226c009d20 tx=0x7f226c009260 comp rx=0 tx=0).stop 2023-07-21T22:19:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c105860 msgr2=0x7f227c19f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c105860 0x7f227c19f080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2274009fd0 tx=0x7f2274007930 comp rx=0 tx=0).stop 2023-07-21T22:19:03.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 shutdown_connections 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22640a11d0 0x7f22640a3680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f227c105860 0x7f227c19f080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f227c100620 0x7f227c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 --2- 172.21.15.66:0/331070693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f227c102f40 0x7f227c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 >> 172.21.15.66:0/331070693 conn(0x7f227c0fa000 msgr2=0x7f227c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 shutdown_connections 2023-07-21T22:19:03.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.412+0000 7f2281fff700 1 -- 172.21.15.66:0/331070693 wait complete. 2023-07-21T22:19:03.478 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-21T22:19:03.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 -- 172.21.15.66:0/3663294992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf00ffb00 msgr2=0x7fdcf00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/3663294992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00fff10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009990 tx=0x7fdce0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 -- 172.21.15.66:0/3663294992 shutdown_connections 2023-07-21T22:19:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/3663294992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf0101680 0x7fdcf0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/3663294992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00fff10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/3663294992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 0x7fdcf0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 -- 172.21.15.66:0/3663294992 >> 172.21.15.66:0/3663294992 conn(0x7fdcf00fb0e0 msgr2=0x7fdcf00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 -- 172.21.15.66:0/3663294992 shutdown_connections 2023-07-21T22:19:03.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.647+0000 7fdcf67ff700 1 -- 172.21.15.66:0/3663294992 wait complete. 2023-07-21T22:19:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.648+0000 7fdcf67ff700 1 Processor -- start 2023-07-21T22:19:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.648+0000 7fdcf67ff700 1 -- start start 2023-07-21T22:19:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf67ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf67ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 0x7fdcf0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf67ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 0x7fdcf0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf67ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00735e0 con 0x7fdcf0101680 2023-07-21T22:19:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf67ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf0073750 con 0x7fdcf0100cf0 2023-07-21T22:19:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf67ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf019f8a0 con 0x7fdcf00ffb00 2023-07-21T22:19:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf4d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 0x7fdcf0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf4d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 0x7fdcf0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43800/0 (socket says 172.21.15.66:43800) 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:59074/0 (socket says 172.21.15.66:59074) 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf4d9c700 1 -- 172.21.15.66:0/688199144 learned_addr learned my addr 172.21.15.66:0/688199144 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcef7fe700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 0x7fdcf0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 -- 172.21.15.66:0/688199144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 msgr2=0x7fdcf0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 0x7fdcf0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 -- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 msgr2=0x7fdcf0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 0x7fdcf0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdceffff700 1 -- 172.21.15.66:0/688199144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce00095f0 con 0x7fdcf00ffb00 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcf4d9c700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 0x7fdcf0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.649+0000 7fdcef7fe700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 0x7fdcf0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.650+0000 7fdceffff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00742c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdce00098e0 tx=0x7fdce0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.650+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce001d070 con 0x7fdcf00ffb00 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.650+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdce00054e0 con 0x7fdcf00ffb00 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.650+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf019fb20 con 0x7fdcf00ffb00 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.650+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0004c40 con 0x7fdcf00ffb00 2023-07-21T22:19:03.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.650+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf019ff40 con 0x7fdcf00ffb00 2023-07-21T22:19:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.651+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf010b320 con 0x7fdcf00ffb00 2023-07-21T22:19:03.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.653+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdce000fb90 con 0x7fdcf00ffb00 2023-07-21T22:19:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.654+0000 7fdced7fa700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdcd80a11a0 0x7fdcd80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.655+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdce00cdd80 con 0x7fdcf00ffb00 2023-07-21T22:19:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.655+0000 7fdcef7fe700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdcd80a11a0 0x7fdcd80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:03.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.655+0000 7fdcef7fe700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdcd80a11a0 0x7fdcd80a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005950 tx=0x7fdce40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:03.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.656+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdce0097960 con 0x7fdcf00ffb00 2023-07-21T22:19:03.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.902+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdcf01069e0 con 0x7fdcf00ffb00 2023-07-21T22:19:03.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.906+0000 7fdced7fa700 1 -- 172.21.15.66:0/688199144 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdce0016e80 con 0x7fdcf00ffb00 2023-07-21T22:19:03.951 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:03.951 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2023-07-21T22:19:03.951 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:19:03.951 INFO:teuthology.orchestra.run.smithi066.stdout: "poolnum": 1, 2023-07-21T22:19:03.951 INFO:teuthology.orchestra.run.smithi066.stdout: "poolname": ".mgr" 2023-07-21T22:19:03.951 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:19:03.952 INFO:teuthology.orchestra.run.smithi066.stdout:] 2023-07-21T22:19:03.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdcd80a11a0 msgr2=0x7fdcd80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdcd80a11a0 0x7fdcd80a3650 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdce4005950 tx=0x7fdce40058e0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 msgr2=0x7fdcf00742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00742c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdce00098e0 tx=0x7fdce0004730 comp rx=0 tx=0).stop 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 shutdown_connections 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdcd80a11a0 0x7fdcd80a3650 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdcf0101680 0x7fdcf0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf00ffb00 0x7fdcf00742c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 --2- 172.21.15.66:0/688199144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcf0100cf0 0x7fdcf0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 >> 172.21.15.66:0/688199144 conn(0x7fdcf00fb0e0 msgr2=0x7fdcf00fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 shutdown_connections 2023-07-21T22:19:03.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:03.909+0000 7fdcf67ff700 1 -- 172.21.15.66:0/688199144 wait complete. 2023-07-21T22:19:03.973 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-21T22:19:03.973 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-21T22:19:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 -- 172.21.15.66:0/3833344276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440074550 msgr2=0x7fa4400729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:04.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3833344276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440074550 0x7fa4400729a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa430009990 tx=0x7fa430009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 -- 172.21.15.66:0/3833344276 shutdown_connections 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3833344276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4401079b0 0x7fa440109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3833344276 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440074550 0x7fa4400729a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3833344276 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa440072f70 0x7fa4400733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 -- 172.21.15.66:0/3833344276 >> 172.21.15.66:0/3833344276 conn(0x7fa4400fb130 msgr2=0x7fa4400fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.148+0000 7fa4459ff700 1 -- 172.21.15.66:0/3833344276 shutdown_connections 2023-07-21T22:19:04.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.149+0000 7fa4459ff700 1 -- 172.21.15.66:0/3833344276 wait complete. 2023-07-21T22:19:04.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.149+0000 7fa4459ff700 1 Processor -- start 2023-07-21T22:19:04.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 -- start start 2023-07-21T22:19:04.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:03 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3924292693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:04.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:03 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/331070693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440072f70 0x7fa440197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa440074550 0x7fa440198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 0x7fa44019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:04.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44019f640 con 0x7fa4401079b0 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44019f780 con 0x7fa440074550 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa4459ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44019f910 con 0x7fa440072f70 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa43effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440072f70 0x7fa440197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa43f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 0x7fa44019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 0x7fa44019f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43824/0 (socket says 172.21.15.66:43824) 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.150+0000 7fa43effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440072f70 0x7fa440197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:59092/0 (socket says 172.21.15.66:59092) 2023-07-21T22:19:04.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 -- 172.21.15.66:0/3043814440 learned_addr learned my addr 172.21.15.66:0/3043814440 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43e7fc700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa440074550 0x7fa440198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 -- 172.21.15.66:0/3043814440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440072f70 msgr2=0x7fa440197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440072f70 0x7fa440197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 -- 172.21.15.66:0/3043814440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa440074550 msgr2=0x7fa440198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa440074550 0x7fa440198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 -- 172.21.15.66:0/3043814440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438009c40 con 0x7fa4401079b0 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa43f7fe700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 0x7fa44019f070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c370 tx=0x7fa43800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:04.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.151+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438017070 con 0x7fa4401079b0 2023-07-21T22:19:04.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.152+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4300095f0 con 0x7fa4401079b0 2023-07-21T22:19:04.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.152+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa43800f040 con 0x7fa4401079b0 2023-07-21T22:19:04.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.152+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800e050 con 0x7fa4401079b0 2023-07-21T22:19:04.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.152+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44019ff60 con 0x7fa4401079b0 2023-07-21T22:19:04.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.153+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa440106060 con 0x7fa4401079b0 2023-07-21T22:19:04.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.155+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa43800cb60 con 0x7fa4401079b0 2023-07-21T22:19:04.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.157+0000 7fa4449fd700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa4280a1150 0x7fa4280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:04.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.157+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa4380cbe30 con 0x7fa4401079b0 2023-07-21T22:19:04.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.157+0000 7fa43effd700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa4280a1150 0x7fa4280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:04.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.158+0000 7fa43effd700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa4280a1150 0x7fa4280a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b2e0 tx=0x7fa430005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:04.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.161+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa438095980 con 0x7fa4401079b0 2023-07-21T22:19:04.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:03 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3924292693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:04.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:03 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/331070693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:04.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:03 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3924292693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:04.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:03 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/331070693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:04.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:04.388+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa44002c850 con 0x7fa4401079b0 2023-07-21T22:19:05.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.150+0000 7fa4449fd700 1 -- 172.21.15.66:0/3043814440 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa438098fa0 con 0x7fa4401079b0 2023-07-21T22:19:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.155+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa4280a1150 msgr2=0x7fa4280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:05.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.155+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa4280a1150 0x7fa4280a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b2e0 tx=0x7fa430005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.155+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 msgr2=0x7fa44019f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:05.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.155+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 0x7fa44019f070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c370 tx=0x7fa43800c680 comp rx=0 tx=0).stop 2023-07-21T22:19:05.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.156+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 shutdown_connections 2023-07-21T22:19:05.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.156+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa4280a1150 0x7fa4280a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.156+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4401079b0 0x7fa44019f070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.156+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa440072f70 0x7fa440197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.156+0000 7fa4459ff700 1 --2- 172.21.15.66:0/3043814440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa440074550 0x7fa440198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.156+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 >> 172.21.15.66:0/3043814440 conn(0x7fa4400fb130 msgr2=0x7fa4400fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:05.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.158+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 shutdown_connections 2023-07-21T22:19:05.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.158+0000 7fa4459ff700 1 -- 172.21.15.66:0/3043814440 wait complete. 2023-07-21T22:19:05.204 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' created 2023-07-21T22:19:05.223 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-07-21T22:19:05.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/488173745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 msgr2=0x7f1f84100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:05.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/488173745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84100020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009990 tx=0x7f1f70009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/488173745 shutdown_connections 2023-07-21T22:19:05.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/488173745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f84100560 0x7f1f84106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/488173745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f840fe9e0 0x7f1f840fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/488173745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84100020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.395+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/488173745 >> 172.21.15.66:0/488173745 conn(0x7f1f840fa000 msgr2=0x7f1f840fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:05.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.396+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/488173745 shutdown_connections 2023-07-21T22:19:05.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.396+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/488173745 wait complete. 2023-07-21T22:19:05.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.397+0000 7f1f8b1ff700 1 Processor -- start 2023-07-21T22:19:05.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.397+0000 7f1f8b1ff700 1 -- start start 2023-07-21T22:19:05.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8b1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f840fe9e0 0x7f1f841958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:05.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8b1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:05.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8b1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f84100560 0x7f1f8419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:05.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8b1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419d420 con 0x7f1f840fe9e0 2023-07-21T22:19:05.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8b1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419d560 con 0x7f1f840ffbd0 2023-07-21T22:19:05.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8b1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419d6f0 con 0x7f1f84100560 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f88f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f840fe9e0 0x7f1f841958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f83fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:46436/0 (socket says 172.21.15.66:46436) 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f83fff700 1 -- 172.21.15.66:0/691119873 learned_addr learned my addr 172.21.15.66:0/691119873 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.398+0000 7f1f8979c700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f84100560 0x7f1f8419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8979c700 1 -- 172.21.15.66:0/691119873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 msgr2=0x7f1f84195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:05.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8979c700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8979c700 1 -- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f840fe9e0 msgr2=0x7f1f841958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8979c700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f840fe9e0 0x7f1f841958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8979c700 1 -- 172.21.15.66:0/691119873 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009520 con 0x7f1f84100560 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f83fff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8979c700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f84100560 0x7f1f8419ce50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00efc0 tx=0x7f1f7c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c018070 con 0x7f1f84100560 2023-07-21T22:19:05.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f700095f0 con 0x7f1f84100560 2023-07-21T22:19:05.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c00f040 con 0x7f1f84100560 2023-07-21T22:19:05.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00cd40 con 0x7f1f84100560 2023-07-21T22:19:05.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.399+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8419dcf0 con 0x7f1f84100560 2023-07-21T22:19:05.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/688199144' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:19:05.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:05.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:05 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3043814440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:19:05.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.400+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8410a200 con 0x7f1f84100560 2023-07-21T22:19:05.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.403+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1f7c003a00 con 0x7f1f84100560 2023-07-21T22:19:05.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.404+0000 7f1f81ffb700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1f740a1150 0x7f1f740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:05.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.404+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f1f7c0cc3b0 con 0x7f1f84100560 2023-07-21T22:19:05.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.404+0000 7f1f88f9b700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1f740a1150 0x7f1f740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:05.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.405+0000 7f1f88f9b700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1f740a1150 0x7f1f740a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f840ffa30 tx=0x7f1f78009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:05.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.406+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1f7c095e20 con 0x7f1f84100560 2023-07-21T22:19:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/688199144' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:19:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:05.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:05 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3043814440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:19:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/688199144' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T22:19:05.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:05.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:05.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:05 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3043814440' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T22:19:05.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:05.618+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f1f8410a390 con 0x7f1f84100560 2023-07-21T22:19:06.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.356+0000 7f1f81ffb700 1 -- 172.21.15.66:0/691119873 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) v1 ==== 171+0+0 (secure 0 0 0) 0x7f1f7c099440 con 0x7f1f84100560 2023-07-21T22:19:06.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.365+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1f740a1150 msgr2=0x7f1f740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.365+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1f740a1150 0x7f1f740a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1f840ffa30 tx=0x7f1f78009260 comp rx=0 tx=0).stop 2023-07-21T22:19:06.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.365+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f84100560 msgr2=0x7f1f8419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.365+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f84100560 0x7f1f8419ce50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00efc0 tx=0x7f1f7c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.369+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 shutdown_connections 2023-07-21T22:19:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.369+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1f740a1150 0x7f1f740a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.369+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f84100560 0x7f1f8419ce50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.369+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1f840fe9e0 0x7f1f841958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.369+0000 7f1f8b1ff700 1 --2- 172.21.15.66:0/691119873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f840ffbd0 0x7f1f84195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.369+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 >> 172.21.15.66:0/691119873 conn(0x7f1f840fa000 msgr2=0x7f1f84067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.376+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 shutdown_connections 2023-07-21T22:19:06.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.377+0000 7f1f8b1ff700 1 -- 172.21.15.66:0/691119873 wait complete. 2023-07-21T22:19:06.424 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' created 2023-07-21T22:19:06.443 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 -- 172.21.15.66:0/643966177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 msgr2=0x7f93a0107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 --2- 172.21.15.66:0/643966177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 0x7f93a0107f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9398009f90 tx=0x7f939800b330 comp rx=0 tx=0).stop 2023-07-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 -- 172.21.15.66:0/643966177 shutdown_connections 2023-07-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 --2- 172.21.15.66:0/643966177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 0x7f93a0107f20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 --2- 172.21.15.66:0/643966177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a00ffae0 0x7f93a00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 --2- 172.21.15.66:0/643966177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a0100cd0 0x7f93a0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 -- 172.21.15.66:0/643966177 >> 172.21.15.66:0/643966177 conn(0x7f93a00fb0e0 msgr2=0x7f93a00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:06.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 -- 172.21.15.66:0/643966177 shutdown_connections 2023-07-21T22:19:06.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.616+0000 7f93a67ff700 1 -- 172.21.15.66:0/643966177 wait complete. 2023-07-21T22:19:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.617+0000 7f93a67ff700 1 Processor -- start 2023-07-21T22:19:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.617+0000 7f93a67ff700 1 -- start start 2023-07-21T22:19:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.617+0000 7f93a67ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a00ffae0 0x7f93a0197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.617+0000 7f93a67ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0100cd0 0x7f93a01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.617+0000 7f93a67ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 0x7f93a019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f93a67ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a019f700 con 0x7f93a00ffae0 2023-07-21T22:19:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f93a67ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a019f840 con 0x7f93a0101660 2023-07-21T22:19:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f93a67ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a019f9d0 con 0x7f93a0100cd0 2023-07-21T22:19:06.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a00ffae0 0x7f93a0197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0100cd0 0x7f93a01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f93a4d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 0x7f93a019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0100cd0 0x7f93a01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:59144/0 (socket says 172.21.15.66:59144) 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939f7fe700 1 -- 172.21.15.66:0/308331733 learned_addr learned my addr 172.21.15.66:0/308331733 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939ffff700 1 -- 172.21.15.66:0/308331733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0100cd0 msgr2=0x7f93a01980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939ffff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0100cd0 0x7f93a01980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939ffff700 1 -- 172.21.15.66:0/308331733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 msgr2=0x7f93a019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939ffff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 0x7f93a019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.618+0000 7f939ffff700 1 -- 172.21.15.66:0/308331733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9398009c40 con 0x7f93a00ffae0 2023-07-21T22:19:06.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.619+0000 7f939ffff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a00ffae0 0x7f93a0197b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9390005fd0 tx=0x7f939000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.619+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390004220 con 0x7f93a00ffae0 2023-07-21T22:19:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.619+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a019fcb0 con 0x7f93a00ffae0 2023-07-21T22:19:06.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.619+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9390004380 con 0x7f93a00ffae0 2023-07-21T22:19:06.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.619+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390003be0 con 0x7f93a00ffae0 2023-07-21T22:19:06.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.619+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a01a01c0 con 0x7f93a00ffae0 2023-07-21T22:19:06.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.620+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a010b310 con 0x7f93a00ffae0 2023-07-21T22:19:06.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.622+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9390003d40 con 0x7f93a00ffae0 2023-07-21T22:19:06.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.623+0000 7f939d7fa700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93880a1150 0x7f93880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:06.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.624+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f9390014070 con 0x7f93a00ffae0 2023-07-21T22:19:06.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.624+0000 7f939f7fe700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93880a1150 0x7f93880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:06.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.624+0000 7f939f7fe700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93880a1150 0x7f93880a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9394009d20 tx=0x7f9394009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:06.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.628+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f93900960d0 con 0x7f93a00ffae0 2023-07-21T22:19:06.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:06.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:06.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3043814440' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:19:06.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:19:06.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:06 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:06.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:06.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:06.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3043814440' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:19:06.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:19:06.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:06 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:06.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3043814440' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T22:19:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T22:19:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:06 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:19:06.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:06.897+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f93a010b4d0 con 0x7f93a00ffae0 2023-07-21T22:19:07.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.401+0000 7f939d7fa700 1 -- 172.21.15.66:0/308331733 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f93a010b4d0 con 0x7f93a00ffae0 2023-07-21T22:19:07.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93880a1150 msgr2=0x7f93880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93880a1150 0x7f93880a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9394009d20 tx=0x7f9394009260 comp rx=0 tx=0).stop 2023-07-21T22:19:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a00ffae0 msgr2=0x7f93a0197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:07.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a00ffae0 0x7f93a0197b80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9390005fd0 tx=0x7f939000eca0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 shutdown_connections 2023-07-21T22:19:07.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93880a1150 0x7f93880a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93a0101660 0x7f93a019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93a00ffae0 0x7f93a0197b80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 --2- 172.21.15.66:0/308331733 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93a0100cd0 0x7f93a01980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.406+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 >> 172.21.15.66:0/308331733 conn(0x7f93a00fb0e0 msgr2=0x7f93a0106560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:07.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.408+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 shutdown_connections 2023-07-21T22:19:07.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.408+0000 7f93a67ff700 1 -- 172.21.15.66:0/308331733 wait complete. 2023-07-21T22:19:07.454 INFO:teuthology.orchestra.run.smithi066.stderr:new fs with metadata pool 2 and data pool 3 2023-07-21T22:19:07.472 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-07-21T22:19:07.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-07-21T22:19:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/691119873' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-21T22:19:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-21T22:19:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: pgmap v267: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-21T22:19:07.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:19:07.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/308331733' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:19:07.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:07 smithi066 ceph-mon[126379]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T22:19:07.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 -- 172.21.15.66:0/404405714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1380ffae0 msgr2=0x7fb1380ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:07.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/404405714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1380ffae0 0x7fb1380ffef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb134009990 tx=0x7fb134009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 -- 172.21.15.66:0/404405714 shutdown_connections 2023-07-21T22:19:07.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/404405714 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 0x7fb138107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/404405714 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1380ffae0 0x7fb1380ffef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/404405714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb138100cd0 0x7fb138101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.653+0000 7fb13f9ff700 1 -- 172.21.15.66:0/404405714 >> 172.21.15.66:0/404405714 conn(0x7fb1380fb0e0 msgr2=0x7fb1380fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.654+0000 7fb13f9ff700 1 -- 172.21.15.66:0/404405714 shutdown_connections 2023-07-21T22:19:07.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.654+0000 7fb13f9ff700 1 -- 172.21.15.66:0/404405714 wait complete. 2023-07-21T22:19:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 Processor -- start 2023-07-21T22:19:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 -- start start 2023-07-21T22:19:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 0x7fb138197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb138100cd0 0x7fb138198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 0x7fb13819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:07.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13819f660 con 0x7fb138100cd0 2023-07-21T22:19:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13819f7a0 con 0x7fb1380ffae0 2023-07-21T22:19:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13f9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13819f930 con 0x7fb138101660 2023-07-21T22:19:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13d79b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 0x7fb138197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13d79b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 0x7fb138197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:46464/0 (socket says 172.21.15.66:46464) 2023-07-21T22:19:07.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13cf9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb138100cd0 0x7fb138198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.655+0000 7fb13d79b700 1 -- 172.21.15.66:0/3398896424 learned_addr learned my addr 172.21.15.66:0/3398896424 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13df9c700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 0x7fb13819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13d79b700 1 -- 172.21.15.66:0/3398896424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 msgr2=0x7fb13819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13d79b700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 0x7fb13819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13d79b700 1 -- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb138100cd0 msgr2=0x7fb138198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13d79b700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb138100cd0 0x7fb138198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13d79b700 1 -- 172.21.15.66:0/3398896424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340095f0 con 0x7fb1380ffae0 2023-07-21T22:19:07.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13df9c700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 0x7fb13819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13d79b700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 0x7fb138197ae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b620 tx=0x7fb134005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13401d070 con 0x7fb1380ffae0 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb134021410 con 0x7fb1380ffae0 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb13819fbb0 con 0x7fb1380ffae0 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.656+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134017650 con 0x7fb1380ffae0 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.657+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13819ffd0 con 0x7fb1380ffae0 2023-07-21T22:19:07.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.657+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1381921c0 con 0x7fb1380ffae0 2023-07-21T22:19:07.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.660+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb134021580 con 0x7fb1380ffae0 2023-07-21T22:19:07.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.661+0000 7fb12e7fc700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb1240a1150 0x7fb1240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:07.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.661+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1340cde90 con 0x7fb1380ffae0 2023-07-21T22:19:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.661+0000 7fb13cf9a700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb1240a1150 0x7fb1240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.662+0000 7fb13cf9a700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb1240a1150 0x7fb1240a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb128009c60 tx=0x7fb128009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:07.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.663+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb134097720 con 0x7fb1380ffae0 2023-07-21T22:19:07.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/691119873' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-21T22:19:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-21T22:19:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: pgmap v267: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-21T22:19:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:19:07.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/308331733' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:19:07.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:07 smithi149 ceph-mon[129268]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T22:19:07.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/691119873' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-21T22:19:07.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-07-21T22:19:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: pgmap v267: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:19:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-07-21T22:19:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T22:19:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/308331733' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T22:19:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:07 smithi104 ceph-mon[129021]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T22:19:07.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:07.898+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fb13802caa0 con 0x7fb1380ffae0 2023-07-21T22:19:08.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.358+0000 7fb12e7fc700 1 -- 172.21.15.66:0/3398896424 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) v1 ==== 157+0+0 (secure 0 0 0) 0x7fb13409ad40 con 0x7fb1380ffae0 2023-07-21T22:19:08.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.360+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb1240a1150 msgr2=0x7fb1240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:08.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.360+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb1240a1150 0x7fb1240a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb128009c60 tx=0x7fb128009260 comp rx=0 tx=0).stop 2023-07-21T22:19:08.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 msgr2=0x7fb138197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:08.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 0x7fb138197ae0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb13400b620 tx=0x7fb134005740 comp rx=0 tx=0).stop 2023-07-21T22:19:08.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 shutdown_connections 2023-07-21T22:19:08.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb1240a1150 0x7fb1240a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1380ffae0 0x7fb138197ae0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb138101660 0x7fb13819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 --2- 172.21.15.66:0/3398896424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb138100cd0 0x7fb138198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 >> 172.21.15.66:0/3398896424 conn(0x7fb1380fb0e0 msgr2=0x7fb1381065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:08.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 shutdown_connections 2023-07-21T22:19:08.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.361+0000 7fb13f9ff700 1 -- 172.21.15.66:0/3398896424 wait complete. 2023-07-21T22:19:08.425 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-07-21T22:19:08.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 -- 172.21.15.66:0/2634519529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 msgr2=0x7f281c073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:08.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 --2- 172.21.15.66:0/2634519529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c073360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2804009990 tx=0x7f2804009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 -- 172.21.15.66:0/2634519529 shutdown_connections 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 --2- 172.21.15.66:0/2634519529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 0x7f281c072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 --2- 172.21.15.66:0/2634519529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 --2- 172.21.15.66:0/2634519529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c073360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 -- 172.21.15.66:0/2634519529 >> 172.21.15.66:0/2634519529 conn(0x7f281c0fb160 msgr2=0x7f281c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 -- 172.21.15.66:0/2634519529 shutdown_connections 2023-07-21T22:19:08.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.613+0000 7f28227ff700 1 -- 172.21.15.66:0/2634519529 wait complete. 2023-07-21T22:19:08.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.615+0000 7f28227ff700 1 Processor -- start 2023-07-21T22:19:08.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.615+0000 7f28227ff700 1 -- start start 2023-07-21T22:19:08.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.616+0000 7f28227ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:08.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.616+0000 7f28227ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 0x7f281c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:08.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.616+0000 7f28227ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:08.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.616+0000 7f28227ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c19f540 con 0x7f281c072ef0 2023-07-21T22:19:08.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.616+0000 7f28227ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c069f30 con 0x7f281c0744d0 2023-07-21T22:19:08.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.616+0000 7f28227ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c06a0c0 con 0x7f281c1079f0 2023-07-21T22:19:08.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:08.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:59162/0 (socket says 172.21.15.66:59162) 2023-07-21T22:19:08.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 -- 172.21.15.66:0/2063039604 learned_addr learned my addr 172.21.15.66:0/2063039604 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2813fff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 0x7f281c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f281bfff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 -- 172.21.15.66:0/2063039604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 msgr2=0x7f281c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 0x7f281c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 -- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 msgr2=0x7f281c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2820d9c700 1 -- 172.21.15.66:0/2063039604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c009520 con 0x7f281c1079f0 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f2813fff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 0x7f281c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.617+0000 7f281bfff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.618+0000 7f2820d9c700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c19ef70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f281400ef30 tx=0x7f281400c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.618+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814018070 con 0x7f281c1079f0 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.618+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281400f040 con 0x7f281c1079f0 2023-07-21T22:19:08.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.618+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040095f0 con 0x7f281c1079f0 2023-07-21T22:19:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.618+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814003dc0 con 0x7f281c1079f0 2023-07-21T22:19:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.618+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c06a6c0 con 0x7f281c1079f0 2023-07-21T22:19:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.619+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c106130 con 0x7f281c1079f0 2023-07-21T22:19:08.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.621+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f281400caa0 con 0x7f281c1079f0 2023-07-21T22:19:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.623+0000 7f2819ffb700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f28080a1270 0x7f28080a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:08.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.623+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f28140cc5a0 con 0x7f281c1079f0 2023-07-21T22:19:08.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.623+0000 7f281bfff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f28080a1270 0x7f28080a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:08.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.623+0000 7f281bfff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f28080a1270 0x7f28080a3720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f280c009e40 tx=0x7f280c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:08.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.627+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2814095d40 con 0x7f281c1079f0 2023-07-21T22:19:08.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:19:08.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/308331733' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:19:08.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T22:19:08.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T22:19:08.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] up:active 2023-07-21T22:19:08.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:08.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3398896424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-21T22:19:08.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-21T22:19:08.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:08 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:19:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/308331733' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:19:08.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T22:19:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T22:19:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] up:active 2023-07-21T22:19:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:08.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3398896424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-21T22:19:08.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-21T22:19:08.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:08 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T22:19:08.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/308331733' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] up:active 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3398896424' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-07-21T22:19:08.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:08 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:08.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:08.839+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f281c02c850 con 0x7f281c1079f0 2023-07-21T22:19:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: pgmap v270: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-07-21T22:19:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-21T22:19:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:19:09.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2063039604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:19:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:19:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:09.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: pgmap v270: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-07-21T22:19:09.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-21T22:19:09.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:19:09.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2063039604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:19:09.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:19:09.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:09.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:09.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: pgmap v270: 97 pgs: 11 creating+peering, 20 active+clean, 66 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2063039604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:09.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:19:09.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:19:09.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:19:10.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.371+0000 7f2819ffb700 1 -- 172.21.15.66:0/2063039604 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) v1 ==== 228+0+0 (secure 0 0 0) 0x7f2814099360 con 0x7f281c1079f0 2023-07-21T22:19:10.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f28080a1270 msgr2=0x7f28080a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:10.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f28080a1270 0x7f28080a3720 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f280c009e40 tx=0x7f280c009260 comp rx=0 tx=0).stop 2023-07-21T22:19:10.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 msgr2=0x7f281c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:10.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c19ef70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f281400ef30 tx=0x7f281400c450 comp rx=0 tx=0).stop 2023-07-21T22:19:10.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 shutdown_connections 2023-07-21T22:19:10.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f28080a1270 0x7f28080a3720 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f281c1079f0 0x7f281c19ef70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f281c072ef0 0x7f281c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 --2- 172.21.15.66:0/2063039604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f281c0744d0 0x7f281c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 >> 172.21.15.66:0/2063039604 conn(0x7f281c0fb160 msgr2=0x7f281c109cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:10.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 shutdown_connections 2023-07-21T22:19:10.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.377+0000 7f28227ff700 1 -- 172.21.15.66:0/2063039604 wait complete. 2023-07-21T22:19:10.425 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' created 2023-07-21T22:19:10.442 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-07-21T22:19:10.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 -- 172.21.15.66:0/1549395725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c101660 msgr2=0x7fd84c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:10.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 --2- 172.21.15.66:0/1549395725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c101660 0x7fd84c107f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd844009f90 tx=0x7fd84400b330 comp rx=0 tx=0).stop 2023-07-21T22:19:10.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 -- 172.21.15.66:0/1549395725 shutdown_connections 2023-07-21T22:19:10.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 --2- 172.21.15.66:0/1549395725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 --2- 172.21.15.66:0/1549395725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c101660 0x7fd84c107f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 --2- 172.21.15.66:0/1549395725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd84c100cd0 0x7fd84c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 -- 172.21.15.66:0/1549395725 >> 172.21.15.66:0/1549395725 conn(0x7fd84c0fb0e0 msgr2=0x7fd84c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:10.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 -- 172.21.15.66:0/1549395725 shutdown_connections 2023-07-21T22:19:10.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.608+0000 7fd851bff700 1 -- 172.21.15.66:0/1549395725 wait complete. 2023-07-21T22:19:10.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.609+0000 7fd851bff700 1 Processor -- start 2023-07-21T22:19:10.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 -- start start 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c100cd0 0x7fd84c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd84c101660 0x7fd84c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0735e0 con 0x7fd84c0ffae0 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c073750 con 0x7fd84c100cd0 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd851bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c19f940 con 0x7fd84c101660 2023-07-21T22:19:10.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd84b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd84b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43916/0 (socket says 172.21.15.66:43916) 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd84b7fe700 1 -- 172.21.15.66:0/1518259874 learned_addr learned my addr 172.21.15.66:0/1518259874 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd843fff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c100cd0 0x7fd84c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.610+0000 7fd84bfff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd84c101660 0x7fd84c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd84b7fe700 1 -- 172.21.15.66:0/1518259874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd84c101660 msgr2=0x7fd84c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd84b7fe700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd84c101660 0x7fd84c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd84b7fe700 1 -- 172.21.15.66:0/1518259874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c100cd0 msgr2=0x7fd84c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:10.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd84b7fe700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c100cd0 0x7fd84c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd84b7fe700 1 -- 172.21.15.66:0/1518259874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c009520 con 0x7fd84c0ffae0 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd843fff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c100cd0 0x7fd84c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd84b7fe700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0742c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00ea50 tx=0x7fd83c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00eee0 con 0x7fd84c0ffae0 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd83c00cbd0 con 0x7fd84c0ffae0 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd844009c40 con 0x7fd84c0ffae0 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c018680 con 0x7fd84c0ffae0 2023-07-21T22:19:10.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.611+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c19ffa0 con 0x7fd84c0ffae0 2023-07-21T22:19:10.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.613+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c10b2c0 con 0x7fd84c0ffae0 2023-07-21T22:19:10.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.615+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd83c010430 con 0x7fd84c0ffae0 2023-07-21T22:19:10.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.616+0000 7fd8497fa700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8380a1150 0x7fd8380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:10.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.617+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fd83c014070 con 0x7fd84c0ffae0 2023-07-21T22:19:10.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.617+0000 7fd843fff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8380a1150 0x7fd8380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:10.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.617+0000 7fd843fff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8380a1150 0x7fd8380a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd834005fd0 tx=0x7fd834009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:10.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.619+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd83c0969b0 con 0x7fd84c0ffae0 2023-07-21T22:19:10.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:10 smithi066 ceph-mon[126379]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:19:10.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:10 smithi066 ceph-mon[126379]: pgmap v273: 97 pgs: 11 creating+peering, 82 active+clean, 4 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-07-21T22:19:10.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:10 smithi149 ceph-mon[129268]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:19:10.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:10 smithi149 ceph-mon[129268]: pgmap v273: 97 pgs: 11 creating+peering, 82 active+clean, 4 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-07-21T22:19:10.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:10 smithi104 ceph-mon[129021]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T22:19:10.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:10 smithi104 ceph-mon[129021]: pgmap v273: 97 pgs: 11 creating+peering, 82 active+clean, 4 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 7 op/s 2023-07-21T22:19:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:10.828+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd84c106960 con 0x7fd84c0ffae0 2023-07-21T22:19:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.374+0000 7fd8497fa700 1 -- 172.21.15.66:0/1518259874 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd83c099fd0 con 0x7fd84c0ffae0 2023-07-21T22:19:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.385+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8380a1150 msgr2=0x7fd8380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:11.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.385+0000 7fd851bff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8380a1150 0x7fd8380a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd834005fd0 tx=0x7fd834009310 comp rx=0 tx=0).stop 2023-07-21T22:19:11.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.385+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 msgr2=0x7fd84c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:11.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.385+0000 7fd851bff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0742c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00ea50 tx=0x7fd83c00ed60 comp rx=0 tx=0).stop 2023-07-21T22:19:11.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.390+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 shutdown_connections 2023-07-21T22:19:11.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.390+0000 7fd851bff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8380a1150 0x7fd8380a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.390+0000 7fd851bff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd84c101660 0x7fd84c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.390+0000 7fd851bff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd84c0ffae0 0x7fd84c0742c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.390+0000 7fd851bff700 1 --2- 172.21.15.66:0/1518259874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd84c100cd0 0x7fd84c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.390+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 >> 172.21.15.66:0/1518259874 conn(0x7fd84c0fb0e0 msgr2=0x7fd84c0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:11.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.391+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 shutdown_connections 2023-07-21T22:19:11.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.392+0000 7fd851bff700 1 -- 172.21.15.66:0/1518259874 wait complete. 2023-07-21T22:19:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:set pool 4 allow_ec_overwrites to true 2023-07-21T22:19:11.461 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-07-21T22:19:11.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 -- 172.21.15.66:0/3074866742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc1079a0 msgr2=0x7f36bc109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 --2- 172.21.15.66:0/3074866742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc1079a0 0x7f36bc109e20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b0a0 tx=0x7f36b400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 -- 172.21.15.66:0/3074866742 shutdown_connections 2023-07-21T22:19:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 --2- 172.21.15.66:0/3074866742 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc1079a0 0x7f36bc109e20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 --2- 172.21.15.66:0/3074866742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36bc074550 0x7f36bc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 --2- 172.21.15.66:0/3074866742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc072f70 0x7f36bc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 -- 172.21.15.66:0/3074866742 >> 172.21.15.66:0/3074866742 conn(0x7f36bc0fb0e0 msgr2=0x7f36bc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:11.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 -- 172.21.15.66:0/3074866742 shutdown_connections 2023-07-21T22:19:11.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.627+0000 7f36c2bff700 1 -- 172.21.15.66:0/3074866742 wait complete. 2023-07-21T22:19:11.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.628+0000 7f36c2bff700 1 Processor -- start 2023-07-21T22:19:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 -- start start 2023-07-21T22:19:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc072f70 0x7f36bc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 0x7f36bc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36bc1079a0 0x7f36bc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19f710 con 0x7f36bc074550 2023-07-21T22:19:11.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19f850 con 0x7f36bc072f70 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c2bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc19f9e0 con 0x7f36bc1079a0 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 0x7f36bc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 0x7f36bc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:43928/0 (socket says 172.21.15.66:43928) 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36bbfff700 1 -- 172.21.15.66:0/2161366827 learned_addr learned my addr 172.21.15.66:0/2161366827 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c099b700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc072f70 0x7f36bc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.629+0000 7f36c119c700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36bc1079a0 0x7f36bc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:11.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36bbfff700 1 -- 172.21.15.66:0/2161366827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36bc1079a0 msgr2=0x7f36bc19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36bbfff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36bc1079a0 0x7f36bc19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36bbfff700 1 -- 172.21.15.66:0/2161366827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc072f70 msgr2=0x7f36bc197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36bbfff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc072f70 0x7f36bc197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36bbfff700 1 -- 172.21.15.66:0/2161366827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b4009d00 con 0x7f36bc074550 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36bbfff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 0x7f36bc1980d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36a800e970 tx=0x7f36a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8009790 con 0x7f36bc074550 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc19fcc0 con 0x7f36bc074550 2023-07-21T22:19:11.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a800c7e0 con 0x7f36bc074550 2023-07-21T22:19:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8003c70 con 0x7f36bc074550 2023-07-21T22:19:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.630+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc1a01d0 con 0x7f36bc074550 2023-07-21T22:19:11.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.631+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc106050 con 0x7f36bc074550 2023-07-21T22:19:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.633+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f36a800cc50 con 0x7f36bc074550 2023-07-21T22:19:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.635+0000 7f36b9ffb700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36ac0a1150 0x7f36ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.635+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f36a8014070 con 0x7f36bc074550 2023-07-21T22:19:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.635+0000 7f36c099b700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36ac0a1150 0x7f36ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:11.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.635+0000 7f36c099b700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36ac0a1150 0x7f36ac0a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36bc074920 tx=0x7f36b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:11.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.638+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f36a80961f0 con 0x7f36bc074550 2023-07-21T22:19:11.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:11 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:19:11.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:11 smithi066 ceph-mon[126379]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:19:11.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:11 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1518259874' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-21T22:19:11.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:11 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:19:11.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:11 smithi149 ceph-mon[129268]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:19:11.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:11 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1518259874' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-21T22:19:11.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:11 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T22:19:11.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:11 smithi104 ceph-mon[129021]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T22:19:11.787 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:11 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1518259874' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-07-21T22:19:11.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:11.900+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f36bc02c850 con 0x7f36bc074550 2023-07-21T22:19:12.118 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:11 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: 2023-07-21T22:19:11.742+0000 7f2fde972700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-21T22:19:12.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.395+0000 7f36b9ffb700 1 -- 172.21.15.66:0/2161366827 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) v1 ==== 137+0+0 (secure 0 0 0) 0x7f36a8003dd0 con 0x7f36bc074550 2023-07-21T22:19:12.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36ac0a1150 msgr2=0x7f36ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36ac0a1150 0x7f36ac0a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36bc074920 tx=0x7f36b0009260 comp rx=0 tx=0).stop 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 msgr2=0x7f36bc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 0x7f36bc1980d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36a800e970 tx=0x7f36a800ec80 comp rx=0 tx=0).stop 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 shutdown_connections 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36ac0a1150 0x7f36ac0a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36bc072f70 0x7f36bc197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36bc1079a0 0x7f36bc19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 --2- 172.21.15.66:0/2161366827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36bc074550 0x7f36bc1980d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.397+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 >> 172.21.15.66:0/2161366827 conn(0x7f36bc0fb0e0 msgr2=0x7f36bc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.398+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 shutdown_connections 2023-07-21T22:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.398+0000 7f36c2bff700 1 -- 172.21.15.66:0/2161366827 wait complete. 2023-07-21T22:19:12.444 INFO:teuthology.orchestra.run.smithi066.stderr:added data pool 4 to fsmap 2023-07-21T22:19:12.462 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:19:12.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 -- 172.21.15.66:0/959375288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2380ffb00 msgr2=0x7fc2380fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/959375288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2380ffb00 0x7fc2380fff10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc228009990 tx=0x7fc228009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 -- 172.21.15.66:0/959375288 shutdown_connections 2023-07-21T22:19:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/959375288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc238101680 0x7fc238107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/959375288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc2380ffb00 0x7fc2380fff10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/959375288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc238100cf0 0x7fc238101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 -- 172.21.15.66:0/959375288 >> 172.21.15.66:0/959375288 conn(0x7fc2380fb0e0 msgr2=0x7fc2380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:12.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 -- 172.21.15.66:0/959375288 shutdown_connections 2023-07-21T22:19:12.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.626+0000 7fc23e3ff700 1 -- 172.21.15.66:0/959375288 wait complete. 2023-07-21T22:19:12.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.627+0000 7fc23e3ff700 1 Processor -- start 2023-07-21T22:19:12.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.627+0000 7fc23e3ff700 1 -- start start 2023-07-21T22:19:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23e3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2380ffb00 0x7fc2381957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23e3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 0x7fc238195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23e3ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc238101680 0x7fc23819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23e3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23819d350 con 0x7fc2380ffb00 2023-07-21T22:19:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23e3ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23819d490 con 0x7fc238101680 2023-07-21T22:19:12.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23e3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23819d620 con 0x7fc238100cf0 2023-07-21T22:19:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 0x7fc238195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc237fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2380ffb00 0x7fc2381957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 0x7fc238195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:41840/0 (socket says 172.21.15.66:41840) 2023-07-21T22:19:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 -- 172.21.15.66:0/2279684601 learned_addr learned my addr 172.21.15.66:0/2279684601 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc23c99c700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc238101680 0x7fc23819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:12.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 -- 172.21.15.66:0/2279684601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc238101680 msgr2=0x7fc23819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc238101680 0x7fc23819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 -- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2380ffb00 msgr2=0x7fc2381957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2380ffb00 0x7fc2381957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.628+0000 7fc2377fe700 1 -- 172.21.15.66:0/2279684601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280095f0 con 0x7fc238100cf0 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.629+0000 7fc2377fe700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 0x7fc238195d10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d980 tx=0x7fc22c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.629+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c004220 con 0x7fc238100cf0 2023-07-21T22:19:12.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.629+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc22c007640 con 0x7fc238100cf0 2023-07-21T22:19:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.629+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23819d900 con 0x7fc238100cf0 2023-07-21T22:19:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.629+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22c005690 con 0x7fc238100cf0 2023-07-21T22:19:12.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.629+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23819de10 con 0x7fc238100cf0 2023-07-21T22:19:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.630+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc238107070 con 0x7fc238100cf0 2023-07-21T22:19:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.633+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc22c004750 con 0x7fc238100cf0 2023-07-21T22:19:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.634+0000 7fc2357fa700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2200a11a0 0x7fc2200a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:12.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.634+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc22c0ccc40 con 0x7fc238100cf0 2023-07-21T22:19:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.634+0000 7fc237fff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2200a11a0 0x7fc2200a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:12.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.635+0000 7fc237fff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2200a11a0 0x7fc2200a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc22800b2e0 tx=0x7fc228005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:12.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.637+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc22c096140 con 0x7fc238100cf0 2023-07-21T22:19:12.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:12 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1518259874' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-21T22:19:12.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:12 smithi066 ceph-mon[126379]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T22:19:12.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:12 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2161366827' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-21T22:19:12.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:12 smithi066 ceph-mon[126379]: pgmap v276: 129 pgs: 16 creating+peering, 92 active+clean, 21 unknown; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2023-07-21T22:19:12.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:12 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1518259874' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-21T22:19:12.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:12 smithi149 ceph-mon[129268]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T22:19:12.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:12 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2161366827' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-21T22:19:12.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:12 smithi149 ceph-mon[129268]: pgmap v276: 129 pgs: 16 creating+peering, 92 active+clean, 21 unknown; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2023-07-21T22:19:12.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:12 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1518259874' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-07-21T22:19:12.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:12 smithi104 ceph-mon[129021]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T22:19:12.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:12 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2161366827' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-07-21T22:19:12.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:12 smithi104 ceph-mon[129021]: pgmap v276: 129 pgs: 16 creating+peering, 92 active+clean, 21 unknown; 579 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 8 op/s 2023-07-21T22:19:12.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.894+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc23802c850 con 0x7fc238100cf0 2023-07-21T22:19:12.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.896+0000 7fc2357fa700 1 -- 172.21.15.66:0/2279684601 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+10759 (secure 0 0 0) 0x7fc22c016020 con 0x7fc238100cf0 2023-07-21T22:19:12.940 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:12.940 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:12.386591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-21T22:19:12.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2200a11a0 msgr2=0x7fc2200a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2200a11a0 0x7fc2200a3650 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc22800b2e0 tx=0x7fc228005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 msgr2=0x7fc238195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 0x7fc238195d10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00d980 tx=0x7fc22c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 shutdown_connections 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2200a11a0 0x7fc2200a3650 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc238101680 0x7fc23819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc2380ffb00 0x7fc2381957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 --2- 172.21.15.66:0/2279684601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc238100cf0 0x7fc238195d10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:12.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 >> 172.21.15.66:0/2279684601 conn(0x7fc2380fb0e0 msgr2=0x7fc2381041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:12.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 shutdown_connections 2023-07-21T22:19:12.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:12.898+0000 7fc23e3ff700 1 -- 172.21.15.66:0/2279684601 wait complete. 2023-07-21T22:19:12.944 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 16 2023-07-21T22:19:12.962 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 -- 172.21.15.66:0/1483240347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e00ffb00 msgr2=0x7f77e00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 --2- 172.21.15.66:0/1483240347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e00ffb00 0x7f77e00fff10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009990 tx=0x7f77dc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 -- 172.21.15.66:0/1483240347 shutdown_connections 2023-07-21T22:19:13.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 --2- 172.21.15.66:0/1483240347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e0101680 0x7f77e0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 --2- 172.21.15.66:0/1483240347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e00ffb00 0x7f77e00fff10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 --2- 172.21.15.66:0/1483240347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e0100cf0 0x7f77e0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 -- 172.21.15.66:0/1483240347 >> 172.21.15.66:0/1483240347 conn(0x7f77e00fb0e0 msgr2=0x7f77e00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 -- 172.21.15.66:0/1483240347 shutdown_connections 2023-07-21T22:19:13.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.126+0000 7f77e7fff700 1 -- 172.21.15.66:0/1483240347 wait complete. 2023-07-21T22:19:13.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.127+0000 7f77e7fff700 1 Processor -- start 2023-07-21T22:19:13.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.127+0000 7f77e7fff700 1 -- start start 2023-07-21T22:19:13.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.127+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e00ffb00 0x7f77e00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e0100cf0 0x7f77e0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 0x7f77e0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e7fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e00735e0 con 0x7f77e0101680 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e0073750 con 0x7f77e0100cf0 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e7fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e019f8a0 con 0x7f77e00ffb00 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 0x7f77e0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e00ffb00 0x7f77e00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e00ffb00 0x7f77e00742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:41852/0 (socket says 172.21.15.66:41852) 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 0x7f77e0072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41868/0 (socket says 172.21.15.66:41868) 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e5d9b700 1 -- 172.21.15.66:0/4195169551 learned_addr learned my addr 172.21.15.66:0/4195169551 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:13.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e559a700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e0100cf0 0x7f77e0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 -- 172.21.15.66:0/4195169551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e00ffb00 msgr2=0x7f77e00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e00ffb00 0x7f77e00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 -- 172.21.15.66:0/4195169551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e0100cf0 msgr2=0x7f77e0072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e0100cf0 0x7f77e0072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.128+0000 7f77e659c700 1 -- 172.21.15.66:0/4195169551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc0095f0 con 0x7f77e0101680 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.129+0000 7f77e659c700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 0x7f77e0072e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77d800efc0 tx=0x7f77d800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:13.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.129+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8018070 con 0x7f77e0101680 2023-07-21T22:19:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.129+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d800f040 con 0x7f77e0101680 2023-07-21T22:19:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.129+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e019fb20 con 0x7f77e0101680 2023-07-21T22:19:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.129+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800cd40 con 0x7f77e0101680 2023-07-21T22:19:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.129+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e019ffe0 con 0x7f77e0101680 2023-07-21T22:19:13.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.130+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e010b320 con 0x7f77e0101680 2023-07-21T22:19:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.132+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f77d8003a00 con 0x7f77e0101680 2023-07-21T22:19:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.133+0000 7f77d6ffd700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f77cc0a11a0 0x7f77cc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.133+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f77d80cc660 con 0x7f77e0101680 2023-07-21T22:19:13.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.134+0000 7f77e5d9b700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f77cc0a11a0 0x7f77cc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.134+0000 7f77e5d9b700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f77cc0a11a0 0x7f77cc0a3650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b2e0 tx=0x7f77dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:13.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.138+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f77d8095bf0 con 0x7f77e0101680 2023-07-21T22:19:13.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.349+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77e01069e0 con 0x7f77e0101680 2023-07-21T22:19:13.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.351+0000 7f77d6ffd700 1 -- 172.21.15.66:0/4195169551 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f77d8099210 con 0x7f77e0101680 2023-07-21T22:19:13.396 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:13.396 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:13.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f77cc0a11a0 msgr2=0x7f77cc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f77cc0a11a0 0x7f77cc0a3650 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b2e0 tx=0x7f77dc005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 msgr2=0x7f77e0072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 0x7f77e0072e40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f77d800efc0 tx=0x7f77d800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 shutdown_connections 2023-07-21T22:19:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f77cc0a11a0 0x7f77cc0a3650 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f77e0101680 0x7f77e0072e40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77e00ffb00 0x7f77e00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 --2- 172.21.15.66:0/4195169551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f77e0100cf0 0x7f77e0072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 >> 172.21.15.66:0/4195169551 conn(0x7f77e00fb0e0 msgr2=0x7f77e00fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 shutdown_connections 2023-07-21T22:19:13.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.353+0000 7f77e7fff700 1 -- 172.21.15.66:0/4195169551 wait complete. 2023-07-21T22:19:13.418 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:13.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.581+0000 7f8c965ff700 1 -- 172.21.15.66:0/2827881610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 msgr2=0x7f8c90109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.581+0000 7f8c965ff700 1 --2- 172.21.15.66:0/2827881610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c90109e40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800b0a0 tx=0x7f8c8800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 -- 172.21.15.66:0/2827881610 shutdown_connections 2023-07-21T22:19:13.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 --2- 172.21.15.66:0/2827881610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90074550 0x7f8c900729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 --2- 172.21.15.66:0/2827881610 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c90109e40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 --2- 172.21.15.66:0/2827881610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c90072f70 0x7f8c900733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 -- 172.21.15.66:0/2827881610 >> 172.21.15.66:0/2827881610 conn(0x7f8c900fb0e0 msgr2=0x7f8c900fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:13.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 -- 172.21.15.66:0/2827881610 shutdown_connections 2023-07-21T22:19:13.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.582+0000 7f8c965ff700 1 -- 172.21.15.66:0/2827881610 wait complete. 2023-07-21T22:19:13.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 Processor -- start 2023-07-21T22:19:13.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 -- start start 2023-07-21T22:19:13.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90072f70 0x7f8c90197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c90074550 0x7f8c90198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c9019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9019f6c0 con 0x7f8c901079c0 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9019f800 con 0x7f8c90072f70 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c965ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9019f990 con 0x7f8c90074550 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c94b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c9019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c94b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c9019f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41896/0 (socket says 172.21.15.66:41896) 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c94b9c700 1 -- 172.21.15.66:0/3692339291 learned_addr learned my addr 172.21.15.66:0/3692339291 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:13.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c8ffff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90072f70 0x7f8c90197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.583+0000 7f8c8f7fe700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c90074550 0x7f8c90198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c94b9c700 1 -- 172.21.15.66:0/3692339291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c90074550 msgr2=0x7f8c90198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c94b9c700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c90074550 0x7f8c90198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c94b9c700 1 -- 172.21.15.66:0/3692339291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90072f70 msgr2=0x7f8c90197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c94b9c700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90072f70 0x7f8c90197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c94b9c700 1 -- 172.21.15.66:0/3692339291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88009d00 con 0x7f8c901079c0 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c94b9c700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c9019f0f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009fd0 tx=0x7f8c8800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:13.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800e040 con 0x7f8c901079c0 2023-07-21T22:19:13.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c880094a0 con 0x7f8c901079c0 2023-07-21T22:19:13.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9019fc10 con 0x7f8c901079c0 2023-07-21T22:19:13.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8801b6c0 con 0x7f8c901079c0 2023-07-21T22:19:13.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.584+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9019ffe0 con 0x7f8c901079c0 2023-07-21T22:19:13.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.586+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c90106070 con 0x7f8c901079c0 2023-07-21T22:19:13.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.587+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8c8801d470 con 0x7f8c901079c0 2023-07-21T22:19:13.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.589+0000 7f8c8d7fa700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c780a11a0 0x7f8c780a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:13.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.589+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8c880cdd60 con 0x7f8c901079c0 2023-07-21T22:19:13.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.589+0000 7f8c8ffff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c780a11a0 0x7f8c780a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:13.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.590+0000 7f8c8ffff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c780a11a0 0x7f8c780a3650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c90074920 tx=0x7f8c80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:13.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.593+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8c880972f0 con 0x7f8c901079c0 2023-07-21T22:19:13.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:13 smithi066 ceph-mon[126379]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T22:19:13.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2161366827' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-21T22:19:13.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:13 smithi066 ceph-mon[126379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:13.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2279684601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:13.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4195169551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:13.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:13 smithi149 ceph-mon[129268]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T22:19:13.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2161366827' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-21T22:19:13.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:13 smithi149 ceph-mon[129268]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:13.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2279684601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:13.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4195169551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.808+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c9002c850 con 0x7f8c901079c0 2023-07-21T22:19:13.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.810+0000 7f8c8d7fa700 1 -- 172.21.15.66:0/3692339291 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f8c8809a910 con 0x7f8c901079c0 2023-07-21T22:19:13.856 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:13.856 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:13.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c780a11a0 msgr2=0x7f8c780a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c780a11a0 0x7f8c780a3650 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8c90074920 tx=0x7f8c80009260 comp rx=0 tx=0).stop 2023-07-21T22:19:13.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 msgr2=0x7f8c9019f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:13.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c9019f0f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009fd0 tx=0x7f8c8800bce0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 shutdown_connections 2023-07-21T22:19:13.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8c780a11a0 0x7f8c780a3650 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c90072f70 0x7f8c90197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8c901079c0 0x7f8c9019f0f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.812+0000 7f8c965ff700 1 --2- 172.21.15.66:0/3692339291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8c90074550 0x7f8c90198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:13.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.813+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 >> 172.21.15.66:0/3692339291 conn(0x7f8c900fb0e0 msgr2=0x7f8c900fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:13.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.813+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 shutdown_connections 2023-07-21T22:19:13.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:13.813+0000 7f8c965ff700 1 -- 172.21.15.66:0/3692339291 wait complete. 2023-07-21T22:19:13.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:13 smithi104 ceph-mon[129021]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T22:19:13.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2161366827' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-07-21T22:19:13.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:13 smithi104 ceph-mon[129021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:13.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2279684601' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:13.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4195169551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:13.897 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:19:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 -- 172.21.15.66:0/3110739441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 msgr2=0x7f9ed00fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/3110739441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed00fff10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009990 tx=0x7f9ecc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 -- 172.21.15.66:0/3110739441 shutdown_connections 2023-07-21T22:19:14.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/3110739441 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0101680 0x7f9ed0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/3110739441 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed00fff10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/3110739441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 -- 172.21.15.66:0/3110739441 >> 172.21.15.66:0/3110739441 conn(0x7f9ed00fb0e0 msgr2=0x7f9ed00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 -- 172.21.15.66:0/3110739441 shutdown_connections 2023-07-21T22:19:14.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.059+0000 7f9ed81ff700 1 -- 172.21.15.66:0/3110739441 wait complete. 2023-07-21T22:19:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.060+0000 7f9ed81ff700 1 Processor -- start 2023-07-21T22:19:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.060+0000 7f9ed81ff700 1 -- start start 2023-07-21T22:19:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed81ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed81ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed81ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0101680 0x7f9ed019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed81ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019f5a0 con 0x7f9ed00ffb00 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed81ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019f6e0 con 0x7f9ed0101680 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed81ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed019f870 con 0x7f9ed0100cf0 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed5f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed5f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41900/0 (socket says 172.21.15.66:41900) 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed5f9b700 1 -- 172.21.15.66:0/2154201080 learned_addr learned my addr 172.21.15.66:0/2154201080 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:14.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:41904/0 (socket says 172.21.15.66:41904) 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed679c700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0101680 0x7f9ed019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 -- 172.21.15.66:0/2154201080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0101680 msgr2=0x7f9ed019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0101680 0x7f9ed019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 -- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 msgr2=0x7f9ed0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed579a700 1 -- 172.21.15.66:0/2154201080 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec0009520 con 0x7f9ed0100cf0 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed679c700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0101680 0x7f9ed019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:14.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.061+0000 7f9ed5f9b700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.062+0000 7f9ed579a700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0197f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9f0 tx=0x7f9ec000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.062+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000cb70 con 0x7f9ed0100cf0 2023-07-21T22:19:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.062+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec8009c40 con 0x7f9ed0100cf0 2023-07-21T22:19:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.062+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ec00044e0 con 0x7f9ed0100cf0 2023-07-21T22:19:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.062+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0003d50 con 0x7f9ed0100cf0 2023-07-21T22:19:14.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.062+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc0095f0 con 0x7f9ed0100cf0 2023-07-21T22:19:14.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.063+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed010b2d0 con 0x7f9ed0100cf0 2023-07-21T22:19:14.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.065+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f9ec0003ed0 con 0x7f9ed0100cf0 2023-07-21T22:19:14.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.067+0000 7f9ec6ffd700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9ebc0a11a0 0x7f9ebc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.067+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9ec0014070 con 0x7f9ed0100cf0 2023-07-21T22:19:14.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.067+0000 7f9ed5f9b700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9ebc0a11a0 0x7f9ebc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.068+0000 7f9ed5f9b700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9ebc0a11a0 0x7f9ebc0a3650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc006010 tx=0x7f9ecc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:14.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.069+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ec00965e0 con 0x7f9ed0100cf0 2023-07-21T22:19:14.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.325+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ed010b4c0 con 0x7f9ed0100cf0 2023-07-21T22:19:14.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.326+0000 7f9ec6ffd700 1 -- 172.21.15.66:0/2154201080 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+10759 (secure 0 0 0) 0x7f9ec0099c00 con 0x7f9ed0100cf0 2023-07-21T22:19:14.371 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:14.372 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24530,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:12.386591+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24572},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-07-21T22:19:14.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9ebc0a11a0 msgr2=0x7f9ebc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9ebc0a11a0 0x7f9ebc0a3650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc006010 tx=0x7f9ecc005960 comp rx=0 tx=0).stop 2023-07-21T22:19:14.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 msgr2=0x7f9ed0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0197f60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9f0 tx=0x7f9ec000ed00 comp rx=0 tx=0).stop 2023-07-21T22:19:14.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 shutdown_connections 2023-07-21T22:19:14.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9ebc0a11a0 0x7f9ebc0a3650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ed0101680 0x7f9ed019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ed00ffb00 0x7f9ed0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 --2- 172.21.15.66:0/2154201080 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed0100cf0 0x7f9ed0197f60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 >> 172.21.15.66:0/2154201080 conn(0x7f9ed00fb0e0 msgr2=0x7f9ed00fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:14.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 shutdown_connections 2023-07-21T22:19:14.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.329+0000 7f9ed81ff700 1 -- 172.21.15.66:0/2154201080 wait complete. 2023-07-21T22:19:14.376 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 16 2023-07-21T22:19:14.393 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:14.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 -- 172.21.15.66:0/1640622510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 msgr2=0x7f92b0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 --2- 172.21.15.66:0/1640622510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0101140 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009990 tx=0x7f92a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 -- 172.21.15.66:0/1640622510 shutdown_connections 2023-07-21T22:19:14.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 --2- 172.21.15.66:0/1640622510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92b0101680 0x7f92b0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 --2- 172.21.15.66:0/1640622510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b00ffb00 0x7f92b00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 --2- 172.21.15.66:0/1640622510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0101140 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 -- 172.21.15.66:0/1640622510 >> 172.21.15.66:0/1640622510 conn(0x7f92b00fb0e0 msgr2=0x7f92b00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:14.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 -- 172.21.15.66:0/1640622510 shutdown_connections 2023-07-21T22:19:14.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.559+0000 7f92b79ff700 1 -- 172.21.15.66:0/1640622510 wait complete. 2023-07-21T22:19:14.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.560+0000 7f92b79ff700 1 Processor -- start 2023-07-21T22:19:14.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 -- start start 2023-07-21T22:19:14.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b00ffb00 0x7f92b00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92b0101680 0x7f92b0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b00735e0 con 0x7f92b0100cf0 2023-07-21T22:19:14.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b0073750 con 0x7f92b0101680 2023-07-21T22:19:14.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.561+0000 7f92b79ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b019f870 con 0x7f92b00ffb00 2023-07-21T22:19:14.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b00ffb00 0x7f92b00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41912/0 (socket says 172.21.15.66:41912) 2023-07-21T22:19:14.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 -- 172.21.15.66:0/3678469626 learned_addr learned my addr 172.21.15.66:0/3678469626 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:14.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b5f9c700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92b0101680 0x7f92b0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 -- 172.21.15.66:0/3678469626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b00ffb00 msgr2=0x7f92b00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b00ffb00 0x7f92b00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 -- 172.21.15.66:0/3678469626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92b0101680 msgr2=0x7f92b0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92b0101680 0x7f92b0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 -- 172.21.15.66:0/3678469626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a00095f0 con 0x7f92b0100cf0 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92b4f9a700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0072900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b620 tx=0x7f92a0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a001d070 con 0x7f92b0100cf0 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92a0022470 con 0x7f92b0100cf0 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.562+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a0017650 con 0x7f92b0100cf0 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.563+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b019faf0 con 0x7f92b0100cf0 2023-07-21T22:19:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.563+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b019fec0 con 0x7f92b0100cf0 2023-07-21T22:19:14.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.564+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b0107050 con 0x7f92b0100cf0 2023-07-21T22:19:14.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.566+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f92a00225e0 con 0x7f92b0100cf0 2023-07-21T22:19:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.567+0000 7f92a67fc700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f929c0a1220 0x7f929c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.567+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f92a00cdfe0 con 0x7f92b0100cf0 2023-07-21T22:19:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.568+0000 7f92b579b700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f929c0a1220 0x7f929c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.568+0000 7f92b579b700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f929c0a1220 0x7f929c0a36d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92b0100b50 tx=0x7f92ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.571+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f92a0097590 con 0x7f92b0100cf0 2023-07-21T22:19:14.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3692339291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:14.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:14 smithi066 ceph-mon[126379]: pgmap v278: 129 pgs: 11 creating+peering, 112 active+clean, 6 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 850 B/s wr, 6 op/s 2023-07-21T22:19:14.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2154201080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:14.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3692339291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:14.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:14 smithi149 ceph-mon[129268]: pgmap v278: 129 pgs: 11 creating+peering, 112 active+clean, 6 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 850 B/s wr, 6 op/s 2023-07-21T22:19:14.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2154201080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:14.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.781+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92b010b320 con 0x7f92b0100cf0 2023-07-21T22:19:14.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.783+0000 7f92a67fc700 1 -- 172.21.15.66:0/3678469626 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f92a0095e60 con 0x7f92b0100cf0 2023-07-21T22:19:14.828 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:14.828 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:14.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f929c0a1220 msgr2=0x7f929c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f929c0a1220 0x7f929c0a36d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f92b0100b50 tx=0x7f92ac009260 comp rx=0 tx=0).stop 2023-07-21T22:19:14.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 msgr2=0x7f92b0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:14.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0072900 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f92a000b620 tx=0x7f92a0005740 comp rx=0 tx=0).stop 2023-07-21T22:19:14.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 shutdown_connections 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f929c0a1220 0x7f929c0a36d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92b0101680 0x7f92b0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b00ffb00 0x7f92b00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 --2- 172.21.15.66:0/3678469626 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92b0100cf0 0x7f92b0072900 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 >> 172.21.15.66:0/3678469626 conn(0x7f92b00fb0e0 msgr2=0x7f92b0104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 shutdown_connections 2023-07-21T22:19:14.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:14.785+0000 7f92b79ff700 1 -- 172.21.15.66:0/3678469626 wait complete. 2023-07-21T22:19:14.850 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-21T22:19:06.899289+0000', 'modified': '2023-07-21T22:19:12.386591+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24572}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/1299576617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1299576617}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1299576617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-07-21T22:19:14.851 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-21T22:19:14.851 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:19:14.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3692339291' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:14.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:14 smithi104 ceph-mon[129021]: pgmap v278: 129 pgs: 11 creating+peering, 112 active+clean, 6 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 850 B/s wr, 6 op/s 2023-07-21T22:19:14.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2154201080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 -- 172.21.15.66:0/3317992423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8800ffae0 msgr2=0x7fd8800ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 --2- 172.21.15.66:0/3317992423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8800ffae0 0x7fd8800ffef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd870009990 tx=0x7fd870009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 -- 172.21.15.66:0/3317992423 shutdown_connections 2023-07-21T22:19:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 --2- 172.21.15.66:0/3317992423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd8800ffae0 0x7fd8800ffef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 --2- 172.21.15.66:0/3317992423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd880101660 0x7fd880107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 --2- 172.21.15.66:0/3317992423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880100cd0 0x7fd880101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 -- 172.21.15.66:0/3317992423 >> 172.21.15.66:0/3317992423 conn(0x7fd8800fb0e0 msgr2=0x7fd8800fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 -- 172.21.15.66:0/3317992423 shutdown_connections 2023-07-21T22:19:15.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.014+0000 7fd885fff700 1 -- 172.21.15.66:0/3317992423 wait complete. 2023-07-21T22:19:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.015+0000 7fd885fff700 1 Processor -- start 2023-07-21T22:19:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.015+0000 7fd885fff700 1 -- start start 2023-07-21T22:19:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd885fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 0x7fd880197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd885fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880100cd0 0x7fd880198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd885fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd880101660 0x7fd88019f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd885fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019f640 con 0x7fd8800ffae0 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd885fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019f780 con 0x7fd880101660 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd885fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88019f910 con 0x7fd880100cd0 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 0x7fd880197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 0x7fd880197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41932/0 (socket says 172.21.15.66:41932) 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1095078015 learned_addr learned my addr 172.21.15.66:0/1095078015 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87ffff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd880101660 0x7fd88019f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87effd700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880100cd0 0x7fd880198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1095078015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880100cd0 msgr2=0x7fd880198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880100cd0 0x7fd880198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1095078015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd880101660 msgr2=0x7fd88019f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd880101660 0x7fd88019f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1095078015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8700095f0 con 0x7fd8800ffae0 2023-07-21T22:19:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.016+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 0x7fd880197ac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd870004730 tx=0x7fd870004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.017+0000 7fd87cff9700 1 -- 172.21.15.66:0/1095078015 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87001d070 con 0x7fd8800ffae0 2023-07-21T22:19:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.017+0000 7fd87cff9700 1 -- 172.21.15.66:0/1095078015 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd870021410 con 0x7fd8800ffae0 2023-07-21T22:19:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.017+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88019fb90 con 0x7fd8800ffae0 2023-07-21T22:19:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.017+0000 7fd87cff9700 1 -- 172.21.15.66:0/1095078015 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870017650 con 0x7fd8800ffae0 2023-07-21T22:19:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.017+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8801a0010 con 0x7fd8800ffae0 2023-07-21T22:19:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.020+0000 7fd87cff9700 1 -- 172.21.15.66:0/1095078015 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd870021580 con 0x7fd8800ffae0 2023-07-21T22:19:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.021+0000 7fd8999e0b80 1 Processor -- start 2023-07-21T22:19:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.021+0000 7fd87cff9700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8680a11d0 0x7fd8680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 -- start start 2023-07-21T22:19:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87cff9700 1 -- 172.21.15.66:0/1095078015 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd8700cdec0 con 0x7fd8800ffae0 2023-07-21T22:19:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87ffff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8680a11d0 0x7fd8680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b257dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257e150 0x5623b257c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257c9c0 0x5623b257ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b2481c60 con 0x5623b257d7e0 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b24d49f0 con 0x5623b257c9c0 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b24cde80 con 0x5623b257e150 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b257dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b257dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41940/0 (socket says 172.21.15.66:41940) 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1591169814 learned_addr learned my addr 172.21.15.66:0/1591169814 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87effd700 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257e150 0x5623b257c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.022+0000 7fd87ffff700 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257c9c0 0x5623b257ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1591169814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257e150 msgr2=0x5623b257c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257e150 0x5623b257c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1591169814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257c9c0 msgr2=0x5623b257ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257c9c0 0x5623b257ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87f7fe700 1 -- 172.21.15.66:0/1591169814 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623b22fbd90 con 0x5623b257d7e0 2023-07-21T22:19:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87ffff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8680a11d0 0x7fd8680a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd87800f590 tx=0x7fd87800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b257dbf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd87009ae10 tx=0x7fd87009acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5a5810432e5464 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd865ffb700 1 -- 172.21.15.66:0/1591169814 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87009f140 con 0x5623b257d7e0 2023-07-21T22:19:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd865ffb700 1 -- 172.21.15.66:0/1591169814 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8700a34d0 con 0x5623b257d7e0 2023-07-21T22:19:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.023+0000 7fd865ffb700 1 -- 172.21.15.66:0/1591169814 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87009a860 con 0x5623b257d7e0 2023-07-21T22:19:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 -- 172.21.15.66:0/1591169814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 msgr2=0x5623b257dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b257dbf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd87009ae10 tx=0x7fd87009acd0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 -- 172.21.15.66:0/1591169814 shutdown_connections 2023-07-21T22:19:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257c9c0 0x5623b257ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b257dbf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/1591169814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257e150 0x5623b257c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 -- 172.21.15.66:0/1591169814 >> 172.21.15.66:0/1591169814 conn(0x5623b2571ea0 msgr2=0x5623b2577310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 -- 172.21.15.66:0/1591169814 shutdown_connections 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.024+0000 7fd8999e0b80 1 -- 172.21.15.66:0/1591169814 wait complete. 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 -- start start 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 Processor -- start 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2585d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2586240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b258d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b246da70 con 0x5623b257d7e0 2023-07-21T22:19:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b246d770 con 0x5623b257e150 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b23518d0 con 0x5623b257c9c0 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.025+0000 7fd8999e0b80 10 client.0 _ll_register_callbacks cb 0x5623b246b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2586240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2586240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41944/0 (socket says 172.21.15.66:41944) 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 -- 172.21.15.66:0/3619033592 learned_addr learned my addr 172.21.15.66:0/3619033592 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2585d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87ffff700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b258d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 -- 172.21.15.66:0/3619033592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 msgr2=0x5623b2585d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2585d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 -- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 msgr2=0x5623b258d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b258d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 -- 172.21.15.66:0/3619033592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8801a0010 con 0x5623b257d7e0 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd87effd700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2586240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd874012860 tx=0x7fd874012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81a98af73d71a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.026+0000 7fd85effd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:19:15.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd85effd700 1 -- 172.21.15.66:0/3619033592 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874017070 con 0x5623b257d7e0 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623b24274a0 con 0x5623b257d7e0 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd85effd700 1 -- 172.21.15.66:0/3619033592 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd87400f540 con 0x5623b257d7e0 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd85effd700 1 -- 172.21.15.66:0/3619033592 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8740122e0 con 0x5623b257d7e0 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 10 client.14622 fetch_fsmap learned FSMap version 16 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x5623b22e4c10 con 0x5623b257d7e0 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd85effd700 1 -- 172.21.15.66:0/3619033592 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623b24274a0 con 0x5623b257d7e0 2023-07-21T22:19:15.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd85effd700 1 -- 172.21.15.66:0/3619033592 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 16) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd8740108a0 con 0x5623b257d7e0 2023-07-21T22:19:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 10 client.14622 fetch_fsmap finished waiting for FSMap version 16 2023-07-21T22:19:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 10 client.14622 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:19:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 20 client.14622 populate_metadata read hostname 'smithi066' 2023-07-21T22:19:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b2591080 0x5623b2590430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.027+0000 7fd8999e0b80 4 client.14622 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:19:15.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.028+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623b24c6e60 con 0x5623b2591080 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.028+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b2591080 0x5623b2590430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.028+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b2591080 0x5623b2590430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b5f400b074f7817 server_cookie=dad7cb75af81a314 in_seq=0 out_seq=0 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.028+0000 7fd85effd700 10 client.14622 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.029+0000 7fd85effd700 1 -- 172.21.15.66:0/3619033592 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5623b24c6e60 con 0x5623b2591080 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.029+0000 7fd85effd700 10 client.14622 handle_command_reply: tid=0 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.029+0000 7fd8999e0b80 1 client.14622 shutdown 2023-07-21T22:19:15.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.029+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 msgr2=0x5623b2586240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.029+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2586240 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd874012860 tx=0x7fd874012b70 comp rx=0 tx=0).stop 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 shutdown_connections 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b258d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b2591080 0x5623b2590430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2585d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/3619033592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2586240 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 >> 172.21.15.66:0/3619033592 conn(0x5623b2571ea0 msgr2=0x5623b25769b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:15.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd85effd700 0 client.14622 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:19:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 shutdown_connections 2023-07-21T22:19:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 1 -- 172.21.15.66:0/3619033592 wait complete. 2023-07-21T22:19:15.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.030+0000 7fd8999e0b80 20 client.14622 trim_cache size 0 max 16384 2023-07-21T22:19:15.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.048+0000 7fd8999e0b80 1 Processor -- start 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 -- start start 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 0x5623b2599dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257d7e0 0x5623b258d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b256f050 con 0x5623b257c9c0 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b24c6e60 con 0x5623b257e150 2023-07-21T22:19:15.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b24274a0 con 0x5623b257d7e0 2023-07-21T22:19:15.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 0x5623b2599dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 0x5623b2599dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41954/0 (socket says 172.21.15.66:41954) 2023-07-21T22:19:15.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 -- 172.21.15.66:0/2685524004 learned_addr learned my addr 172.21.15.66:0/2685524004 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:15.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87ffff700 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87effd700 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257d7e0 0x5623b258d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 -- 172.21.15.66:0/2685524004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257d7e0 msgr2=0x5623b258d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257d7e0 0x5623b258d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 -- 172.21.15.66:0/2685524004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 msgr2=0x5623b25769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.049+0000 7fd87f7fe700 1 -- 172.21.15.66:0/2685524004 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623b2322200 con 0x5623b257c9c0 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.050+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 0x5623b2599dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd870020000 tx=0x7fd870005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d9a05c980a0d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:15.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.050+0000 7fd8667fc700 1 -- 172.21.15.66:0/2685524004 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87009b610 con 0x5623b257c9c0 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.050+0000 7fd8667fc700 1 -- 172.21.15.66:0/2685524004 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8700994b0 con 0x5623b257c9c0 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.050+0000 7fd8667fc700 1 -- 172.21.15.66:0/2685524004 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87000fb00 con 0x5623b257c9c0 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.050+0000 7fd8999e0b80 1 -- 172.21.15.66:0/2685524004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 msgr2=0x5623b2599dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.050+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 0x5623b2599dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd870020000 tx=0x7fd870005a90 comp rx=0 tx=0).stop 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 -- 172.21.15.66:0/2685524004 shutdown_connections 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257c9c0 0x5623b2599dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/2685524004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257d7e0 0x5623b258d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 -- 172.21.15.66:0/2685524004 >> 172.21.15.66:0/2685524004 conn(0x5623b2571ea0 msgr2=0x5623b25772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:15.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 -- 172.21.15.66:0/2685524004 shutdown_connections 2023-07-21T22:19:15.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 -- 172.21.15.66:0/2685524004 wait complete. 2023-07-21T22:19:15.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 -- start start 2023-07-21T22:19:15.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.051+0000 7fd8999e0b80 1 Processor -- start 2023-07-21T22:19:15.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2587e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2588370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b258d3a0 con 0x5623b257d7e0 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b258d510 con 0x5623b257e150 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623b258d6d0 con 0x5623b257c9c0 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd8999e0b80 10 client.0 _ll_register_callbacks cb 0x7fd874009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2588370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2588370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41960/0 (socket says 172.21.15.66:41960) 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 -- 172.21.15.66:0/4232337809 learned_addr learned my addr 172.21.15.66:0/4232337809 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87ffff700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2587e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 -- 172.21.15.66:0/4232337809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 msgr2=0x5623b2587e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2587e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 -- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 msgr2=0x5623b25e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.052+0000 7fd87effd700 1 -- 172.21.15.66:0/4232337809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623b22fbd90 con 0x5623b257d7e0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd87effd700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2588370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd874010d00 tx=0x7fd8740096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f95cb0f2840b7e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd85effd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd85effd700 1 -- 172.21.15.66:0/4232337809 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874017040 con 0x5623b257d7e0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8700a34d0 con 0x5623b257d7e0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd85effd700 1 -- 172.21.15.66:0/4232337809 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd874000e40 con 0x5623b257d7e0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd85effd700 1 -- 172.21.15.66:0/4232337809 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd874005ce0 con 0x5623b257d7e0 2023-07-21T22:19:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd8999e0b80 10 client.14628 fetch_fsmap learned FSMap version 16 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) v3 -- 0x5623b22fa260 con 0x5623b257d7e0 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.053+0000 7fd85effd700 1 -- 172.21.15.66:0/4232337809 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8700a34d0 con 0x5623b257d7e0 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd85effd700 1 -- 172.21.15.66:0/4232337809 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 16) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd874003bd0 con 0x5623b257d7e0 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd8999e0b80 10 client.14628 fetch_fsmap finished waiting for FSMap version 16 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd8999e0b80 10 client.14628 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd8999e0b80 20 client.14628 populate_metadata read hostname 'smithi066' 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b25e9fa0 0x5623b25e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd8999e0b80 4 client.14628 mds_command: new command op to 24572 tid=0[{"prefix": "status"}] 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623b2489000 con 0x5623b25e9fa0 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.054+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b25e9fa0 0x5623b25e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd87f7fe700 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b25e9fa0 0x5623b25e6910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7834133b652450e server_cookie=77eaffe6eae2f106 in_seq=0 out_seq=0 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd85effd700 10 client.14628 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd85effd700 1 -- 172.21.15.66:0/4232337809 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5623b2489000 con 0x5623b25e9fa0 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd85effd700 10 client.14628 handle_command_reply: tid=0 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 client.14628 shutdown 2023-07-21T22:19:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 msgr2=0x5623b2588370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2588370 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd874010d00 tx=0x7fd8740096d0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 shutdown_connections 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623b257e150 0x5623b25e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623b25e9fa0 0x5623b25e6910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623b257c9c0 0x5623b2587e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 --2- 172.21.15.66:0/4232337809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623b257d7e0 0x5623b2588370 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd85effd700 0 client.14628 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:19:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 >> 172.21.15.66:0/4232337809 conn(0x5623b2571ea0 msgr2=0x5623b2590430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 shutdown_connections 2023-07-21T22:19:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 1 -- 172.21.15.66:0/4232337809 wait complete. 2023-07-21T22:19:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.055+0000 7fd8999e0b80 20 client.14628 trim_cache size 0 max 16384 2023-07-21T22:19:15.108 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:19:15.108 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24572, 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 7.6548446959999996, 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 16, 2023-07-21T22:19:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 89, 2023-07-21T22:19:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:19:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 14.280942807000001 2023-07-21T22:19:15.110 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8680a11d0 msgr2=0x7fd8680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8680a11d0 0x7fd8680a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd87800f590 tx=0x7fd87800b040 comp rx=0 tx=0).stop 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 msgr2=0x7fd880197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 0x7fd880197ac0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd870004730 tx=0x7fd870004760 comp rx=0 tx=0).stop 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 shutdown_connections 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd8680a11d0 0x7fd8680a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd880101660 0x7fd88019f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd8800ffae0 0x7fd880197ac0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 --2- 172.21.15.66:0/1095078015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd880100cd0 0x7fd880198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 >> 172.21.15.66:0/1095078015 conn(0x7fd8800fb0e0 msgr2=0x7fd880106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 shutdown_connections 2023-07-21T22:19:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:15.056+0000 7fd885fff700 1 -- 172.21.15.66:0/1095078015 wait complete. 2023-07-21T22:19:15.121 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-21T22:19:15.123 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:15.124 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-21T22:19:15.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3678469626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:15.859 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:15.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3678469626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:15.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3678469626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:16.490 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:16 smithi066 ceph-mon[126379]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2023-07-21T22:19:16.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:16 smithi149 ceph-mon[129268]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2023-07-21T22:19:16.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:16 smithi104 ceph-mon[129021]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2023-07-21T22:19:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 -- 172.21.15.66:0/3022339600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 msgr2=0x7fb1041001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 --2- 172.21.15.66:0/3022339600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb1041001d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40099c0 tx=0x7fb0f4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 -- 172.21.15.66:0/3022339600 shutdown_connections 2023-07-21T22:19:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 --2- 172.21.15.66:0/3022339600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 0x7fb10410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 --2- 172.21.15.66:0/3022339600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb1041001d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 --2- 172.21.15.66:0/3022339600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 0x7fb104104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 -- 172.21.15.66:0/3022339600 >> 172.21.15.66:0/3022339600 conn(0x7fb1040fb660 msgr2=0x7fb1040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 -- 172.21.15.66:0/3022339600 shutdown_connections 2023-07-21T22:19:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.371+0000 7fb109001700 1 -- 172.21.15.66:0/3022339600 wait complete. 2023-07-21T22:19:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.372+0000 7fb109001700 1 Processor -- start 2023-07-21T22:19:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.372+0000 7fb109001700 1 -- start start 2023-07-21T22:19:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb109001700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb104197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb109001700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 0x7fb1041983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb109001700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 0x7fb10419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb109001700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10419ce70 con 0x7fb1040ffe00 2023-07-21T22:19:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb109001700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10419cfe0 con 0x7fb104104ea0 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb109001700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10419d1a0 con 0x7fb1041007a0 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb102d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb104197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb102d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb104197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41970/0 (socket says 172.21.15.66:41970) 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb102d9d700 1 -- 172.21.15.66:0/3617983015 learned_addr learned my addr 172.21.15.66:0/3617983015 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb10259c700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 0x7fb1041983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.373+0000 7fb10359e700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 0x7fb10419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb102d9d700 1 -- 172.21.15.66:0/3617983015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 msgr2=0x7fb1041983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb102d9d700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 0x7fb1041983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb102d9d700 1 -- 172.21.15.66:0/3617983015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 msgr2=0x7fb10419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb102d9d700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 0x7fb10419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb102d9d700 1 -- 172.21.15.66:0/3617983015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f40095f0 con 0x7fb1040ffe00 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb10259c700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 0x7fb1041983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb10359e700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 0x7fb10419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:17.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb102d9d700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb104197eb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400b580 tx=0x7fb0f400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f401d070 con 0x7fb1040ffe00 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10419d420 con 0x7fb1040ffe00 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0f4005050 con 0x7fb1040ffe00 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4004a40 con 0x7fb1040ffe00 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.374+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1041a4ce0 con 0x7fb1040ffe00 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.376+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10404e210 con 0x7fb1040ffe00 2023-07-21T22:19:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.378+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb0f40051c0 con 0x7fb1040ffe00 2023-07-21T22:19:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.379+0000 7fb0ebfff700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ec0a1150 0x7fb0ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.379+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0f40cd8c0 con 0x7fb1040ffe00 2023-07-21T22:19:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.379+0000 7fb10259c700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ec0a1150 0x7fb0ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.380+0000 7fb10259c700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ec0a1150 0x7fb0ec0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1041994d0 tx=0x7fb0f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:17.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.383+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb0f4096e50 con 0x7fb1040ffe00 2023-07-21T22:19:17.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.634+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb10419d5b0 con 0x7fb1040ffe00 2023-07-21T22:19:17.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.634+0000 7fb0ebfff700 1 -- 172.21.15.66:0/3617983015 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb0f409a470 con 0x7fb1040ffe00 2023-07-21T22:19:17.680 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-21T22:19:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ec0a1150 msgr2=0x7fb0ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ec0a1150 0x7fb0ec0a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1041994d0 tx=0x7fb0f8009260 comp rx=0 tx=0).stop 2023-07-21T22:19:17.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 msgr2=0x7fb104197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb104197eb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400b580 tx=0x7fb0f400bfd0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 shutdown_connections 2023-07-21T22:19:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ec0a1150 0x7fb0ec0a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb104104ea0 0x7fb10419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb1040ffe00 0x7fb104197eb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 --2- 172.21.15.66:0/3617983015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb1041007a0 0x7fb1041983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.636+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 >> 172.21.15.66:0/3617983015 conn(0x7fb1040fb660 msgr2=0x7fb1040fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.637+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 shutdown_connections 2023-07-21T22:19:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:17.637+0000 7fb109001700 1 -- 172.21.15.66:0/3617983015 wait complete. 2023-07-21T22:19:17.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:17 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3617983015' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:19:18.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:17 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3617983015' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:19:18.140 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:17 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3617983015' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:19:18.143 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-07-21T22:19:18.144 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-21T22:19:18.144 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2023-07-21T22:19:18.144 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-07-21T22:19:18.144 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:18.145 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-07-21T22:19:18.145 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-21T22:19:18.145 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-21T22:19:18.145 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-07-21T22:19:18.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:18.146 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2023-07-21T22:19:18.192 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-21T22:19:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-21T22:19:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2023-07-21T22:19:18.193 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-21T22:19:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41342sec preferred_lft 41342sec 2023-07-21T22:19:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::1007:d553:b784:5869/64 scope link noprefixroute 2023-07-21T22:19:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:19:18.196 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-21T22:19:18.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:18.197 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:19:18.197 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:19:18.197 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2023-07-21T22:19:18.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2023-07-21T22:19:18.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2023-07-21T22:19:18.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-21T22:19:18.198 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:19:18.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:18.698 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-21T22:19:18.775 INFO:teuthology.orchestra.run.smithi066.stdout:1 2023-07-21T22:19:18.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:18.780 DEBUG:teuthology.orchestra.run.smithi066:> route 2023-07-21T22:19:18.843 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2023-07-21T22:19:18.843 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-21T22:19:18.844 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-21T22:19:18.844 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-21T22:19:18.844 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-21T22:19:18.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:18.846 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:19:18.846 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:19:18.846 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-21T22:19:18.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-21T22:19:18.847 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-21T22:19:18.847 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:19:18.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:18 smithi066 ceph-mon[126379]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 393 B/s wr, 1 op/s 2023-07-21T22:19:19.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:18 smithi104 ceph-mon[129021]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 393 B/s wr, 1 op/s 2023-07-21T22:19:19.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:18 smithi149 ceph-mon[129268]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 393 B/s wr, 1 op/s 2023-07-21T22:19:19.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:19.572 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-07-21T22:19:19.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:19.619 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2023-07-21T22:19:19.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:19.704 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:19:19.704 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:19:19.705 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-21T22:19:19.705 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-07-21T22:19:19.705 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:19:19.897 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-07-21T22:19:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:19.898 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:19:19.898 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:19:19.898 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-07-21T22:19:19.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-21T22:19:19.899 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-07-21T22:19:19.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-07-21T22:19:19.900 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-07-21T22:19:19.900 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:19:20.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:20.189 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:19:20.189 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:19:20.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2023-07-21T22:19:20.189 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2023-07-21T22:19:20.190 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:19:20.371 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-07-21T22:19:20.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:19:20.372 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:20.417 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-07-21T22:19:20.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:19:20.418 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:20.501 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2023-07-21T22:19:20.614 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-21T22:19:20.614 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-21T22:19:20.685 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-21T22:19:20.688 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-21T22:19:20.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:20.689 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2023-07-21T22:19:20.734 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-21T22:19:20.734 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-07-21T22:19:20.778 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2023-07-21T22:19:20.853 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-21T22:19:20.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-21T22:19:20.926 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-21T22:19:20.928 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-21T22:19:20.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:20.929 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2023-07-21T22:19:20.980 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:2023-07-21T22:19:20.936+0000 7f22c1dae5c0 -1 init, newargv = 0x55ed2cd11cf0 newargc=15 2023-07-21T22:19:20.981 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[163000]: starting ceph client 2023-07-21T22:19:20.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[163000]: starting fuse 2023-07-21T22:19:21.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:21 smithi104 ceph-mon[129021]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 232 B/s wr, 0 op/s 2023-07-21T22:19:21.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:21 smithi066 ceph-mon[126379]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 232 B/s wr, 0 op/s 2023-07-21T22:19:21.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:21 smithi149 ceph-mon[129268]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 232 B/s wr, 0 op/s 2023-07-21T22:19:22.077 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe fuse 2023-07-21T22:19:22.159 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-21T22:19:22.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-21T22:19:22.232 INFO:teuthology.orchestra.run.smithi066.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-21T22:19:22.235 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-21T22:19:22.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:22.237 DEBUG:teuthology.orchestra.run.smithi066:> ls /sys/fs/fuse/connections 2023-07-21T22:19:22.282 INFO:teuthology.orchestra.run.smithi066.stdout:121 2023-07-21T22:19:22.282 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-21T22:19:22.283 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-21T22:19:22.283 DEBUG:teuthology.orchestra.run.smithi066:> import glob 2023-07-21T22:19:22.283 DEBUG:teuthology.orchestra.run.smithi066:> import re 2023-07-21T22:19:22.284 DEBUG:teuthology.orchestra.run.smithi066:> import os 2023-07-21T22:19:22.284 DEBUG:teuthology.orchestra.run.smithi066:> import subprocess 2023-07-21T22:19:22.284 DEBUG:teuthology.orchestra.run.smithi066:> 2023-07-21T22:19:22.285 DEBUG:teuthology.orchestra.run.smithi066:> def _find_admin_socket(client_name): 2023-07-21T22:19:22.285 DEBUG:teuthology.orchestra.run.smithi066:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-07-21T22:19:22.285 DEBUG:teuthology.orchestra.run.smithi066:> files = glob.glob(asok_path) 2023-07-21T22:19:22.285 DEBUG:teuthology.orchestra.run.smithi066:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-07-21T22:19:22.286 DEBUG:teuthology.orchestra.run.smithi066:> 2023-07-21T22:19:22.286 DEBUG:teuthology.orchestra.run.smithi066:> # Given a non-glob path, it better be there 2023-07-21T22:19:22.286 DEBUG:teuthology.orchestra.run.smithi066:> if "*" not in asok_path: 2023-07-21T22:19:22.286 DEBUG:teuthology.orchestra.run.smithi066:> assert(len(files) == 1) 2023-07-21T22:19:22.287 DEBUG:teuthology.orchestra.run.smithi066:> return files[0] 2023-07-21T22:19:22.287 DEBUG:teuthology.orchestra.run.smithi066:> 2023-07-21T22:19:22.288 DEBUG:teuthology.orchestra.run.smithi066:> for f in files: 2023-07-21T22:19:22.288 DEBUG:teuthology.orchestra.run.smithi066:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-21T22:19:22.288 DEBUG:teuthology.orchestra.run.smithi066:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-21T22:19:22.289 DEBUG:teuthology.orchestra.run.smithi066:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-21T22:19:22.289 DEBUG:teuthology.orchestra.run.smithi066:> contents = proc_f.read() 2023-07-21T22:19:22.289 DEBUG:teuthology.orchestra.run.smithi066:> if mountpoint in contents: 2023-07-21T22:19:22.289 DEBUG:teuthology.orchestra.run.smithi066:> return f 2023-07-21T22:19:22.290 DEBUG:teuthology.orchestra.run.smithi066:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-21T22:19:22.290 DEBUG:teuthology.orchestra.run.smithi066:> 2023-07-21T22:19:22.290 DEBUG:teuthology.orchestra.run.smithi066:> print(_find_admin_socket("client.admin")) 2023-07-21T22:19:22.290 DEBUG:teuthology.orchestra.run.smithi066:> ' 2023-07-21T22:19:22.473 INFO:teuthology.orchestra.run.smithi066.stdout:/var/run/ceph/ceph-client.admin.163000.asok 2023-07-21T22:19:22.649 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.163000.asok 2023-07-21T22:19:22.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:22.650 DEBUG:teuthology.orchestra.run.smithi066:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.163000.asok status 2023-07-21T22:19:22.883 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "metadata": { 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_sha1": "bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea", 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 18.0.0-4735-gbffe049d (bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea) reef (dev)", 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "entity_id": "admin", 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": "smithi066", 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-07-21T22:19:22.884 INFO:teuthology.orchestra.run.smithi066.stdout: "pid": "163000", 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "root": "/" 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_count": 0, 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "dentry_pinned_count": 0, 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24698, 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "inst": { 2023-07-21T22:19:22.885 INFO:teuthology.orchestra.run.smithi066.stdout: "name": { 2023-07-21T22:19:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "client", 2023-07-21T22:19:22.886 INFO:teuthology.orchestra.run.smithi066.stdout: "num": 24698 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 303713501 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:19:22.887 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": { 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 303713501 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "inst_str": "client.24698 172.21.15.66:0/303713501", 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "addr_str": "172.21.15.66:0/303713501", 2023-07-21T22:19:22.888 INFO:teuthology.orchestra.run.smithi066.stdout: "inode_count": 1, 2023-07-21T22:19:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_epoch": 16, 2023-07-21T22:19:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch": 89, 2023-07-21T22:19:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_epoch_barrier": 0, 2023-07-21T22:19:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "blocklisted": false, 2023-07-21T22:19:22.889 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs" 2023-07-21T22:19:22.889 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:19:22.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:22.899 DEBUG:teuthology.orchestra.run.smithi066:> stat --file-system '--printf=%T 2023-07-21T22:19:22.899 DEBUG:teuthology.orchestra.run.smithi066:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:22.945 INFO:teuthology.orchestra.run.smithi066.stdout:fuseblk 2023-07-21T22:19:22.945 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:22.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:22.946 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:23.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:23.058 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-07-21T22:19:23.192 INFO:teuthology.orchestra.run.smithi066.stdout:# file: . 2023-07-21T22:19:23.193 INFO:teuthology.orchestra.run.smithi066.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-07-21T22:19:23.193 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:23.193 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:19:23.193 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:19:23.238 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:23.339 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:23 smithi066 ceph-mon[126379]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2023-07-21T22:19:23.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 -- 172.21.15.66:0/1925689496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4100cf0 msgr2=0x7fb0c4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/1925689496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4100cf0 0x7fb0c4101140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009990 tx=0x7fb0b8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 -- 172.21.15.66:0/1925689496 shutdown_connections 2023-07-21T22:19:23.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/1925689496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 0x7fb0c40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/1925689496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c4101680 0x7fb0c4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/1925689496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4100cf0 0x7fb0c4101140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 -- 172.21.15.66:0/1925689496 >> 172.21.15.66:0/1925689496 conn(0x7fb0c40fb0e0 msgr2=0x7fb0c40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:23.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 -- 172.21.15.66:0/1925689496 shutdown_connections 2023-07-21T22:19:23.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.443+0000 7fb0c9bff700 1 -- 172.21.15.66:0/1925689496 wait complete. 2023-07-21T22:19:23.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.444+0000 7fb0c9bff700 1 Processor -- start 2023-07-21T22:19:23.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.444+0000 7fb0c9bff700 1 -- start start 2023-07-21T22:19:23.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c9bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 0x7fb0c40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c9bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c4100cf0 0x7fb0c4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c9bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 0x7fb0c4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c9bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40735e0 con 0x7fb0c4101680 2023-07-21T22:19:23.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c9bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4073750 con 0x7fb0c40ffb00 2023-07-21T22:19:23.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c9bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c419f890 con 0x7fb0c4100cf0 2023-07-21T22:19:23.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 0x7fb0c4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 0x7fb0c4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37266/0 (socket says 172.21.15.66:37266) 2023-07-21T22:19:23.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c3fff700 1 -- 172.21.15.66:0/4093198173 learned_addr learned my addr 172.21.15.66:0/4093198173 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:23.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c2ffd700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c4100cf0 0x7fb0c4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.445+0000 7fb0c37fe700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 0x7fb0c40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c3fff700 1 -- 172.21.15.66:0/4093198173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c4100cf0 msgr2=0x7fb0c4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c3fff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c4100cf0 0x7fb0c4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c3fff700 1 -- 172.21.15.66:0/4093198173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 msgr2=0x7fb0c40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c3fff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 0x7fb0c40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c3fff700 1 -- 172.21.15.66:0/4093198173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80095f0 con 0x7fb0c4101680 2023-07-21T22:19:23.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c37fe700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 0x7fb0c40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:23.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c3fff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 0x7fb0c4072e40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00ef30 tx=0x7fb0bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:23.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc018070 con 0x7fb0c4101680 2023-07-21T22:19:23.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0bc00f040 con 0x7fb0c4101680 2023-07-21T22:19:23.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00cd00 con 0x7fb0c4101680 2023-07-21T22:19:23.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c419fb10 con 0x7fb0c4101680 2023-07-21T22:19:23.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.446+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c419ffd0 con 0x7fb0c4101680 2023-07-21T22:19:23.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.447+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c410b320 con 0x7fb0c4101680 2023-07-21T22:19:23.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.449+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb0bc003a00 con 0x7fb0c4101680 2023-07-21T22:19:23.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.452+0000 7fb0c0ff9700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ac0a1220 0x7fb0ac0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.452+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0bc0cc600 con 0x7fb0c4101680 2023-07-21T22:19:23.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.452+0000 7fb0c37fe700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ac0a1220 0x7fb0ac0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.453+0000 7fb0c37fe700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ac0a1220 0x7fb0ac0a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4100b50 tx=0x7fb0b4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:23.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.456+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb0bc095b90 con 0x7fb0c4101680 2023-07-21T22:19:23.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:23 smithi104 ceph-mon[129021]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2023-07-21T22:19:23.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.667+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0c41a05e0 con 0x7fb0c4101680 2023-07-21T22:19:23.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.669+0000 7fb0c0ff9700 1 -- 172.21.15.66:0/4093198173 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7fb0bc0991b0 con 0x7fb0c4101680 2023-07-21T22:19:23.715 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:23.716 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:23.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ac0a1220 msgr2=0x7fb0ac0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ac0a1220 0x7fb0ac0a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4100b50 tx=0x7fb0b4009260 comp rx=0 tx=0).stop 2023-07-21T22:19:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 msgr2=0x7fb0c4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 0x7fb0c4072e40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00ef30 tx=0x7fb0bc00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 shutdown_connections 2023-07-21T22:19:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0ac0a1220 0x7fb0ac0a36d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0c40ffb00 0x7fb0c40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0c4101680 0x7fb0c4072e40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 --2- 172.21.15.66:0/4093198173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0c4100cf0 0x7fb0c4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 >> 172.21.15.66:0/4093198173 conn(0x7fb0c40fb0e0 msgr2=0x7fb0c40fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 shutdown_connections 2023-07-21T22:19:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.672+0000 7fb0c9bff700 1 -- 172.21.15.66:0/4093198173 wait complete. 2023-07-21T22:19:23.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:19:23.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:19:23.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:23 smithi149 ceph-mon[129268]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 204 B/s wr, 0 op/s 2023-07-21T22:19:23.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.902+0000 7f4e1bdff700 1 -- 172.21.15.66:0/3716790179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 msgr2=0x7f4e140ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.902+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/3716790179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140ffef0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009990 tx=0x7f4e10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 -- 172.21.15.66:0/3716790179 shutdown_connections 2023-07-21T22:19:23.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/3716790179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 0x7f4e14107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/3716790179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140ffef0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/3716790179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e14100cd0 0x7f4e14101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 -- 172.21.15.66:0/3716790179 >> 172.21.15.66:0/3716790179 conn(0x7f4e140fb0e0 msgr2=0x7f4e140fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:23.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 -- 172.21.15.66:0/3716790179 shutdown_connections 2023-07-21T22:19:23.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.903+0000 7f4e1bdff700 1 -- 172.21.15.66:0/3716790179 wait complete. 2023-07-21T22:19:23.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 Processor -- start 2023-07-21T22:19:23.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 -- start start 2023-07-21T22:19:23.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e14100cd0 0x7f4e14072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 0x7f4e14072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140735e0 con 0x7f4e140ffae0 2023-07-21T22:19:23.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14073750 con 0x7f4e14100cd0 2023-07-21T22:19:23.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.904+0000 7f4e1bdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1419f8d0 con 0x7f4e14101660 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37276/0 (socket says 172.21.15.66:37276) 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 -- 172.21.15.66:0/1816926487 learned_addr learned my addr 172.21.15.66:0/1816926487 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e1939a700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e14100cd0 0x7f4e14072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e1a39c700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 0x7f4e14072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 -- 172.21.15.66:0/1816926487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 msgr2=0x7f4e14072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 0x7f4e14072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 -- 172.21.15.66:0/1816926487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e14100cd0 msgr2=0x7f4e14072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e14100cd0 0x7f4e14072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 -- 172.21.15.66:0/1816926487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e100095f0 con 0x7f4e140ffae0 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e1a39c700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 0x7f4e14072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.905+0000 7f4e19b9b700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140742c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e10000c00 tx=0x7f4e10005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.906+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001d070 con 0x7f4e140ffae0 2023-07-21T22:19:23.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.906+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e10003f60 con 0x7f4e140ffae0 2023-07-21T22:19:23.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.906+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1419fb50 con 0x7f4e140ffae0 2023-07-21T22:19:23.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.906+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10022c70 con 0x7f4e140ffae0 2023-07-21T22:19:23.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.906+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1419ffd0 con 0x7f4e140ffae0 2023-07-21T22:19:23.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.907+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e14106a60 con 0x7f4e140ffae0 2023-07-21T22:19:23.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.909+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4e100040d0 con 0x7f4e140ffae0 2023-07-21T22:19:23.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.910+0000 7f4e0affd700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e000a11d0 0x7f4e000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:23.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.910+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4e100ce7d0 con 0x7f4e140ffae0 2023-07-21T22:19:23.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.911+0000 7f4e1939a700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e000a11d0 0x7f4e000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:23.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.911+0000 7f4e1939a700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e000a11d0 0x7f4e000a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005950 tx=0x7f4e0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:23.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:23.912+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4e10097d60 con 0x7f4e140ffae0 2023-07-21T22:19:24.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.119+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e1402c9e0 con 0x7f4e140ffae0 2023-07-21T22:19:24.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.120+0000 7f4e0affd700 1 -- 172.21.15.66:0/1816926487 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e1009b380 con 0x7f4e140ffae0 2023-07-21T22:19:24.165 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:19:24.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e000a11d0 msgr2=0x7f4e000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e000a11d0 0x7f4e000a3680 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e04005950 tx=0x7f4e0400b410 comp rx=0 tx=0).stop 2023-07-21T22:19:24.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 msgr2=0x7f4e140742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140742c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e10000c00 tx=0x7f4e10005740 comp rx=0 tx=0).stop 2023-07-21T22:19:24.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 shutdown_connections 2023-07-21T22:19:24.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e000a11d0 0x7f4e000a3680 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e14101660 0x7f4e14072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e140ffae0 0x7f4e140742c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 --2- 172.21.15.66:0/1816926487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e14100cd0 0x7f4e14072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 >> 172.21.15.66:0/1816926487 conn(0x7f4e140fb0e0 msgr2=0x7f4e140fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:24.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 shutdown_connections 2023-07-21T22:19:24.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.122+0000 7f4e1bdff700 1 -- 172.21.15.66:0/1816926487 wait complete. 2023-07-21T22:19:24.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:19:24.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 -- 172.21.15.66:0/2549529497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59881079a0 msgr2=0x7f5988109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 --2- 172.21.15.66:0/2549529497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59881079a0 0x7f5988109e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59800098f0 tx=0x7f5980009c00 comp rx=0 tx=0).stop 2023-07-21T22:19:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 -- 172.21.15.66:0/2549529497 shutdown_connections 2023-07-21T22:19:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 --2- 172.21.15.66:0/2549529497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59881079a0 0x7f5988109e20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 --2- 172.21.15.66:0/2549529497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5988074550 0x7f59880729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 --2- 172.21.15.66:0/2549529497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5988072f70 0x7f59880733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 -- 172.21.15.66:0/2549529497 >> 172.21.15.66:0/2549529497 conn(0x7f59880fb0e0 msgr2=0x7f59880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:24.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 -- 172.21.15.66:0/2549529497 shutdown_connections 2023-07-21T22:19:24.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.354+0000 7f598edff700 1 -- 172.21.15.66:0/2549529497 wait complete. 2023-07-21T22:19:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.355+0000 7f598edff700 1 Processor -- start 2023-07-21T22:19:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 -- start start 2023-07-21T22:19:24.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5988072f70 0x7f5988197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5988074550 0x7f5988197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 0x7f598819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598819f5f0 con 0x7f59881079a0 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598819f730 con 0x7f5988072f70 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598edff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598819f8c0 con 0x7f5988074550 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598d39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 0x7f598819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598d39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 0x7f598819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37290/0 (socket says 172.21.15.66:37290) 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598d39c700 1 -- 172.21.15.66:0/911221392 learned_addr learned my addr 172.21.15.66:0/911221392 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f598cb9b700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5988072f70 0x7f5988197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.356+0000 7f5987fff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5988074550 0x7f5988197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598d39c700 1 -- 172.21.15.66:0/911221392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5988074550 msgr2=0x7f5988197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598d39c700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5988074550 0x7f5988197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598d39c700 1 -- 172.21.15.66:0/911221392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5988072f70 msgr2=0x7f5988197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598d39c700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5988072f70 0x7f5988197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598d39c700 1 -- 172.21.15.66:0/911221392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5980009520 con 0x7f59881079a0 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598cb9b700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5988072f70 0x7f5988197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598d39c700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 0x7f598819f020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f598000fa40 tx=0x7f598000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598001c070 con 0x7f59881079a0 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598819fb40 con 0x7f59881079a0 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f598000f7d0 con 0x7f59881079a0 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980003a00 con 0x7f59881079a0 2023-07-21T22:19:24.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.357+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598819ff10 con 0x7f59881079a0 2023-07-21T22:19:24.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.359+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5988106050 con 0x7f59881079a0 2023-07-21T22:19:24.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.360+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5980020030 con 0x7f59881079a0 2023-07-21T22:19:24.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.362+0000 7f5985ffb700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f59780a1150 0x7f59780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.362+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f59800cdae0 con 0x7f59881079a0 2023-07-21T22:19:24.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.362+0000 7f598cb9b700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f59780a1150 0x7f59780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.363+0000 7f598cb9b700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f59780a1150 0x7f59780a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5988074920 tx=0x7f597c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:24.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.366+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5980097070 con 0x7f59881079a0 2023-07-21T22:19:24.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:24 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4093198173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:24.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:24 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1816926487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:24 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4093198173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:24 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1816926487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.576+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f598810d530 con 0x7f59881079a0 2023-07-21T22:19:24.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.576+0000 7f5985ffb700 1 -- 172.21.15.66:0/911221392 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f598009a690 con 0x7f59881079a0 2023-07-21T22:19:24.621 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:19:24.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f59780a1150 msgr2=0x7f59780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f59780a1150 0x7f59780a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5988074920 tx=0x7f597c00b040 comp rx=0 tx=0).stop 2023-07-21T22:19:24.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 msgr2=0x7f598819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 0x7f598819f020 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f598000fa40 tx=0x7f598000b790 comp rx=0 tx=0).stop 2023-07-21T22:19:24.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 shutdown_connections 2023-07-21T22:19:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f59780a1150 0x7f59780a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5988072f70 0x7f5988197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f59881079a0 0x7f598819f020 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 --2- 172.21.15.66:0/911221392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5988074550 0x7f5988197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 >> 172.21.15.66:0/911221392 conn(0x7f59880fb0e0 msgr2=0x7f59880fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:24.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 shutdown_connections 2023-07-21T22:19:24.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.579+0000 7f598edff700 1 -- 172.21.15.66:0/911221392 wait complete. 2023-07-21T22:19:24.642 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:19:24.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:24 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4093198173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:24.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:24 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1816926487' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:24.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.806+0000 7fa0e3fff700 1 -- 172.21.15.66:0/1416101353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc100cd0 msgr2=0x7fa0dc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.806+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/1416101353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc101120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0098f0 tx=0x7fa0cc009c00 comp rx=0 tx=0).stop 2023-07-21T22:19:24.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 -- 172.21.15.66:0/1416101353 shutdown_connections 2023-07-21T22:19:24.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/1416101353 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc101660 0x7fa0dc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/1416101353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/1416101353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc101120 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 -- 172.21.15.66:0/1416101353 >> 172.21.15.66:0/1416101353 conn(0x7fa0dc0fb0e0 msgr2=0x7fa0dc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:24.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 -- 172.21.15.66:0/1416101353 shutdown_connections 2023-07-21T22:19:24.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.807+0000 7fa0e3fff700 1 -- 172.21.15.66:0/1416101353 wait complete. 2023-07-21T22:19:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 Processor -- start 2023-07-21T22:19:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 -- start start 2023-07-21T22:19:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc101660 0x7fa0dc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0735e0 con 0x7fa0dc100cd0 2023-07-21T22:19:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc073750 con 0x7fa0dc101660 2023-07-21T22:19:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.808+0000 7fa0e3fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc19f980 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38744/0 (socket says 172.21.15.66:38744) 2023-07-21T22:19:24.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37308/0 (socket says 172.21.15.66:37308) 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 -- 172.21.15.66:0/3399846382 learned_addr learned my addr 172.21.15.66:0/3399846382 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e259c700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc101660 0x7fa0dc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 -- 172.21.15.66:0/3399846382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc101660 msgr2=0x7fa0dc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc101660 0x7fa0dc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 -- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 msgr2=0x7fa0dc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:24.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 -- 172.21.15.66:0/3399846382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc009520 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e159a700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.809+0000 7fa0e1d9b700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0742c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800e9f0 tx=0x7fa0d800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.810+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8004220 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.810+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc19fc60 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.810+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d8004380 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.810+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8003be0 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.810+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc1a0170 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.811+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc10b300 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.813+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa0d8003d40 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.814+0000 7fa0d2ffd700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa0c80a1150 0x7fa0c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.814+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa0d8014070 con 0x7fa0dc0ffae0 2023-07-21T22:19:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.814+0000 7fa0e159a700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa0c80a1150 0x7fa0c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:24.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.815+0000 7fa0e159a700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa0c80a1150 0x7fa0c80a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005e90 tx=0x7fa0cc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:24.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:24.816+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa0d80962b0 con 0x7fa0dc0ffae0 2023-07-21T22:19:25.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.025+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0dc106a80 con 0x7fa0dc0ffae0 2023-07-21T22:19:25.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.026+0000 7fa0d2ffd700 1 -- 172.21.15.66:0/3399846382 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0d801f340 con 0x7fa0dc0ffae0 2023-07-21T22:19:25.071 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:19:25.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa0c80a1150 msgr2=0x7fa0c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa0c80a1150 0x7fa0c80a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005e90 tx=0x7fa0cc019040 comp rx=0 tx=0).stop 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 msgr2=0x7fa0dc0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0742c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800e9f0 tx=0x7fa0d800ed00 comp rx=0 tx=0).stop 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 shutdown_connections 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa0c80a1150 0x7fa0c80a3600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0dc101660 0x7fa0dc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa0dc0ffae0 0x7fa0dc0742c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 --2- 172.21.15.66:0/3399846382 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa0dc100cd0 0x7fa0dc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 >> 172.21.15.66:0/3399846382 conn(0x7fa0dc0fb0e0 msgr2=0x7fa0dc0fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:25.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 shutdown_connections 2023-07-21T22:19:25.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.029+0000 7fa0e3fff700 1 -- 172.21.15.66:0/3399846382 wait complete. 2023-07-21T22:19:25.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:19:25.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 -- 172.21.15.66:0/3430849101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2230072f70 msgr2=0x7f22300733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 --2- 172.21.15.66:0/3430849101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2230072f70 0x7f22300733e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2224009990 tx=0x7f2224009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 -- 172.21.15.66:0/3430849101 shutdown_connections 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 --2- 172.21.15.66:0/3430849101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22301079a0 0x7f2230109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 --2- 172.21.15.66:0/3430849101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 0x7f22300729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 --2- 172.21.15.66:0/3430849101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2230072f70 0x7f22300733e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 -- 172.21.15.66:0/3430849101 >> 172.21.15.66:0/3430849101 conn(0x7f22300fb0e0 msgr2=0x7f22300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 -- 172.21.15.66:0/3430849101 shutdown_connections 2023-07-21T22:19:25.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.257+0000 7f2235dff700 1 -- 172.21.15.66:0/3430849101 wait complete. 2023-07-21T22:19:25.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.258+0000 7f2235dff700 1 Processor -- start 2023-07-21T22:19:25.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 -- start start 2023-07-21T22:19:25.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 0x7f2230197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 0x7f2230198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22301079a0 0x7f223019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223019f660 con 0x7f2230072f70 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223019f7a0 con 0x7f22301079a0 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f2235dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223019f930 con 0x7f2230074550 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f222f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 0x7f2230197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f222f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 0x7f2230197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37328/0 (socket says 172.21.15.66:37328) 2023-07-21T22:19:25.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f222f7fe700 1 -- 172.21.15.66:0/2880256196 learned_addr learned my addr 172.21.15.66:0/2880256196 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f222ffff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22301079a0 0x7f223019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.259+0000 7f222effd700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 0x7f2230198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222f7fe700 1 -- 172.21.15.66:0/2880256196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 msgr2=0x7f2230198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222f7fe700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 0x7f2230198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222f7fe700 1 -- 172.21.15.66:0/2880256196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22301079a0 msgr2=0x7f223019f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222f7fe700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22301079a0 0x7f223019f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222f7fe700 1 -- 172.21.15.66:0/2880256196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2228009c40 con 0x7f2230072f70 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222effd700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 0x7f2230198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222f7fe700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 0x7f2230197ae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f222000e970 tx=0x7f222000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2220009790 con 0x7f2230072f70 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22240095f0 con 0x7f2230072f70 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f222000eeb0 con 0x7f2230072f70 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200055c0 con 0x7f2230072f70 2023-07-21T22:19:25.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.260+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223019ff90 con 0x7f2230072f70 2023-07-21T22:19:25.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.261+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230106050 con 0x7f2230072f70 2023-07-21T22:19:25.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.263+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2220004750 con 0x7f2230072f70 2023-07-21T22:19:25.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.266+0000 7f222cff9700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22180a1150 0x7f22180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.266+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2220014070 con 0x7f2230072f70 2023-07-21T22:19:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.266+0000 7f222effd700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22180a1150 0x7f22180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.266+0000 7f222effd700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22180a1150 0x7f22180a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f222400b2e0 tx=0x7f2224005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:25.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.270+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f22200962a0 con 0x7f2230072f70 2023-07-21T22:19:25.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:25 smithi066 ceph-mon[126379]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2023-07-21T22:19:25.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:25 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/911221392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:19:25.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:25 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3399846382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:19:25.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.481+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f223019fc10 con 0x7f2230072f70 2023-07-21T22:19:25.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.482+0000 7f222cff9700 1 -- 172.21.15.66:0/2880256196 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f223019fc10 con 0x7f2230072f70 2023-07-21T22:19:25.527 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:19:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22180a1150 msgr2=0x7f22180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22180a1150 0x7f22180a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f222400b2e0 tx=0x7f2224005fd0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 msgr2=0x7f2230197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 0x7f2230197ae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f222000e970 tx=0x7f222000ed30 comp rx=0 tx=0).stop 2023-07-21T22:19:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 shutdown_connections 2023-07-21T22:19:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22180a1150 0x7f22180a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22301079a0 0x7f223019f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2230072f70 0x7f2230197ae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 --2- 172.21.15.66:0/2880256196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2230074550 0x7f2230198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 >> 172.21.15.66:0/2880256196 conn(0x7f22300fb0e0 msgr2=0x7f22300fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 shutdown_connections 2023-07-21T22:19:25.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.484+0000 7f2235dff700 1 -- 172.21.15.66:0/2880256196 wait complete. 2023-07-21T22:19:25.547 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T22:19:25.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:25 smithi104 ceph-mon[129021]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2023-07-21T22:19:25.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:25 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/911221392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:19:25.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:25 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3399846382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:19:25.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:25 smithi149 ceph-mon[129268]: pgmap v283: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 86 B/s wr, 0 op/s 2023-07-21T22:19:25.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:25 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/911221392' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:19:25.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:25 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3399846382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:19:25.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 -- 172.21.15.66:0/613772372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 msgr2=0x7f574c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 --2- 172.21.15.66:0/613772372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0ffef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f573c009990 tx=0x7f573c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 -- 172.21.15.66:0/613772372 shutdown_connections 2023-07-21T22:19:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 --2- 172.21.15.66:0/613772372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574c101660 0x7f574c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 --2- 172.21.15.66:0/613772372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0ffef0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 --2- 172.21.15.66:0/613772372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c100cd0 0x7f574c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.709+0000 7f5750fff700 1 -- 172.21.15.66:0/613772372 >> 172.21.15.66:0/613772372 conn(0x7f574c0fb0e0 msgr2=0x7f574c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.710+0000 7f5750fff700 1 -- 172.21.15.66:0/613772372 shutdown_connections 2023-07-21T22:19:25.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.710+0000 7f5750fff700 1 -- 172.21.15.66:0/613772372 wait complete. 2023-07-21T22:19:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 Processor -- start 2023-07-21T22:19:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 -- start start 2023-07-21T22:19:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574c100cd0 0x7f574c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c101660 0x7f574c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c199730 con 0x7f574c0ffae0 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c073380 con 0x7f574c101660 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5750fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c073540 con 0x7f574c100cd0 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f574a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f574a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37348/0 (socket says 172.21.15.66:37348) 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f574a59c700 1 -- 172.21.15.66:0/212647904 learned_addr learned my addr 172.21.15.66:0/212647904 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f574ad9d700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c101660 0x7f574c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.711+0000 7f5749d9b700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574c100cd0 0x7f574c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574a59c700 1 -- 172.21.15.66:0/212647904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574c100cd0 msgr2=0x7f574c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574a59c700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574c100cd0 0x7f574c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574a59c700 1 -- 172.21.15.66:0/212647904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c101660 msgr2=0x7f574c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574a59c700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c101660 0x7f574c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574a59c700 1 -- 172.21.15.66:0/212647904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0095f0 con 0x7f574c0ffae0 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574ad9d700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c101660 0x7f574c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:25.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f574a59c700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0742c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f573c004730 tx=0x7f573c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:25.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c01d070 con 0x7f574c0ffae0 2023-07-21T22:19:25.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f573c021410 con 0x7f574c0ffae0 2023-07-21T22:19:25.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c1a40e0 con 0x7f574c0ffae0 2023-07-21T22:19:25.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.712+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c017650 con 0x7f574c0ffae0 2023-07-21T22:19:25.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.713+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1a4450 con 0x7f574c0ffae0 2023-07-21T22:19:25.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.713+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c10b300 con 0x7f574c0ffae0 2023-07-21T22:19:25.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.715+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f573c021580 con 0x7f574c0ffae0 2023-07-21T22:19:25.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.717+0000 7f573b7fe700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f57340a1150 0x7f57340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:25.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.717+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f573c0ce600 con 0x7f574c0ffae0 2023-07-21T22:19:25.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.717+0000 7f5749d9b700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f57340a1150 0x7f57340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:25.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.718+0000 7f5749d9b700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f57340a1150 0x7f57340a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5740009c60 tx=0x7f5740009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:25.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.719+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f573c097b90 con 0x7f574c0ffae0 2023-07-21T22:19:25.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.947+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f574c106a30 con 0x7f574c0ffae0 2023-07-21T22:19:25.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.947+0000 7f573b7fe700 1 -- 172.21.15.66:0/212647904 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+588 (secure 0 0 0) 0x7f573c09b1b0 con 0x7f574c0ffae0 2023-07-21T22:19:25.992 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/662946744 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:25.993 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3065709937 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:25.993 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1244827991 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:25.993 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/979491945 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:25.993 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3065709937 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:25.994 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3161416321 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:25.994 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3483118396 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:25.994 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/979491945 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:25.994 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/690385827 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:25.994 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3486180168 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.949+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f57340a1150 msgr2=0x7f57340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.949+0000 7f5750fff700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f57340a1150 0x7f57340a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5740009c60 tx=0x7f5740009260 comp rx=0 tx=0).stop 2023-07-21T22:19:25.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.949+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 msgr2=0x7f574c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.949+0000 7f5750fff700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0742c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f573c004730 tx=0x7f573c004760 comp rx=0 tx=0).stop 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 shutdown_connections 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f57340a1150 0x7f57340a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f574c101660 0x7f574c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f574c0ffae0 0x7f574c0742c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 --2- 172.21.15.66:0/212647904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f574c100cd0 0x7f574c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:25.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 >> 172.21.15.66:0/212647904 conn(0x7f574c0fb0e0 msgr2=0x7f574c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 shutdown_connections 2023-07-21T22:19:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:25.950+0000 7f5750fff700 1 -- 172.21.15.66:0/212647904 wait complete. 2023-07-21T22:19:25.997 INFO:teuthology.orchestra.run.smithi066.stderr:listed 10 entries 2023-07-21T22:19:26.013 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:19:26.013 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:19:26.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:26.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 -- 172.21.15.66:0/2900464246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 msgr2=0x7f80d40fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 --2- 172.21.15.66:0/2900464246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 0x7f80d40fff10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009990 tx=0x7f80c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 -- 172.21.15.66:0/2900464246 shutdown_connections 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 --2- 172.21.15.66:0/2900464246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 0x7f80d40fff10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 --2- 172.21.15.66:0/2900464246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 --2- 172.21.15.66:0/2900464246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d4100cf0 0x7f80d4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.260+0000 7f80d8bff700 1 -- 172.21.15.66:0/2900464246 >> 172.21.15.66:0/2900464246 conn(0x7f80d40fb0e0 msgr2=0x7f80d40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:26.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.261+0000 7f80d8bff700 1 -- 172.21.15.66:0/2900464246 shutdown_connections 2023-07-21T22:19:26.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.261+0000 7f80d8bff700 1 -- 172.21.15.66:0/2900464246 wait complete. 2023-07-21T22:19:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.262+0000 7f80d8bff700 1 Processor -- start 2023-07-21T22:19:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.262+0000 7f80d8bff700 1 -- start start 2023-07-21T22:19:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d8bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 0x7f80d4195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d8bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d4100cf0 0x7f80d4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d8bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d8bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419d3c0 con 0x7f80d4101680 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d8bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419d500 con 0x7f80d40ffb00 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d8bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d419d690 con 0x7f80d4100cf0 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d4100cf0 0x7f80d4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37374/0 (socket says 172.21.15.66:37374) 2023-07-21T22:19:26.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d2d9d700 1 -- 172.21.15.66:0/3345674079 learned_addr learned my addr 172.21.15.66:0/3345674079 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.263+0000 7f80d259c700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 0x7f80d4195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d2d9d700 1 -- 172.21.15.66:0/3345674079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d4100cf0 msgr2=0x7f80d4195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d2d9d700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d4100cf0 0x7f80d4195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d2d9d700 1 -- 172.21.15.66:0/3345674079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 msgr2=0x7f80d4195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d2d9d700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 0x7f80d4195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d2d9d700 1 -- 172.21.15.66:0/3345674079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc009c40 con 0x7f80d4101680 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d2d9d700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d419cdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c370 tx=0x7f80cc00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc017070 con 0x7f80d4101680 2023-07-21T22:19:26.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c40095f0 con 0x7f80d4101680 2023-07-21T22:19:26.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80cc00f040 con 0x7f80d4101680 2023-07-21T22:19:26.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc00e050 con 0x7f80d4101680 2023-07-21T22:19:26.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.264+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d419dc90 con 0x7f80d4101680 2023-07-21T22:19:26.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.265+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d410b320 con 0x7f80d4101680 2023-07-21T22:19:26.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.267+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f80cc00cb60 con 0x7f80d4101680 2023-07-21T22:19:26.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.268+0000 7f80c37fe700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80bc0a11a0 0x7f80bc0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.269+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f80cc0cc4e0 con 0x7f80d4101680 2023-07-21T22:19:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.269+0000 7f80d259c700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80bc0a11a0 0x7f80bc0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.269+0000 7f80d259c700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80bc0a11a0 0x7f80bc0a3650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80c400b2e0 tx=0x7f80c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:26.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.272+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f80cc0959e0 con 0x7f80d4101680 2023-07-21T22:19:26.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.485+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80d410b4e0 con 0x7f80d4101680 2023-07-21T22:19:26.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.487+0000 7f80c37fe700 1 -- 172.21.15.66:0/3345674079 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f80cc099000 con 0x7f80d4101680 2023-07-21T22:19:26.532 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:26.533 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:26.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.489+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80bc0a11a0 msgr2=0x7f80bc0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.489+0000 7f80d8bff700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80bc0a11a0 0x7f80bc0a3650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80c400b2e0 tx=0x7f80c4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.489+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 msgr2=0x7f80d419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.489+0000 7f80d8bff700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d419cdf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c370 tx=0x7f80cc00c680 comp rx=0 tx=0).stop 2023-07-21T22:19:26.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 shutdown_connections 2023-07-21T22:19:26.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f80bc0a11a0 0x7f80bc0a3650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d40ffb00 0x7f80d4195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f80d4101680 0x7f80d419cdf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 --2- 172.21.15.66:0/3345674079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80d4100cf0 0x7f80d4195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 >> 172.21.15.66:0/3345674079 conn(0x7f80d40fb0e0 msgr2=0x7f80d40fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:26.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 shutdown_connections 2023-07-21T22:19:26.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.490+0000 7f80d8bff700 1 -- 172.21.15.66:0/3345674079 wait complete. 2023-07-21T22:19:26.551 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:26 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2880256196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:19:26.551 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:26 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/212647904' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:19:26.553 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:19:26.554 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:19:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:26 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2880256196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:19:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:26 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/212647904' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:19:26.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:26 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2880256196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:19:26.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:26 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/212647904' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:19:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 -- 172.21.15.66:0/2637496914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440ffae0 msgr2=0x7f36440ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2637496914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440ffae0 0x7f36440ffef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3638009990 tx=0x7f3638009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 -- 172.21.15.66:0/2637496914 shutdown_connections 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2637496914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644101660 0x7f3644107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2637496914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36440ffae0 0x7f36440ffef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2637496914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3644100cd0 0x7f3644101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 -- 172.21.15.66:0/2637496914 >> 172.21.15.66:0/2637496914 conn(0x7f36440fb0e0 msgr2=0x7f36440fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 -- 172.21.15.66:0/2637496914 shutdown_connections 2023-07-21T22:19:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.718+0000 7f364b1ff700 1 -- 172.21.15.66:0/2637496914 wait complete. 2023-07-21T22:19:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.719+0000 7f364b1ff700 1 Processor -- start 2023-07-21T22:19:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 -- start start 2023-07-21T22:19:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36440ffae0 0x7f3644197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644100cd0 0x7f3644197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 0x7f364419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419f5f0 con 0x7f3644101660 2023-07-21T22:19:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419f730 con 0x7f3644100cd0 2023-07-21T22:19:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364b1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364419f8c0 con 0x7f36440ffae0 2023-07-21T22:19:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364979c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 0x7f364419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364979c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 0x7f364419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37392/0 (socket says 172.21.15.66:37392) 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f364979c700 1 -- 172.21.15.66:0/2581572973 learned_addr learned my addr 172.21.15.66:0/2581572973 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f3648f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36440ffae0 0x7f3644197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.720+0000 7f3643fff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644100cd0 0x7f3644197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364979c700 1 -- 172.21.15.66:0/2581572973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36440ffae0 msgr2=0x7f3644197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364979c700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36440ffae0 0x7f3644197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364979c700 1 -- 172.21.15.66:0/2581572973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644100cd0 msgr2=0x7f3644197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364979c700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644100cd0 0x7f3644197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364979c700 1 -- 172.21.15.66:0/2581572973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36380095f0 con 0x7f3644101660 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f3648f9b700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36440ffae0 0x7f3644197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f3643fff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644100cd0 0x7f3644197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364979c700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 0x7f364419f020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f363c00ef30 tx=0x7f363c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c018070 con 0x7f3644101660 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364419fb40 con 0x7f3644101660 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f363c00f040 con 0x7f3644101660 2023-07-21T22:19:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c00cd00 con 0x7f3644101660 2023-07-21T22:19:26.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.721+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36441a0050 con 0x7f3644101660 2023-07-21T22:19:26.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.722+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36441921c0 con 0x7f3644101660 2023-07-21T22:19:26.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.724+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f363c003a00 con 0x7f3644101660 2023-07-21T22:19:26.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.726+0000 7f3641ffb700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36340a1150 0x7f36340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:26.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.726+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f363c0cc6b0 con 0x7f3644101660 2023-07-21T22:19:26.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.726+0000 7f3648f9b700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36340a1150 0x7f36340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:26.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.727+0000 7f3648f9b700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36340a1150 0x7f36340a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f363800b2e0 tx=0x7f3638005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.729+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f363c095c40 con 0x7f3644101660 2023-07-21T22:19:26.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.937+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3644061120 con 0x7f3644101660 2023-07-21T22:19:26.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.938+0000 7f3641ffb700 1 -- 172.21.15.66:0/2581572973 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f363c099260 con 0x7f3644101660 2023-07-21T22:19:26.983 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36340a1150 msgr2=0x7f36340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36340a1150 0x7f36340a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f363800b2e0 tx=0x7f3638005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 msgr2=0x7f364419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 0x7f364419f020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f363c00ef30 tx=0x7f363c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 shutdown_connections 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f36340a1150 0x7f36340a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3644101660 0x7f364419f020 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36440ffae0 0x7f3644197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 --2- 172.21.15.66:0/2581572973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3644100cd0 0x7f3644197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:26.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 >> 172.21.15.66:0/2581572973 conn(0x7f36440fb0e0 msgr2=0x7f36441065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:26.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 shutdown_connections 2023-07-21T22:19:26.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:26.940+0000 7f364b1ff700 1 -- 172.21.15.66:0/2581572973 wait complete. 2023-07-21T22:19:27.004 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:19:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 -- 172.21.15.66:0/2470580695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22201079a0 msgr2=0x7f2220109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 --2- 172.21.15.66:0/2470580695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22201079a0 0x7f2220109e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2218009f90 tx=0x7f221800b330 comp rx=0 tx=0).stop 2023-07-21T22:19:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 -- 172.21.15.66:0/2470580695 shutdown_connections 2023-07-21T22:19:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 --2- 172.21.15.66:0/2470580695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22201079a0 0x7f2220109e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 --2- 172.21.15.66:0/2470580695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220074550 0x7f22200729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 --2- 172.21.15.66:0/2470580695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2220072f70 0x7f22200733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 -- 172.21.15.66:0/2470580695 >> 172.21.15.66:0/2470580695 conn(0x7f22200fb0e0 msgr2=0x7f22200fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:27.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 -- 172.21.15.66:0/2470580695 shutdown_connections 2023-07-21T22:19:27.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.169+0000 7f22257ff700 1 -- 172.21.15.66:0/2470580695 wait complete. 2023-07-21T22:19:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.170+0000 7f22257ff700 1 Processor -- start 2023-07-21T22:19:27.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.170+0000 7f22257ff700 1 -- start start 2023-07-21T22:19:27.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f22257ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 0x7f2220197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f22257ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2220074550 0x7f22201981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f22257ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22201079a0 0x7f222019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f22257ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f7c0 con 0x7f2220072f70 2023-07-21T22:19:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f22257ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019f930 con 0x7f2220074550 2023-07-21T22:19:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f22257ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222019faf0 con 0x7f22201079a0 2023-07-21T22:19:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f221effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 0x7f2220197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f221f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22201079a0 0x7f222019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f221e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2220074550 0x7f22201981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f221effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 0x7f2220197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37410/0 (socket says 172.21.15.66:37410) 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.171+0000 7f221effd700 1 -- 172.21.15.66:0/3742302879 learned_addr learned my addr 172.21.15.66:0/3742302879 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221f7fe700 1 -- 172.21.15.66:0/3742302879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2220074550 msgr2=0x7f22201981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221f7fe700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2220074550 0x7f22201981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221f7fe700 1 -- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 msgr2=0x7f2220197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221f7fe700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 0x7f2220197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221f7fe700 1 -- 172.21.15.66:0/3742302879 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100095f0 con 0x7f22201079a0 2023-07-21T22:19:27.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221effd700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 0x7f2220197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f221f7fe700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22201079a0 0x7f222019f1f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f221800b6c0 tx=0x7f2218004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221800e040 con 0x7f22201079a0 2023-07-21T22:19:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2218009c40 con 0x7f22201079a0 2023-07-21T22:19:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221800bbf0 con 0x7f22201079a0 2023-07-21T22:19:27.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218005040 con 0x7f22201079a0 2023-07-21T22:19:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.172+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22201a0060 con 0x7f22201079a0 2023-07-21T22:19:27.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.173+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2220106050 con 0x7f22201079a0 2023-07-21T22:19:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.176+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f221801d470 con 0x7f22201079a0 2023-07-21T22:19:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.177+0000 7f2207fff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22080a1150 0x7f22080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.177+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f22180ce4c0 con 0x7f22201079a0 2023-07-21T22:19:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.177+0000 7f221effd700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22080a1150 0x7f22080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.178+0000 7f221effd700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22080a1150 0x7f22080a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2220199150 tx=0x7f2210009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:27.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.179+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2218097a50 con 0x7f22201079a0 2023-07-21T22:19:27.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.397+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f222002cc30 con 0x7f22201079a0 2023-07-21T22:19:27.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.398+0000 7f2207fff700 1 -- 172.21.15.66:0/3742302879 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f221809b070 con 0x7f22201079a0 2023-07-21T22:19:27.443 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:19:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.400+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22080a1150 msgr2=0x7f22080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.400+0000 7f22257ff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22080a1150 0x7f22080a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2220199150 tx=0x7f2210009310 comp rx=0 tx=0).stop 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.400+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22201079a0 msgr2=0x7f222019f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.400+0000 7f22257ff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22201079a0 0x7f222019f1f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f221800b6c0 tx=0x7f2218004c60 comp rx=0 tx=0).stop 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.400+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 shutdown_connections 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.400+0000 7f22257ff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f22080a1150 0x7f22080a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.401+0000 7f22257ff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22201079a0 0x7f222019f1f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.401+0000 7f22257ff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2220072f70 0x7f2220197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.401+0000 7f22257ff700 1 --2- 172.21.15.66:0/3742302879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2220074550 0x7f22201981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.401+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 >> 172.21.15.66:0/3742302879 conn(0x7f22200fb0e0 msgr2=0x7f22200fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:27.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.401+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 shutdown_connections 2023-07-21T22:19:27.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.401+0000 7f22257ff700 1 -- 172.21.15.66:0/3742302879 wait complete. 2023-07-21T22:19:27.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:19:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:27 smithi104 ceph-mon[129021]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-21T22:19:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:27 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3345674079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:27 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2581572973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 -- 172.21.15.66:0/1242880320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc100620 msgr2=0x7f15dc102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 --2- 172.21.15.66:0/1242880320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc100620 0x7f15dc102a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15d0009990 tx=0x7f15d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 -- 172.21.15.66:0/1242880320 shutdown_connections 2023-07-21T22:19:27.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 --2- 172.21.15.66:0/1242880320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15dc105860 0x7f15dc107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 --2- 172.21.15.66:0/1242880320 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc100620 0x7f15dc102a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 --2- 172.21.15.66:0/1242880320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 -- 172.21.15.66:0/1242880320 >> 172.21.15.66:0/1242880320 conn(0x7f15dc0fa000 msgr2=0x7f15dc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:27.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 -- 172.21.15.66:0/1242880320 shutdown_connections 2023-07-21T22:19:27.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.628+0000 7f15e2fff700 1 -- 172.21.15.66:0/1242880320 wait complete. 2023-07-21T22:19:27.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 Processor -- start 2023-07-21T22:19:27.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 -- start start 2023-07-21T22:19:27.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15dc100620 0x7f15dc0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc0fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 0x7f15dc0feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0ff2b0 con 0x7f15dc105860 2023-07-21T22:19:27.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0ff420 con 0x7f15dc102f40 2023-07-21T22:19:27.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.630+0000 7f15e2fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc19f900 con 0x7f15dc100620 2023-07-21T22:19:27.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 0x7f15dc0feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15dbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc0fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 0x7f15dc0feb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37430/0 (socket says 172.21.15.66:37430) 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15dbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc0fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:35780/0 (socket says 172.21.15.66:35780) 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 -- 172.21.15.66:0/2811885384 learned_addr learned my addr 172.21.15.66:0/2811885384 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e0d9b700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15dc100620 0x7f15dc0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 -- 172.21.15.66:0/2811885384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15dc100620 msgr2=0x7f15dc0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15dc100620 0x7f15dc0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 -- 172.21.15.66:0/2811885384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 msgr2=0x7f15dc0fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc0fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 -- 172.21.15.66:0/2811885384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00095f0 con 0x7f15dc105860 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15dbfff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc0fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.631+0000 7f15e159c700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 0x7f15dc0feb10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15d400ef30 tx=0x7f15d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.632+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4018070 con 0x7f15dc105860 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.632+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d400f040 con 0x7f15dc105860 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.632+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400cd00 con 0x7f15dc105860 2023-07-21T22:19:27.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.632+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15dc19fb80 con 0x7f15dc105860 2023-07-21T22:19:27.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.632+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15dc1a0040 con 0x7f15dc105860 2023-07-21T22:19:27.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.633+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15dc0fbb70 con 0x7f15dc105860 2023-07-21T22:19:27.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.635+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f15d4003a00 con 0x7f15dc105860 2023-07-21T22:19:27.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.636+0000 7f15d9ffb700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f15cc0a1150 0x7f15cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:27.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.636+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f15d40cc640 con 0x7f15dc105860 2023-07-21T22:19:27.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.637+0000 7f15e0d9b700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f15cc0a1150 0x7f15cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:27.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.637+0000 7f15e0d9b700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f15cc0a1150 0x7f15cc0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b2e0 tx=0x7f15d0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:27.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.640+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f15d4095bd0 con 0x7f15dc105860 2023-07-21T22:19:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:27 smithi066 ceph-mon[126379]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-21T22:19:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:27 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3345674079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:27 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2581572973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:27.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:27 smithi149 ceph-mon[129268]: pgmap v284: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2023-07-21T22:19:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:27 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3345674079' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:27 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2581572973' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:19:27.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.856+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15dc0611c0 con 0x7f15dc105860 2023-07-21T22:19:27.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.856+0000 7f15d9ffb700 1 -- 172.21.15.66:0/2811885384 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15d40991f0 con 0x7f15dc105860 2023-07-21T22:19:27.902 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:19:27.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f15cc0a1150 msgr2=0x7f15cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f15cc0a1150 0x7f15cc0a3600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f15d000b2e0 tx=0x7f15d0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 msgr2=0x7f15dc0feb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:27.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 0x7f15dc0feb10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f15d400ef30 tx=0x7f15d400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 shutdown_connections 2023-07-21T22:19:27.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f15cc0a1150 0x7f15cc0a3600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f15dc105860 0x7f15dc0feb10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15dc100620 0x7f15dc0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 --2- 172.21.15.66:0/2811885384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15dc102f40 0x7f15dc0fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:27.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 >> 172.21.15.66:0/2811885384 conn(0x7f15dc0fa000 msgr2=0x7f15dc101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:27.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 shutdown_connections 2023-07-21T22:19:27.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:27.859+0000 7f15e2fff700 1 -- 172.21.15.66:0/2811885384 wait complete. 2023-07-21T22:19:27.922 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:19:28.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/3671843882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d88100cd0 msgr2=0x7f2d88101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/3671843882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d88100cd0 0x7f2d88101120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0098f0 tx=0x7f2d7c009c00 comp rx=0 tx=0).stop 2023-07-21T22:19:28.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/3671843882 shutdown_connections 2023-07-21T22:19:28.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/3671843882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d88101660 0x7f2d88107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/3671843882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d880ffae0 0x7f2d880ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/3671843882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d88100cd0 0x7f2d88101120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/3671843882 >> 172.21.15.66:0/3671843882 conn(0x7f2d880fb0e0 msgr2=0x7f2d880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:28.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/3671843882 shutdown_connections 2023-07-21T22:19:28.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.087+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/3671843882 wait complete. 2023-07-21T22:19:28.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.088+0000 7f2d8e1ff700 1 Processor -- start 2023-07-21T22:19:28.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.088+0000 7f2d8e1ff700 1 -- start start 2023-07-21T22:19:28.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d8e1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d880ffae0 0x7f2d88195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d8e1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 0x7f2d88195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d8e1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d88101660 0x7f2d8819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d8e1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8819d3f0 con 0x7f2d88100cd0 2023-07-21T22:19:28.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d8e1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8819d530 con 0x7f2d880ffae0 2023-07-21T22:19:28.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d8e1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8819d6c0 con 0x7f2d88101660 2023-07-21T22:19:28.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 0x7f2d88195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 0x7f2d88195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37454/0 (socket says 172.21.15.66:37454) 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 -- 172.21.15.66:0/195090548 learned_addr learned my addr 172.21.15.66:0/195090548 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d877fe700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d880ffae0 0x7f2d88195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d87fff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d88101660 0x7f2d8819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 -- 172.21.15.66:0/195090548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d88101660 msgr2=0x7f2d8819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d88101660 0x7f2d8819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 -- 172.21.15.66:0/195090548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d880ffae0 msgr2=0x7f2d88195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d880ffae0 0x7f2d88195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.089+0000 7f2d86ffd700 1 -- 172.21.15.66:0/195090548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780095f0 con 0x7f2d88100cd0 2023-07-21T22:19:28.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.090+0000 7f2d86ffd700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 0x7f2d88195db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00b730 tx=0x7f2d7c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:28.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.090+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c01c070 con 0x7f2d88100cd0 2023-07-21T22:19:28.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.090+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d7c00f7b0 con 0x7f2d88100cd0 2023-07-21T22:19:28.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.090+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c009520 con 0x7f2d88100cd0 2023-07-21T22:19:28.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.090+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004a70 con 0x7f2d88100cd0 2023-07-21T22:19:28.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.090+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8819dbd0 con 0x7f2d88100cd0 2023-07-21T22:19:28.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.091+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8810b2b0 con 0x7f2d88100cd0 2023-07-21T22:19:28.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.093+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2d7c020070 con 0x7f2d88100cd0 2023-07-21T22:19:28.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.095+0000 7f2d84ff9700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2d700a1150 0x7f2d700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.095+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2d7c0cd9c0 con 0x7f2d88100cd0 2023-07-21T22:19:28.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.095+0000 7f2d877fe700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2d700a1150 0x7f2d700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.096+0000 7f2d877fe700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2d700a1150 0x7f2d700a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d88100b30 tx=0x7f2d78009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:28.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.099+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2d7c096f50 con 0x7f2d88100cd0 2023-07-21T22:19:28.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.310+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2d88106a70 con 0x7f2d88100cd0 2023-07-21T22:19:28.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.311+0000 7f2d84ff9700 1 -- 172.21.15.66:0/195090548 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2d7c09a570 con 0x7f2d88100cd0 2023-07-21T22:19:28.356 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:19:28.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2d700a1150 msgr2=0x7f2d700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2d700a1150 0x7f2d700a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2d88100b30 tx=0x7f2d78009310 comp rx=0 tx=0).stop 2023-07-21T22:19:28.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 msgr2=0x7f2d88195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 0x7f2d88195db0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00b730 tx=0x7f2d7c00b760 comp rx=0 tx=0).stop 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 shutdown_connections 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2d700a1150 0x7f2d700a3600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d880ffae0 0x7f2d88195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d88101660 0x7f2d8819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 --2- 172.21.15.66:0/195090548 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2d88100cd0 0x7f2d88195db0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 >> 172.21.15.66:0/195090548 conn(0x7f2d880fb0e0 msgr2=0x7f2d880fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 shutdown_connections 2023-07-21T22:19:28.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.313+0000 7f2d8e1ff700 1 -- 172.21.15.66:0/195090548 wait complete. 2023-07-21T22:19:28.377 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T22:19:28.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 -- 172.21.15.66:0/4205972911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 msgr2=0x7f180c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 --2- 172.21.15.66:0/4205972911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c0ffef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009990 tx=0x7f17fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 -- 172.21.15.66:0/4205972911 shutdown_connections 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 --2- 172.21.15.66:0/4205972911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 0x7f180c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 --2- 172.21.15.66:0/4205972911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c0ffef0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 --2- 172.21.15.66:0/4205972911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 0x7f180c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 -- 172.21.15.66:0/4205972911 >> 172.21.15.66:0/4205972911 conn(0x7f180c0fb0e0 msgr2=0x7f180c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 -- 172.21.15.66:0/4205972911 shutdown_connections 2023-07-21T22:19:28.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.540+0000 7f18121ff700 1 -- 172.21.15.66:0/4205972911 wait complete. 2023-07-21T22:19:28.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.541+0000 7f18121ff700 1 Processor -- start 2023-07-21T22:19:28.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.541+0000 7f18121ff700 1 -- start start 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f18121ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f18121ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 0x7f180c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f18121ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 0x7f180c19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f18121ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19f630 con 0x7f180c0ffae0 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f18121ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19f770 con 0x7f180c100cd0 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f18121ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c19f900 con 0x7f180c101660 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37466/0 (socket says 172.21.15.66:37466) 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 -- 172.21.15.66:0/3301359962 learned_addr learned my addr 172.21.15.66:0/3301359962 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180affd700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 0x7f180c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180bfff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 0x7f180c19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 -- 172.21.15.66:0/3301359962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 msgr2=0x7f180c19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 0x7f180c19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 -- 172.21.15.66:0/3301359962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 msgr2=0x7f180c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 0x7f180c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180b7fe700 1 -- 172.21.15.66:0/3301359962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0095f0 con 0x7f180c0ffae0 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.542+0000 7f180bfff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 0x7f180c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f180affd700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 0x7f180c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f180b7fe700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c197ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005740 tx=0x7f17fc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01d070 con 0x7f180c0ffae0 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17fc021410 con 0x7f180c0ffae0 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc017650 con 0x7f180c0ffae0 2023-07-21T22:19:28.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c19fb80 con 0x7f180c0ffae0 2023-07-21T22:19:28.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.543+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c19ffa0 con 0x7f180c0ffae0 2023-07-21T22:19:28.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.544+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c1921c0 con 0x7f180c0ffae0 2023-07-21T22:19:28.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.546+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f17fc021810 con 0x7f180c0ffae0 2023-07-21T22:19:28.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.549+0000 7f1808ff9700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f17f40a11d0 0x7f17f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:28.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.549+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17fc0cdfe0 con 0x7f180c0ffae0 2023-07-21T22:19:28.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.549+0000 7f180affd700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f17f40a11d0 0x7f17f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:28.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.550+0000 7f180affd700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f17f40a11d0 0x7f17f40a3680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1800009cb0 tx=0x7f1800009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:28.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.553+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f17fc097590 con 0x7f180c0ffae0 2023-07-21T22:19:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:28 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3742302879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:19:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:28 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2811885384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:19:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:28 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3742302879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:19:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:28 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2811885384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:19:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:28 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3742302879' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:19:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:28 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2811885384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:19:28.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.784+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f180c02c850 con 0x7f180c0ffae0 2023-07-21T22:19:28.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.785+0000 7f1808ff9700 1 -- 172.21.15.66:0/3301359962 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) v1 ==== 81+0+588 (secure 0 0 0) 0x7f17fc095e60 con 0x7f180c0ffae0 2023-07-21T22:19:28.830 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/662946744 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:28.830 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3065709937 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:28.830 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1244827991 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/979491945 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3065709937 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3161416321 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3483118396 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/979491945 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/690385827 2023-07-22T22:12:09.674594+0000 2023-07-21T22:19:28.831 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3486180168 2023-07-22T22:12:36.098613+0000 2023-07-21T22:19:28.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.787+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f17f40a11d0 msgr2=0x7f17f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.787+0000 7f18121ff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f17f40a11d0 0x7f17f40a3680 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1800009cb0 tx=0x7f1800009260 comp rx=0 tx=0).stop 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.787+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 msgr2=0x7f180c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.787+0000 7f18121ff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c197ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005740 tx=0x7f17fc009510 comp rx=0 tx=0).stop 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 shutdown_connections 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f17f40a11d0 0x7f17f40a3680 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f180c101660 0x7f180c19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f180c0ffae0 0x7f180c197ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 --2- 172.21.15.66:0/3301359962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f180c100cd0 0x7f180c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:28.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 >> 172.21.15.66:0/3301359962 conn(0x7f180c0fb0e0 msgr2=0x7f180c1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:28.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 shutdown_connections 2023-07-21T22:19:28.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:28.788+0000 7f18121ff700 1 -- 172.21.15.66:0/3301359962 wait complete. 2023-07-21T22:19:28.834 INFO:teuthology.orchestra.run.smithi066.stderr:listed 10 entries 2023-07-21T22:19:28.850 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi066.front.sepia.ceph.com... 2023-07-21T22:19:28.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:28.851 DEBUG:teuthology.orchestra.run.smithi066:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-07-21T22:19:28.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi066.stderr:ceph-fuse[163000]: fuse finished with error 0 and tester_r 0 2023-07-21T22:19:28.923 INFO:teuthology.orchestra.run:waiting for 300 2023-07-21T22:19:28.960 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:19:28.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:19:28.960 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-07-21T22:19:29.056 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:29.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 -- 172.21.15.66:0/3388595513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4101680 msgr2=0x7f27d4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 --2- 172.21.15.66:0/3388595513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4101680 0x7f27d4107f40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00b0a0 tx=0x7f27cc00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 -- 172.21.15.66:0/3388595513 shutdown_connections 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 --2- 172.21.15.66:0/3388595513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27d40ffb00 0x7f27d40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 --2- 172.21.15.66:0/3388595513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4101680 0x7f27d4107f40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 --2- 172.21.15.66:0/3388595513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d4100cf0 0x7f27d4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 -- 172.21.15.66:0/3388595513 >> 172.21.15.66:0/3388595513 conn(0x7f27d40fb0e0 msgr2=0x7f27d40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 -- 172.21.15.66:0/3388595513 shutdown_connections 2023-07-21T22:19:29.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.220+0000 7f27d9dff700 1 -- 172.21.15.66:0/3388595513 wait complete. 2023-07-21T22:19:29.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.221+0000 7f27d9dff700 1 Processor -- start 2023-07-21T22:19:29.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.221+0000 7f27d9dff700 1 -- start start 2023-07-21T22:19:29.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d9dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d40ffb00 0x7f27d40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d9dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 0x7f27d4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d9dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27d4101680 0x7f27d4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d9dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d40735e0 con 0x7f27d4100cf0 2023-07-21T22:19:29.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d9dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d4073750 con 0x7f27d4101680 2023-07-21T22:19:29.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d9dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d419f9a0 con 0x7f27d40ffb00 2023-07-21T22:19:29.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 0x7f27d4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 0x7f27d4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37478/0 (socket says 172.21.15.66:37478) 2023-07-21T22:19:29.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d40ffb00 0x7f27d40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 -- 172.21.15.66:0/2361741634 learned_addr learned my addr 172.21.15.66:0/2361741634 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:29.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d3fff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27d4101680 0x7f27d4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 -- 172.21.15.66:0/2361741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d40ffb00 msgr2=0x7f27d40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d40ffb00 0x7f27d40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 -- 172.21.15.66:0/2361741634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27d4101680 msgr2=0x7f27d4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27d4101680 0x7f27d4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27cbfff700 1 -- 172.21.15.66:0/2361741634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc009d00 con 0x7f27d4100cf0 2023-07-21T22:19:29.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.222+0000 7f27d37fe700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d40ffb00 0x7f27d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:29.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.223+0000 7f27cbfff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 0x7f27d4072900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e9a0 tx=0x7f27bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:29.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.223+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc009790 con 0x7f27d4100cf0 2023-07-21T22:19:29.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.223+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27bc010430 con 0x7f27d4100cf0 2023-07-21T22:19:29.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.223+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27d419fc80 con 0x7f27d4100cf0 2023-07-21T22:19:29.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.223+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc018680 con 0x7f27d4100cf0 2023-07-21T22:19:29.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.223+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27d41a01b0 con 0x7f27d4100cf0 2023-07-21T22:19:29.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.224+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27d410b330 con 0x7f27d4100cf0 2023-07-21T22:19:29.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.226+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f27bc0105a0 con 0x7f27d4100cf0 2023-07-21T22:19:29.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.228+0000 7f27d17fa700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27c00a1220 0x7f27c00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.228+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f27bc014070 con 0x7f27d4100cf0 2023-07-21T22:19:29.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.228+0000 7f27d37fe700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27c00a1220 0x7f27c00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.229+0000 7f27d37fe700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27c00a1220 0x7f27c00a36d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27d4100b50 tx=0x7f27c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:29.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.229+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f27bc0963c0 con 0x7f27d4100cf0 2023-07-21T22:19:29.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:29 smithi066 ceph-mon[126379]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:29.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:29 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/195090548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:19:29.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:29 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3301359962' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:19:29.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.443+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27d410b550 con 0x7f27d4100cf0 2023-07-21T22:19:29.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.445+0000 7f27d17fa700 1 -- 172.21.15.66:0/2361741634 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7f27bc026020 con 0x7f27d4100cf0 2023-07-21T22:19:29.490 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:29.491 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:29.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27c00a1220 msgr2=0x7f27c00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27c00a1220 0x7f27c00a36d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27d4100b50 tx=0x7f27c4009260 comp rx=0 tx=0).stop 2023-07-21T22:19:29.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 msgr2=0x7f27d4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 0x7f27d4072900 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e9a0 tx=0x7f27bc00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 shutdown_connections 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27c00a1220 0x7f27c00a36d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27d4101680 0x7f27d4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27d40ffb00 0x7f27d40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 --2- 172.21.15.66:0/2361741634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f27d4100cf0 0x7f27d4072900 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 >> 172.21.15.66:0/2361741634 conn(0x7f27d40fb0e0 msgr2=0x7f27d40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:29.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 shutdown_connections 2023-07-21T22:19:29.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.447+0000 7f27d9dff700 1 -- 172.21.15.66:0/2361741634 wait complete. 2023-07-21T22:19:29.512 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:29.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:29 smithi104 ceph-mon[129021]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:29.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:29 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/195090548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:19:29.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:29 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3301359962' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:19:29.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 -- 172.21.15.66:0/1991799831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94100cf0 msgr2=0x7fbc94101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 --2- 172.21.15.66:0/1991799831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94100cf0 0x7fbc94101140 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009990 tx=0x7fbc80009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 -- 172.21.15.66:0/1991799831 shutdown_connections 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 --2- 172.21.15.66:0/1991799831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 0x7fbc940fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 --2- 172.21.15.66:0/1991799831 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94101680 0x7fbc94107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 --2- 172.21.15.66:0/1991799831 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94100cf0 0x7fbc94101140 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 -- 172.21.15.66:0/1991799831 >> 172.21.15.66:0/1991799831 conn(0x7fbc940fb0e0 msgr2=0x7fbc940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 -- 172.21.15.66:0/1991799831 shutdown_connections 2023-07-21T22:19:29.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.675+0000 7fbc9afff700 1 -- 172.21.15.66:0/1991799831 wait complete. 2023-07-21T22:19:29.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.676+0000 7fbc9afff700 1 Processor -- start 2023-07-21T22:19:29.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 -- start start 2023-07-21T22:19:29.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 0x7fbc94197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94100cf0 0x7fbc94197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 0x7fbc9419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9419f590 con 0x7fbc94101680 2023-07-21T22:19:29.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9419f6d0 con 0x7fbc940ffb00 2023-07-21T22:19:29.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9afff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9419f860 con 0x7fbc94100cf0 2023-07-21T22:19:29.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9959c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 0x7fbc9419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc93fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94100cf0 0x7fbc94197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9959c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 0x7fbc9419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37488/0 (socket says 172.21.15.66:37488) 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc9959c700 1 -- 172.21.15.66:0/3209874967 learned_addr learned my addr 172.21.15.66:0/3209874967 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.677+0000 7fbc98d9b700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 0x7fbc94197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9959c700 1 -- 172.21.15.66:0/3209874967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94100cf0 msgr2=0x7fbc94197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9959c700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94100cf0 0x7fbc94197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9959c700 1 -- 172.21.15.66:0/3209874967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 msgr2=0x7fbc94197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9959c700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 0x7fbc94197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9959c700 1 -- 172.21.15.66:0/3209874967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc88009520 con 0x7fbc94101680 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc93fff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94100cf0 0x7fbc94197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc98d9b700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 0x7fbc94197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9959c700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 0x7fbc9419efc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00ef30 tx=0x7fbc8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c018070 con 0x7fbc94101680 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc800095f0 con 0x7fbc94101680 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc8c00f040 con 0x7fbc94101680 2023-07-21T22:19:29.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c00cd00 con 0x7fbc94101680 2023-07-21T22:19:29.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.678+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9419feb0 con 0x7fbc94101680 2023-07-21T22:19:29.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.679+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9410b320 con 0x7fbc94101680 2023-07-21T22:19:29.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.681+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc8c003a00 con 0x7fbc94101680 2023-07-21T22:19:29.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.683+0000 7fbc91ffb700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbc840a11a0 0x7fbc840a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:29.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.683+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbc8c0cc680 con 0x7fbc94101680 2023-07-21T22:19:29.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.683+0000 7fbc98d9b700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbc840a11a0 0x7fbc840a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:29.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.684+0000 7fbc98d9b700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbc840a11a0 0x7fbc840a3650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009fd0 tx=0x7fbc88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:29.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.687+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbc8c095c10 con 0x7fbc94101680 2023-07-21T22:19:29.738 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:29 smithi149 ceph-mon[129268]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:29.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:29 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/195090548' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:19:29.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:29 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3301359962' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:19:29.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.898+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc9410b4e0 con 0x7fbc94101680 2023-07-21T22:19:29.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.899+0000 7fbc91ffb700 1 -- 172.21.15.66:0/3209874967 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+23970 (secure 0 0 0) 0x7fbc8c099230 con 0x7fbc94101680 2023-07-21T22:19:29.945 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:29.946 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":89,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:11.902791+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:29.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbc840a11a0 msgr2=0x7fbc840a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbc840a11a0 0x7fbc840a3650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009fd0 tx=0x7fbc88009260 comp rx=0 tx=0).stop 2023-07-21T22:19:29.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 msgr2=0x7fbc9419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:29.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 0x7fbc9419efc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00ef30 tx=0x7fbc8c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 shutdown_connections 2023-07-21T22:19:29.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbc840a11a0 0x7fbc840a3650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc940ffb00 0x7fbc94197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbc94101680 0x7fbc9419efc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 --2- 172.21.15.66:0/3209874967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc94100cf0 0x7fbc94197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:29.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 >> 172.21.15.66:0/3209874967 conn(0x7fbc940fb0e0 msgr2=0x7fbc94106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:29.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 shutdown_connections 2023-07-21T22:19:29.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:29.902+0000 7fbc9afff700 1 -- 172.21.15.66:0/3209874967 wait complete. 2023-07-21T22:19:29.967 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-21T22:19:30.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.134+0000 7f5700bff700 1 -- 172.21.15.66:0/2057204726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc101670 msgr2=0x7f56fc107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.134+0000 7f5700bff700 1 --2- 172.21.15.66:0/2057204726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc101670 0x7f56fc107f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009990 tx=0x7f56f4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 -- 172.21.15.66:0/2057204726 shutdown_connections 2023-07-21T22:19:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 --2- 172.21.15.66:0/2057204726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc101670 0x7f56fc107f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 --2- 172.21.15.66:0/2057204726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 --2- 172.21.15.66:0/2057204726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc100ce0 0x7f56fc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:30.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 -- 172.21.15.66:0/2057204726 >> 172.21.15.66:0/2057204726 conn(0x7f56fc0fb130 msgr2=0x7f56fc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 -- 172.21.15.66:0/2057204726 shutdown_connections 2023-07-21T22:19:30.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.135+0000 7f5700bff700 1 -- 172.21.15.66:0/2057204726 wait complete. 2023-07-21T22:19:30.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 Processor -- start 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 -- start start 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc101670 0x7f56fc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19f570 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19f6b0 con 0x7f56fc101670 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.136+0000 7f5700bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc19f840 con 0x7f56fc100ce0 2023-07-21T22:19:30.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37504/0 (socket says 172.21.15.66:37504) 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 -- 172.21.15.66:0/1167284249 learned_addr learned my addr 172.21.15.66:0/1167284249 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fad9d700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc101670 0x7f56fc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56f9d9b700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 -- 172.21.15.66:0/1167284249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 msgr2=0x7f56fc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:30.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 -- 172.21.15.66:0/1167284249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc101670 msgr2=0x7f56fc19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc101670 0x7f56fc19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 -- 172.21.15.66:0/1167284249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec009c40 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fad9d700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc101670 0x7f56fc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56f9d9b700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56fa59c700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc1979f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c140 tx=0x7f56ec00c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.137+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00e050 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.138+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f40095f0 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.138+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56ec00f040 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.138+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec013660 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.138+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc19fe40 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.139+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc10b320 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.141+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f56ec007500 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.142+0000 7f56eb7fe700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56e40a1150 0x7f56e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.142+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f56ec0cca60 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.143+0000 7f56f9d9b700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56e40a1150 0x7f56e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:30.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.143+0000 7f56f9d9b700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56e40a1150 0x7f56e40a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009cb0 tx=0x7f56f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:30.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.146+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f56ec095f60 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:30.403+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56fc10b510 con 0x7f56fc0ffaf0 2023-07-21T22:19:30.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:30 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2361741634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:30.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:30 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3209874967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:30.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:30 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2361741634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:30.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:30 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3209874967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:30.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:30 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2361741634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:30.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:30 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3209874967' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:31.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.268+0000 7f56eb7fe700 1 -- 172.21.15.66:0/1167284249 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56ec099580 con 0x7f56fc0ffaf0 2023-07-21T22:19:31.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56e40a1150 msgr2=0x7f56e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:31.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56e40a1150 0x7f56e40a3600 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009cb0 tx=0x7f56f0009260 comp rx=0 tx=0).stop 2023-07-21T22:19:31.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 msgr2=0x7f56fc1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:31.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc1979f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00c140 tx=0x7f56ec00c500 comp rx=0 tx=0).stop 2023-07-21T22:19:31.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 shutdown_connections 2023-07-21T22:19:31.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56e40a1150 0x7f56e40a3600 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.271+0000 7f5700bff700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56fc101670 0x7f56fc19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.272+0000 7f5700bff700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56fc0ffaf0 0x7f56fc1979f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.272+0000 7f5700bff700 1 --2- 172.21.15.66:0/1167284249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56fc100ce0 0x7f56fc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.272+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 >> 172.21.15.66:0/1167284249 conn(0x7f56fc0fb130 msgr2=0x7f56fc106580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:31.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.272+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 shutdown_connections 2023-07-21T22:19:31.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.272+0000 7f5700bff700 1 -- 172.21.15.66:0/1167284249 wait complete. 2023-07-21T22:19:31.338 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-07-21T22:19:31.339 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-21T22:19:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/3196802645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54101660 msgr2=0x7f8f54107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:31.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/3196802645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54101660 0x7f8f54107f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c009f90 tx=0x7f8f4c00b330 comp rx=0 tx=0).stop 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/3196802645 shutdown_connections 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/3196802645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54101660 0x7f8f54107f20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/3196802645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f540ffae0 0x7f8f540ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/3196802645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f54100cd0 0x7f8f54101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/3196802645 >> 172.21.15.66:0/3196802645 conn(0x7f8f540fb0e0 msgr2=0x7f8f540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/3196802645 shutdown_connections 2023-07-21T22:19:31.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.513+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/3196802645 wait complete. 2023-07-21T22:19:31.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.514+0000 7f8f5b3ff700 1 Processor -- start 2023-07-21T22:19:31.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 -- start start 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f540ffae0 0x7f8f54197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 0x7f8f541980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f54101660 0x7f8f5419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5419f6e0 con 0x7f8f54100cd0 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5419f820 con 0x7f8f540ffae0 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5b3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5419f9b0 con 0x7f8f54101660 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5899a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 0x7f8f541980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:31.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5899a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 0x7f8f541980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:37526/0 (socket says 172.21.15.66:37526) 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5899a700 1 -- 172.21.15.66:0/1283114120 learned_addr learned my addr 172.21.15.66:0/1283114120 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5919b700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f540ffae0 0x7f8f54197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.515+0000 7f8f5999c700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f54101660 0x7f8f5419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5899a700 1 -- 172.21.15.66:0/1283114120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f54101660 msgr2=0x7f8f5419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5899a700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f54101660 0x7f8f5419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5899a700 1 -- 172.21.15.66:0/1283114120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f540ffae0 msgr2=0x7f8f54197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5899a700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f540ffae0 0x7f8f54197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:31.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5899a700 1 -- 172.21.15.66:0/1283114120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500095f0 con 0x7f8f54100cd0 2023-07-21T22:19:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5899a700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 0x7f8f541980a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e9a0 tx=0x7f8f4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44009790 con 0x7f8f54100cd0 2023-07-21T22:19:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4400cbd0 con 0x7f8f54100cd0 2023-07-21T22:19:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c009c40 con 0x7f8f54100cd0 2023-07-21T22:19:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f44018680 con 0x7f8f54100cd0 2023-07-21T22:19:31.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.516+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f541a0060 con 0x7f8f54100cd0 2023-07-21T22:19:31.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.517+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5410b310 con 0x7f8f54100cd0 2023-07-21T22:19:31.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.519+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8f4400cd40 con 0x7f8f54100cd0 2023-07-21T22:19:31.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.521+0000 7f8f4a7fc700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f400a11d0 0x7f8f400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:31.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.521+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8f44014070 con 0x7f8f54100cd0 2023-07-21T22:19:31.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.521+0000 7f8f5919b700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f400a11d0 0x7f8f400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:31.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.522+0000 7f8f5919b700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f400a11d0 0x7f8f400a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f54100b30 tx=0x7f8f50009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:31.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.525+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8f44051770 con 0x7f8f54100cd0 2023-07-21T22:19:31.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:31 smithi104 ceph-mon[129021]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:31.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1167284249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:19:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:31 smithi066 ceph-mon[126379]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1167284249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:19:31.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:31 smithi149 ceph-mon[129268]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:31.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1167284249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T22:19:31.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:31.783+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f8f541a0610 con 0x7f8f54100cd0 2023-07-21T22:19:32.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.278+0000 7f8f4a7fc700 1 -- 172.21.15.66:0/1283114120 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8f44096280 con 0x7f8f54100cd0 2023-07-21T22:19:32.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f400a11d0 msgr2=0x7f8f400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f400a11d0 0x7f8f400a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8f54100b30 tx=0x7f8f50009310 comp rx=0 tx=0).stop 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 msgr2=0x7f8f541980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 0x7f8f541980a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f4400e9a0 tx=0x7f8f4400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 shutdown_connections 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f400a11d0 0x7f8f400a3680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f540ffae0 0x7f8f54197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f54101660 0x7f8f5419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 --2- 172.21.15.66:0/1283114120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f54100cd0 0x7f8f541980a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.281+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 >> 172.21.15.66:0/1283114120 conn(0x7f8f540fb0e0 msgr2=0x7f8f540fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.282+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 shutdown_connections 2023-07-21T22:19:32.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.282+0000 7f8f5b3ff700 1 -- 172.21.15.66:0/1283114120 wait complete. 2023-07-21T22:19:32.350 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-21T22:19:32.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 -- 172.21.15.66:0/250618643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f8100cd0 msgr2=0x7fa2f8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:32.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/250618643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8101120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009990 tx=0x7fa2ec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 -- 172.21.15.66:0/250618643 shutdown_connections 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/250618643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8101660 0x7fa2f8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/250618643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f80ffae0 0x7fa2f80ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/250618643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8101120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 -- 172.21.15.66:0/250618643 >> 172.21.15.66:0/250618643 conn(0x7fa2f80fb0e0 msgr2=0x7fa2f80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 -- 172.21.15.66:0/250618643 shutdown_connections 2023-07-21T22:19:32.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.516+0000 7fa2fcdff700 1 -- 172.21.15.66:0/250618643 wait complete. 2023-07-21T22:19:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.517+0000 7fa2fcdff700 1 Processor -- start 2023-07-21T22:19:32.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 -- start start 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 0x7fa2f8105860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8105da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f8101660 0x7fa2f81023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8102b90 con 0x7fa2f80ffae0 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8102d00 con 0x7fa2f8100cd0 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2fcdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f8102ec0 con 0x7fa2f8101660 2023-07-21T22:19:32.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 0x7fa2f8105860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 0x7fa2f8105860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46874/0 (socket says 172.21.15.66:46874) 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 -- 172.21.15.66:0/3680938152 learned_addr learned my addr 172.21.15.66:0/3680938152 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f6d9d700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f8101660 0x7fa2f81023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f5d9b700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 -- 172.21.15.66:0/3680938152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f8101660 msgr2=0x7fa2f81023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f8101660 0x7fa2f81023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 -- 172.21.15.66:0/3680938152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8100cd0 msgr2=0x7fa2f8105da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:32.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8105da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f659c700 1 -- 172.21.15.66:0/3680938152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0095f0 con 0x7fa2f80ffae0 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f5d9b700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.518+0000 7fa2f6d9d700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f8101660 0x7fa2f81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.519+0000 7fa2f659c700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 0x7fa2f8105860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800e970 tx=0x7fa2e800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.519+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8009790 con 0x7fa2f80ffae0 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.519+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e80044e0 con 0x7fa2f80ffae0 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.519+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f81a3ff0 con 0x7fa2f80ffae0 2023-07-21T22:19:32.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.519+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8003e30 con 0x7fa2f80ffae0 2023-07-21T22:19:32.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.519+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f81a44c0 con 0x7fa2f80ffae0 2023-07-21T22:19:32.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.520+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f81921c0 con 0x7fa2f80ffae0 2023-07-21T22:19:32.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.522+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa2e8003f90 con 0x7fa2f80ffae0 2023-07-21T22:19:32.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.524+0000 7fa2e77fe700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa2e00a11d0 0x7fa2e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:32.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.524+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa2e8014070 con 0x7fa2f80ffae0 2023-07-21T22:19:32.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.524+0000 7fa2f5d9b700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa2e00a11d0 0x7fa2e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:32.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.525+0000 7fa2f5d9b700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa2e00a11d0 0x7fa2e00a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00b2e0 tx=0x7fa2ec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:32.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.527+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa2e80962e0 con 0x7fa2f80ffae0 2023-07-21T22:19:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:32 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1167284249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:19:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:32 smithi104 ceph-mon[129021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:32.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:32 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1283114120' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-21T22:19:32.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:32 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1167284249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:19:32.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:32 smithi066 ceph-mon[126379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:32.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:32 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1283114120' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-21T22:19:32.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:32 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1167284249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T22:19:32.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:32 smithi149 ceph-mon[129268]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:32.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:32 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1283114120' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-21T22:19:32.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:32.786+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa2f802c850 con 0x7fa2f80ffae0 2023-07-21T22:19:33.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.285+0000 7fa2e77fe700 1 -- 172.21.15.66:0/3680938152 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa2e8099900 con 0x7fa2f80ffae0 2023-07-21T22:19:33.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa2e00a11d0 msgr2=0x7fa2e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa2e00a11d0 0x7fa2e00a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00b2e0 tx=0x7fa2ec005fd0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 msgr2=0x7fa2f8105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 0x7fa2f8105860 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800e970 tx=0x7fa2e800ed30 comp rx=0 tx=0).stop 2023-07-21T22:19:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 shutdown_connections 2023-07-21T22:19:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa2e00a11d0 0x7fa2e00a3680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa2f8101660 0x7fa2f81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa2f80ffae0 0x7fa2f8105860 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 --2- 172.21.15.66:0/3680938152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2f8100cd0 0x7fa2f8105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 >> 172.21.15.66:0/3680938152 conn(0x7fa2f80fb0e0 msgr2=0x7fa2f81065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 shutdown_connections 2023-07-21T22:19:33.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.288+0000 7fa2fcdff700 1 -- 172.21.15.66:0/3680938152 wait complete. 2023-07-21T22:19:33.358 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-21T22:19:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 -- 172.21.15.66:0/3581410800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fe3b0 msgr2=0x7f51c00fe7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3581410800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fe3b0 0x7f51c00fe7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009990 tx=0x7f51bc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 -- 172.21.15.66:0/3581410800 shutdown_connections 2023-07-21T22:19:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3581410800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51c00ff6b0 0x7f51c00ffb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3581410800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fe3b0 0x7f51c00fe7c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3581410800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51c00fed00 0x7f51c00ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 -- 172.21.15.66:0/3581410800 >> 172.21.15.66:0/3581410800 conn(0x7f51c00fa000 msgr2=0x7f51c00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 -- 172.21.15.66:0/3581410800 shutdown_connections 2023-07-21T22:19:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.526+0000 7f51c7bff700 1 -- 172.21.15.66:0/3581410800 wait complete. 2023-07-21T22:19:33.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.527+0000 7f51c7bff700 1 Processor -- start 2023-07-21T22:19:33.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.527+0000 7f51c7bff700 1 -- start start 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c7bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51c00fe3b0 0x7f51c0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c7bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 0x7f51c0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c7bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51c00ff6b0 0x7f51c019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c7bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c019f5d0 con 0x7f51c00fed00 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c7bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c019f710 con 0x7f51c00ff6b0 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c7bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c019f8a0 con 0x7f51c00fe3b0 2023-07-21T22:19:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 0x7f51c0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 0x7f51c0197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46882/0 (socket says 172.21.15.66:46882) 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 -- 172.21.15.66:0/3070538918 learned_addr learned my addr 172.21.15.66:0/3070538918 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c599b700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51c00fe3b0 0x7f51c0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c619c700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51c00ff6b0 0x7f51c019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 -- 172.21.15.66:0/3070538918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51c00fe3b0 msgr2=0x7f51c0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51c00fe3b0 0x7f51c0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 -- 172.21.15.66:0/3070538918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51c00ff6b0 msgr2=0x7f51c019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51c00ff6b0 0x7f51c019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.528+0000 7f51c519a700 1 -- 172.21.15.66:0/3070538918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc0095f0 con 0x7f51c00fed00 2023-07-21T22:19:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.529+0000 7f51c519a700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 0x7f51c0197f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f51b000e9a0 tx=0x7f51b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.529+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0009790 con 0x7f51c00fed00 2023-07-21T22:19:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.529+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b0010430 con 0x7f51c00fed00 2023-07-21T22:19:33.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.529+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c019fb80 con 0x7f51c00fed00 2023-07-21T22:19:33.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.529+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0018680 con 0x7f51c00fed00 2023-07-21T22:19:33.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.529+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c01a00e0 con 0x7f51c00fed00 2023-07-21T22:19:33.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.530+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c0109190 con 0x7f51c00fed00 2023-07-21T22:19:33.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.532+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f51b00105a0 con 0x7f51c00fed00 2023-07-21T22:19:33.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.533+0000 7f51b6ffd700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51ac0a11d0 0x7f51ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:33.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.534+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f51b0014070 con 0x7f51c00fed00 2023-07-21T22:19:33.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.534+0000 7f51c599b700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51ac0a11d0 0x7f51ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:33.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.534+0000 7f51c599b700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51ac0a11d0 0x7f51ac0a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b2e0 tx=0x7f51bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:33.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.538+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f51b0051770 con 0x7f51c00fed00 2023-07-21T22:19:33.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:33.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:19:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1283114120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-21T22:19:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:19:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T22:19:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T22:19:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:33 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3680938152' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:19:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:19:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1283114120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-21T22:19:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:19:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T22:19:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T22:19:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:33 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3680938152' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:19:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-07-21T22:19:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T22:19:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1283114120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-21T22:19:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T22:19:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T22:19:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T22:19:33.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T22:19:33.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:33 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3680938152' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T22:19:33.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:33.799+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f51c002c850 con 0x7f51c00fed00 2023-07-21T22:19:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.298+0000 7f51b6ffd700 1 -- 172.21.15.66:0/3070538918 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f51b00961b0 con 0x7f51c00fed00 2023-07-21T22:19:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51ac0a11d0 msgr2=0x7f51ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51ac0a11d0 0x7f51ac0a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00b2e0 tx=0x7f51bc005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 msgr2=0x7f51c0197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 0x7f51c0197f90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f51b000e9a0 tx=0x7f51b000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 shutdown_connections 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f51ac0a11d0 0x7f51ac0a3680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51c00ff6b0 0x7f51c019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51c00fe3b0 0x7f51c0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 --2- 172.21.15.66:0/3070538918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51c00fed00 0x7f51c0197f90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 >> 172.21.15.66:0/3070538918 conn(0x7f51c00fa000 msgr2=0x7f51c00fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 shutdown_connections 2023-07-21T22:19:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.301+0000 7f51c7bff700 1 -- 172.21.15.66:0/3070538918 wait complete. 2023-07-21T22:19:34.368 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-07-21T22:19:34.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-21T22:19:34.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 -- 172.21.15.66:0/2580913607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 msgr2=0x7f2b8c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2580913607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 0x7f2b8c101120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009990 tx=0x7f2b7c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 -- 172.21.15.66:0/2580913607 shutdown_connections 2023-07-21T22:19:34.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2580913607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c0ffae0 0x7f2b8c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2580913607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2580913607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 0x7f2b8c101120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 -- 172.21.15.66:0/2580913607 >> 172.21.15.66:0/2580913607 conn(0x7f2b8c0fb0e0 msgr2=0x7f2b8c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:34.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 -- 172.21.15.66:0/2580913607 shutdown_connections 2023-07-21T22:19:34.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.537+0000 7f2b947ff700 1 -- 172.21.15.66:0/2580913607 wait complete. 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.538+0000 7f2b947ff700 1 Processor -- start 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.538+0000 7f2b947ff700 1 -- start start 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.538+0000 7f2b947ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c0ffae0 0x7f2b8c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b947ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 0x7f2b8c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b947ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b947ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c19f580 con 0x7f2b8c101660 2023-07-21T22:19:34.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b947ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c19f6c0 con 0x7f2b8c0ffae0 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b947ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c19f850 con 0x7f2b8c100cd0 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46900/0 (socket says 172.21.15.66:46900) 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 -- 172.21.15.66:0/2235849235 learned_addr learned my addr 172.21.15.66:0/2235849235 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b91d9a700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 0x7f2b8c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b9259b700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c0ffae0 0x7f2b8c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:34.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 -- 172.21.15.66:0/2235849235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 msgr2=0x7f2b8c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 0x7f2b8c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 -- 172.21.15.66:0/2235849235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c0ffae0 msgr2=0x7f2b8c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c0ffae0 0x7f2b8c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.539+0000 7f2b92d9c700 1 -- 172.21.15.66:0/2235849235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b88009520 con 0x7f2b8c101660 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.540+0000 7f2b92d9c700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c19efb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400ef30 tx=0x7f2b8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.540+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84018070 con 0x7f2b8c101660 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.540+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c0095f0 con 0x7f2b8c101660 2023-07-21T22:19:34.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.540+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b8400f040 con 0x7f2b8c101660 2023-07-21T22:19:34.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.540+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8400cd00 con 0x7f2b8c101660 2023-07-21T22:19:34.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.540+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c19fe50 con 0x7f2b8c101660 2023-07-21T22:19:34.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.541+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c10b2a0 con 0x7f2b8c101660 2023-07-21T22:19:34.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.543+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2b84003a00 con 0x7f2b8c101660 2023-07-21T22:19:34.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.544+0000 7f2b837fe700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2b780a1150 0x7f2b780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:34.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.545+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2b840cc640 con 0x7f2b8c101660 2023-07-21T22:19:34.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.545+0000 7f2b9259b700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2b780a1150 0x7f2b780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:34.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.545+0000 7f2b9259b700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2b780a1150 0x7f2b780a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c100b30 tx=0x7f2b88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:34.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.548+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2b84095bd0 con 0x7f2b8c101660 2023-07-21T22:19:34.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] up:active 2023-07-21T22:19:34.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3680938152' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: Cluster is now healthy 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T22:19:34.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3070538918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:19:34.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] up:active 2023-07-21T22:19:34.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3680938152' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:19:34.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:19:34.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:19:34.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: Cluster is now healthy 2023-07-21T22:19:34.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T22:19:34.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T22:19:34.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T22:19:34.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3070538918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:19:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] up:active 2023-07-21T22:19:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3680938152' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T22:19:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T22:19:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T22:19:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: Cluster is now healthy 2023-07-21T22:19:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T22:19:34.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T22:19:34.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T22:19:34.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3070538918' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T22:19:34.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.793+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b8c107670 con 0x7f2b780a1150 2023-07-21T22:19:34.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.843+0000 7f2b837fe700 1 -- 172.21.15.66:0/2235849235 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2b8c107670 con 0x7f2b780a1150 2023-07-21T22:19:34.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2b780a1150 msgr2=0x7f2b780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2b780a1150 0x7f2b780a3600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c100b30 tx=0x7f2b88009260 comp rx=0 tx=0).stop 2023-07-21T22:19:34.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 msgr2=0x7f2b8c19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c19efb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400ef30 tx=0x7f2b8400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 shutdown_connections 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2b780a1150 0x7f2b780a3600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b8c0ffae0 0x7f2b8c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2b8c101660 0x7f2b8c19efb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 --2- 172.21.15.66:0/2235849235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b8c100cd0 0x7f2b8c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:34.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 >> 172.21.15.66:0/2235849235 conn(0x7f2b8c0fb0e0 msgr2=0x7f2b8c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:34.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 shutdown_connections 2023-07-21T22:19:34.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:34.845+0000 7f2b947ff700 1 -- 172.21.15.66:0/2235849235 wait complete. 2023-07-21T22:19:34.909 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-21T22:19:35.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 -- 172.21.15.66:0/1209438634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 msgr2=0x7ff480105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 --2- 172.21.15.66:0/1209438634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 0x7ff480105260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff474009990 tx=0x7ff474009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 -- 172.21.15.66:0/1209438634 shutdown_connections 2023-07-21T22:19:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 --2- 172.21.15.66:0/1209438634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff4801057a0 0x7ff480107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 --2- 172.21.15.66:0/1209438634 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff480100560 0x7ff480102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 --2- 172.21.15.66:0/1209438634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 0x7ff480105260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.073+0000 7ff4849ff700 1 -- 172.21.15.66:0/1209438634 >> 172.21.15.66:0/1209438634 conn(0x7ff4800fa000 msgr2=0x7ff4800fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:35.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.074+0000 7ff4849ff700 1 -- 172.21.15.66:0/1209438634 shutdown_connections 2023-07-21T22:19:35.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.074+0000 7ff4849ff700 1 -- 172.21.15.66:0/1209438634 wait complete. 2023-07-21T22:19:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 Processor -- start 2023-07-21T22:19:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 -- start start 2023-07-21T22:19:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 0x7ff480197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 0x7ff480197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4801057a0 0x7ff48019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019f5a0 con 0x7ff4801057a0 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019f6e0 con 0x7ff480102e80 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff4849ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48019f870 con 0x7ff480100560 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff47e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 0x7ff480197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff47e59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 0x7ff480197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49292/0 (socket says 172.21.15.66:49292) 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.075+0000 7ff47e59c700 1 -- 172.21.15.66:0/4071246062 learned_addr learned my addr 172.21.15.66:0/4071246062 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47dd9b700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 0x7ff480197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47ed9d700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4801057a0 0x7ff48019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47e59c700 1 -- 172.21.15.66:0/4071246062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 msgr2=0x7ff480197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47e59c700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 0x7ff480197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47e59c700 1 -- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4801057a0 msgr2=0x7ff48019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47e59c700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4801057a0 0x7ff48019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47e59c700 1 -- 172.21.15.66:0/4071246062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4740095f0 con 0x7ff480100560 2023-07-21T22:19:35.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff47e59c700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 0x7ff480197a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e9a0 tx=0x7ff47000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.076+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470009790 con 0x7ff480100560 2023-07-21T22:19:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.077+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48019fb50 con 0x7ff480100560 2023-07-21T22:19:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.077+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff470010430 con 0x7ff480100560 2023-07-21T22:19:35.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.077+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470018680 con 0x7ff480100560 2023-07-21T22:19:35.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.077+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4801a00b0 con 0x7ff480100560 2023-07-21T22:19:35.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.078+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff480107360 con 0x7ff480100560 2023-07-21T22:19:35.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.080+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff4700105a0 con 0x7ff480100560 2023-07-21T22:19:35.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.082+0000 7ff46f7fe700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff4680a11d0 0x7ff4680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.082+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff470014070 con 0x7ff480100560 2023-07-21T22:19:35.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.082+0000 7ff47dd9b700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff4680a11d0 0x7ff4680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.083+0000 7ff47dd9b700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff4680a11d0 0x7ff4680a3680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff47400b680 tx=0x7ff474005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:35.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.084+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff4700963c0 con 0x7ff480100560 2023-07-21T22:19:35.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:19:34.801+0000 7f7bc9514700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:19:35.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:19:34.801+0000 7f7bc9514700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:19:35.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:19:34.801+0000 7f7bc9514700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:19:35.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:19:34.801+0000 7f7bc9514700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:19:35.196 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:19:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:19:34.801+0000 7f7bc9514700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:19:35.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.332+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7ff480060950 con 0x7ff4680a11d0 2023-07-21T22:19:35.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.337+0000 7ff46f7fe700 1 -- 172.21.15.66:0/4071246062 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7ff480060950 con 0x7ff4680a11d0 2023-07-21T22:19:35.382 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv_0/e05711b4-48c0-4866-a5e0-ffd7e4225197 2023-07-21T22:19:35.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff4680a11d0 msgr2=0x7ff4680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff4680a11d0 0x7ff4680a3680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff47400b680 tx=0x7ff474005fd0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 msgr2=0x7ff480197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 0x7ff480197a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff47000e9a0 tx=0x7ff47000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 shutdown_connections 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff4680a11d0 0x7ff4680a3680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff4801057a0 0x7ff48019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff480100560 0x7ff480197a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 --2- 172.21.15.66:0/4071246062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff480102e80 0x7ff480197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 >> 172.21.15.66:0/4071246062 conn(0x7ff4800fa000 msgr2=0x7ff4800febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 shutdown_connections 2023-07-21T22:19:35.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.339+0000 7ff4849ff700 1 -- 172.21.15.66:0/4071246062 wait complete. 2023-07-21T22:19:35.403 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-21T22:19:35.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.570+0000 7fac97bff700 1 -- 172.21.15.66:0/1648396741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90100cd0 msgr2=0x7fac90101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.570+0000 7fac97bff700 1 --2- 172.21.15.66:0/1648396741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90100cd0 0x7fac90101120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac80009990 tx=0x7fac80009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.570+0000 7fac97bff700 1 -- 172.21.15.66:0/1648396741 shutdown_connections 2023-07-21T22:19:35.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.571+0000 7fac97bff700 1 --2- 172.21.15.66:0/1648396741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac900ffae0 0x7fac900ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.571+0000 7fac97bff700 1 --2- 172.21.15.66:0/1648396741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac90101660 0x7fac90107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.571+0000 7fac97bff700 1 --2- 172.21.15.66:0/1648396741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90100cd0 0x7fac90101120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.571+0000 7fac97bff700 1 -- 172.21.15.66:0/1648396741 >> 172.21.15.66:0/1648396741 conn(0x7fac900fb0e0 msgr2=0x7fac900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:35.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.571+0000 7fac97bff700 1 -- 172.21.15.66:0/1648396741 shutdown_connections 2023-07-21T22:19:35.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.571+0000 7fac97bff700 1 -- 172.21.15.66:0/1648396741 wait complete. 2023-07-21T22:19:35.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.572+0000 7fac97bff700 1 Processor -- start 2023-07-21T22:19:35.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.572+0000 7fac97bff700 1 -- start start 2023-07-21T22:19:35.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac97bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac900ffae0 0x7fac900742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac97bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac90100cd0 0x7fac90072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac97bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 0x7fac90072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac97bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac900735e0 con 0x7fac90101660 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac97bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90073750 con 0x7fac90100cd0 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac97bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9019f8c0 con 0x7fac900ffae0 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac9619c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 0x7fac90072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac9619c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 0x7fac90072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46926/0 (socket says 172.21.15.66:46926) 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac9619c700 1 -- 172.21.15.66:0/2838397375 learned_addr learned my addr 172.21.15.66:0/2838397375 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac9599b700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac900ffae0 0x7fac900742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.573+0000 7fac9519a700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac90100cd0 0x7fac90072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:35 smithi104 ceph-mon[129021]: pgmap v288: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2023-07-21T22:19:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:35 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:active 2023-07-21T22:19:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:35 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3070538918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:19:35.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:35 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-07-21T22:19:35.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:35 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:19:35.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac9619c700 1 -- 172.21.15.66:0/2838397375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac900ffae0 msgr2=0x7fac900742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac9619c700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac900ffae0 0x7fac900742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac9619c700 1 -- 172.21.15.66:0/2838397375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac90100cd0 msgr2=0x7fac90072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac9619c700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac90100cd0 0x7fac90072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac9619c700 1 -- 172.21.15.66:0/2838397375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800095f0 con 0x7fac90101660 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac9619c700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 0x7fac90072e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fac8800ef30 tx=0x7fac8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac88018070 con 0x7fac90101660 2023-07-21T22:19:35.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac8800f040 con 0x7fac90101660 2023-07-21T22:19:35.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800cd00 con 0x7fac90101660 2023-07-21T22:19:35.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9019fb40 con 0x7fac90101660 2023-07-21T22:19:35.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.574+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac901a0000 con 0x7fac90101660 2023-07-21T22:19:35.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.575+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9010b300 con 0x7fac90101660 2023-07-21T22:19:35.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.577+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fac88003a00 con 0x7fac90101660 2023-07-21T22:19:35.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.580+0000 7fac86ffd700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fac7c0a11d0 0x7fac7c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:35.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.580+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fac880cc5c0 con 0x7fac90101660 2023-07-21T22:19:35.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.580+0000 7fac9599b700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fac7c0a11d0 0x7fac7c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:35.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.581+0000 7fac9599b700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fac7c0a11d0 0x7fac7c0a3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac90100b30 tx=0x7fac8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:35.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.584+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fac88095b50 con 0x7fac90101660 2023-07-21T22:19:35.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:35 smithi066 ceph-mon[126379]: pgmap v288: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2023-07-21T22:19:35.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:35 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:active 2023-07-21T22:19:35.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:35 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3070538918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:19:35.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:35 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-07-21T22:19:35.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:35 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:19:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:35 smithi149 ceph-mon[129268]: pgmap v288: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 3 op/s 2023-07-21T22:19:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:35 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:active 2023-07-21T22:19:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:35 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3070538918' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T22:19:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:35 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-07-21T22:19:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:35 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:19:35.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.826+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fac90060950 con 0x7fac7c0a11d0 2023-07-21T22:19:35.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.849+0000 7fac86ffd700 1 -- 172.21.15.66:0/2838397375 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fac90060950 con 0x7fac7c0a11d0 2023-07-21T22:19:35.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fac7c0a11d0 msgr2=0x7fac7c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fac7c0a11d0 0x7fac7c0a3680 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac90100b30 tx=0x7fac8c009260 comp rx=0 tx=0).stop 2023-07-21T22:19:35.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 msgr2=0x7fac90072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:35.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 0x7fac90072e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fac8800ef30 tx=0x7fac8800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 shutdown_connections 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fac7c0a11d0 0x7fac7c0a3680 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac90101660 0x7fac90072e40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac900ffae0 0x7fac900742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 --2- 172.21.15.66:0/2838397375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac90100cd0 0x7fac90072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 >> 172.21.15.66:0/2838397375 conn(0x7fac900fb0e0 msgr2=0x7fac900fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 shutdown_connections 2023-07-21T22:19:35.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:35.852+0000 7fac97bff700 1 -- 172.21.15.66:0/2838397375 wait complete. 2023-07-21T22:19:35.915 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-21T22:19:36.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 -- 172.21.15.66:0/651396712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 msgr2=0x7f8f44101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/651396712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 0x7f8f44101120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009990 tx=0x7f8f34009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 -- 172.21.15.66:0/651396712 shutdown_connections 2023-07-21T22:19:36.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/651396712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f440ffae0 0x7f8f440ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/651396712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f44101660 0x7f8f44107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/651396712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 0x7f8f44101120 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 -- 172.21.15.66:0/651396712 >> 172.21.15.66:0/651396712 conn(0x7f8f440fb0e0 msgr2=0x7f8f440fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 -- 172.21.15.66:0/651396712 shutdown_connections 2023-07-21T22:19:36.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.080+0000 7f8f4bdff700 1 -- 172.21.15.66:0/651396712 wait complete. 2023-07-21T22:19:36.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 Processor -- start 2023-07-21T22:19:36.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 -- start start 2023-07-21T22:19:36.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 0x7f8f44197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 0x7f8f44197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f44101660 0x7f8f4419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4419f5b0 con 0x7f8f440ffae0 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4419f6f0 con 0x7f8f44101660 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.081+0000 7f8f4bdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4419f880 con 0x7f8f44100cd0 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 0x7f8f44197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f4939a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 0x7f8f44197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 0x7f8f44197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46948/0 (socket says 172.21.15.66:46948) 2023-07-21T22:19:36.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 -- 172.21.15.66:0/243665174 learned_addr learned my addr 172.21.15.66:0/243665174 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f4a39c700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f44101660 0x7f8f4419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 -- 172.21.15.66:0/243665174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 msgr2=0x7f8f44197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 0x7f8f44197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 -- 172.21.15.66:0/243665174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f44101660 msgr2=0x7f8f4419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f44101660 0x7f8f4419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 -- 172.21.15.66:0/243665174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c009c40 con 0x7f8f440ffae0 2023-07-21T22:19:36.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.082+0000 7f8f49b9b700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 0x7f8f44197a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e970 tx=0x7f8f4000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.083+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40009790 con 0x7f8f440ffae0 2023-07-21T22:19:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.083+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340095f0 con 0x7f8f440ffae0 2023-07-21T22:19:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.083+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4000c7c0 con 0x7f8f440ffae0 2023-07-21T22:19:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.083+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40003c20 con 0x7f8f440ffae0 2023-07-21T22:19:36.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.083+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4419fee0 con 0x7f8f440ffae0 2023-07-21T22:19:36.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.084+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4410b2a0 con 0x7f8f440ffae0 2023-07-21T22:19:36.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.086+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8f4000c930 con 0x7f8f440ffae0 2023-07-21T22:19:36.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.087+0000 7f8f3affd700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f300a1150 0x7f8f300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.087+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8f40014070 con 0x7f8f440ffae0 2023-07-21T22:19:36.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.087+0000 7f8f4939a700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f300a1150 0x7f8f300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.088+0000 7f8f4939a700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f300a1150 0x7f8f300a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b2e0 tx=0x7f8f34005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:36.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.091+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8f400961f0 con 0x7f8f440ffae0 2023-07-21T22:19:36.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.327+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8f400965c0 con 0x7f8f440ffae0 2023-07-21T22:19:36.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.348+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f44107670 con 0x7f8f300a1150 2023-07-21T22:19:36.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.352+0000 7f8f3affd700 1 -- 172.21.15.66:0/243665174 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f8f44107670 con 0x7f8f300a1150 2023-07-21T22:19:36.397 INFO:teuthology.orchestra.run.smithi066.stdout:/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca 2023-07-21T22:19:36.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f300a1150 msgr2=0x7f8f300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f300a1150 0x7f8f300a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b2e0 tx=0x7f8f34005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 msgr2=0x7f8f44197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 0x7f8f44197a30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000e970 tx=0x7f8f4000ec80 comp rx=0 tx=0).stop 2023-07-21T22:19:36.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 shutdown_connections 2023-07-21T22:19:36.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8f300a1150 0x7f8f300a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f44101660 0x7f8f4419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8f440ffae0 0x7f8f44197a30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 --2- 172.21.15.66:0/243665174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8f44100cd0 0x7f8f44197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 >> 172.21.15.66:0/243665174 conn(0x7f8f440fb0e0 msgr2=0x7f8f440fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 shutdown_connections 2023-07-21T22:19:36.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.355+0000 7f8f4bdff700 1 -- 172.21.15.66:0/243665174 wait complete. 2023-07-21T22:19:36.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:19:36.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2023-07-21T22:19:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-07-21T22:19:36.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2023-07-21T22:19:36.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-07-21T22:19:36.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-07-21T22:19:36.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-07-21T22:19:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 -- 172.21.15.66:0/78423058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410100cf0 msgr2=0x7fe410101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 --2- 172.21.15.66:0/78423058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410100cf0 0x7fe410101140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe400009990 tx=0x7fe400009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 -- 172.21.15.66:0/78423058 shutdown_connections 2023-07-21T22:19:36.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 --2- 172.21.15.66:0/78423058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe410101680 0x7fe410107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 --2- 172.21.15.66:0/78423058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4100ffb00 0x7fe4100fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 --2- 172.21.15.66:0/78423058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410100cf0 0x7fe410101140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 -- 172.21.15.66:0/78423058 >> 172.21.15.66:0/78423058 conn(0x7fe4100fb0e0 msgr2=0x7fe4100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 -- 172.21.15.66:0/78423058 shutdown_connections 2023-07-21T22:19:36.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.580+0000 7fe4185ff700 1 -- 172.21.15.66:0/78423058 wait complete. 2023-07-21T22:19:36.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 Processor -- start 2023-07-21T22:19:36.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 -- start start 2023-07-21T22:19:36.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 0x7fe410195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe410100cf0 0x7fe410195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410101680 0x7fe41019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41019d3b0 con 0x7fe410101680 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41019d4f0 con 0x7fe4100ffb00 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.582+0000 7fe4185ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41019d680 con 0x7fe410100cf0 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe41639b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 0x7fe410195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410101680 0x7fe41019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe41639b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 0x7fe410195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:35928/0 (socket says 172.21.15.66:35928) 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe41639b700 1 -- 172.21.15.66:0/3117193975 learned_addr learned my addr 172.21.15.66:0/3117193975 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe415b9a700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe410100cf0 0x7fe410195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 -- 172.21.15.66:0/3117193975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe410100cf0 msgr2=0x7fe410195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe410100cf0 0x7fe410195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 -- 172.21.15.66:0/3117193975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 msgr2=0x7fe410195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 0x7fe410195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 -- 172.21.15.66:0/3117193975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000095f0 con 0x7fe410101680 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe41639b700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 0x7fe410195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.583+0000 7fe416b9c700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410101680 0x7fe41019cde0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe408007f10 tx=0x7fe40800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.584+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408009ea0 con 0x7fe410101680 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.584+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe40800f040 con 0x7fe410101680 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.584+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408004350 con 0x7fe410101680 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.584+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41019d960 con 0x7fe410101680 2023-07-21T22:19:36.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.584+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41019de20 con 0x7fe410101680 2023-07-21T22:19:36.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.585+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410107070 con 0x7fe410101680 2023-07-21T22:19:36.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.587+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe40800db20 con 0x7fe410101680 2023-07-21T22:19:36.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.588+0000 7fe4077fe700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe3fc0a1220 0x7fe3fc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:36.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.588+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe4080cc640 con 0x7fe410101680 2023-07-21T22:19:36.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.589+0000 7fe41639b700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe3fc0a1220 0x7fe3fc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:36.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.589+0000 7fe41639b700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe3fc0a1220 0x7fe3fc0a36d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0105c0 tx=0x7fe40c00b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:36.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.593+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe4080958c0 con 0x7fe410101680 2023-07-21T22:19:36.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:36.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:36.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2023-07-21T22:19:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-07-21T22:19:36.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-07-21T22:19:36.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:36 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-07-21T22:19:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='client.24802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-21T22:19:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-07-21T22:19:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2023-07-21T22:19:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:36 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2023-07-21T22:19:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.850+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe41002c850 con 0x7fe410101680 2023-07-21T22:19:36.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.850+0000 7fe4077fe700 1 -- 172.21.15.66:0/3117193975 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7fe408098ee0 con 0x7fe410101680 2023-07-21T22:19:36.896 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:36.896 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:19:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe3fc0a1220 msgr2=0x7fe3fc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe3fc0a1220 0x7fe3fc0a36d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe40c0105c0 tx=0x7fe40c00b4a0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410101680 msgr2=0x7fe41019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410101680 0x7fe41019cde0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe408007f10 tx=0x7fe40800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 shutdown_connections 2023-07-21T22:19:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe3fc0a1220 0x7fe3fc0a36d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4100ffb00 0x7fe410195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe410101680 0x7fe41019cde0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 --2- 172.21.15.66:0/3117193975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe410100cf0 0x7fe410195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 >> 172.21.15.66:0/3117193975 conn(0x7fe4100fb0e0 msgr2=0x7fe4101041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 shutdown_connections 2023-07-21T22:19:36.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:36.853+0000 7fe4185ff700 1 -- 172.21.15.66:0/3117193975 wait complete. 2023-07-21T22:19:36.900 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:19:36.917 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:19:37.118 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: 2023-07-21T22:19:36.743+0000 7f2fde972700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-07-21T22:19:37.118 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:19:36 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: 2023-07-21T22:19:36.743+0000 7f2fde972700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-07-21T22:19:37.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 -- 172.21.15.66:0/248040641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680103880 msgr2=0x7fc68010a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 --2- 172.21.15.66:0/248040641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680103880 0x7fc68010a140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc678009f90 tx=0x7fc67800b330 comp rx=0 tx=0).stop 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 -- 172.21.15.66:0/248040641 shutdown_connections 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 --2- 172.21.15.66:0/248040641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680103880 0x7fc68010a140 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 --2- 172.21.15.66:0/248040641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc680101d00 0x7fc680102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 --2- 172.21.15.66:0/248040641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc680102ef0 0x7fc680103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 -- 172.21.15.66:0/248040641 >> 172.21.15.66:0/248040641 conn(0x7fc6800fd320 msgr2=0x7fc6800ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:37.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 -- 172.21.15.66:0/248040641 shutdown_connections 2023-07-21T22:19:37.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.081+0000 7fc6859ff700 1 -- 172.21.15.66:0/248040641 wait complete. 2023-07-21T22:19:37.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.082+0000 7fc6859ff700 1 Processor -- start 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 -- start start 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 0x7fc680197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc680102ef0 0x7fc680198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc680103880 0x7fc68019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019f680 con 0x7fc680102ef0 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019f7c0 con 0x7fc680101d00 2023-07-21T22:19:37.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc6859ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019f950 con 0x7fc680103880 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc67effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 0x7fc680197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc67effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 0x7fc680197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:35948/0 (socket says 172.21.15.66:35948) 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc67effd700 1 -- 172.21.15.66:0/4185567880 learned_addr learned my addr 172.21.15.66:0/4185567880 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.083+0000 7fc67e7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc680102ef0 0x7fc680198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67f7fe700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc680103880 0x7fc68019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67effd700 1 -- 172.21.15.66:0/4185567880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc680103880 msgr2=0x7fc68019f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67effd700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc680103880 0x7fc68019f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67effd700 1 -- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc680102ef0 msgr2=0x7fc680198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67effd700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc680102ef0 0x7fc680198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67effd700 1 -- 172.21.15.66:0/4185567880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740095f0 con 0x7fc680101d00 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc67effd700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 0x7fc680197b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc67000e970 tx=0x7fc67000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670009790 con 0x7fc680101d00 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc67000eec0 con 0x7fc680101d00 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc678009c40 con 0x7fc680101d00 2023-07-21T22:19:37.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc670003cf0 con 0x7fc680101d00 2023-07-21T22:19:37.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.084+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6801a0000 con 0x7fc680101d00 2023-07-21T22:19:37.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.085+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6801091c0 con 0x7fc680101d00 2023-07-21T22:19:37.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.088+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc67000ccd0 con 0x7fc680101d00 2023-07-21T22:19:37.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.089+0000 7fc6849fd700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc6680a12a0 0x7fc6680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.089+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc670014070 con 0x7fc680101d00 2023-07-21T22:19:37.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.089+0000 7fc67e7fc700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc6680a12a0 0x7fc6680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.090+0000 7fc67e7fc700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc6680a12a0 0x7fc6680a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc674005fd0 tx=0x7fc674009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:37.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.091+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc670096400 con 0x7fc680101d00 2023-07-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.308+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc68002c850 con 0x7fc680101d00 2023-07-21T22:19:37.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.312+0000 7fc6849fd700 1 -- 172.21.15.66:0/4185567880 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24442 (secure 0 0 0) 0x7fc670099a20 con 0x7fc680101d00 2023-07-21T22:19:37.357 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:37.358 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:37.301201+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:37.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc6680a12a0 msgr2=0x7fc6680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc6680a12a0 0x7fc6680a3750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc674005fd0 tx=0x7fc674009310 comp rx=0 tx=0).stop 2023-07-21T22:19:37.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 msgr2=0x7fc680197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 0x7fc680197b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc67000e970 tx=0x7fc67000ec80 comp rx=0 tx=0).stop 2023-07-21T22:19:37.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 shutdown_connections 2023-07-21T22:19:37.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc6680a12a0 0x7fc6680a3750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc680101d00 0x7fc680197b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc680103880 0x7fc68019f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 --2- 172.21.15.66:0/4185567880 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc680102ef0 0x7fc680198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 >> 172.21.15.66:0/4185567880 conn(0x7fc6800fd320 msgr2=0x7fc680106300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:37.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 shutdown_connections 2023-07-21T22:19:37.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.315+0000 7fc6859ff700 1 -- 172.21.15.66:0/4185567880 wait complete. 2023-07-21T22:19:37.382 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-21T22:19:37.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:19:37.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 -- 172.21.15.66:0/3166538287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec100cf0 msgr2=0x7f7eec101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/3166538287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec100cf0 0x7f7eec101140 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009990 tx=0x7f7ed8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 -- 172.21.15.66:0/3166538287 shutdown_connections 2023-07-21T22:19:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/3166538287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7eec101680 0x7f7eec107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/3166538287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7eec0ffb00 0x7f7eec0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/3166538287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec100cf0 0x7f7eec101140 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.546+0000 7f7ef2bff700 1 -- 172.21.15.66:0/3166538287 >> 172.21.15.66:0/3166538287 conn(0x7f7eec0fb0e0 msgr2=0x7f7eec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:37.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.547+0000 7f7ef2bff700 1 -- 172.21.15.66:0/3166538287 shutdown_connections 2023-07-21T22:19:37.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.547+0000 7f7ef2bff700 1 -- 172.21.15.66:0/3166538287 wait complete. 2023-07-21T22:19:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.548+0000 7f7ef2bff700 1 Processor -- start 2023-07-21T22:19:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.548+0000 7f7ef2bff700 1 -- start start 2023-07-21T22:19:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef2bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 0x7f7eec195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef2bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7eec100cf0 0x7f7eec195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef2bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7eec101680 0x7f7eec19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef2bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec19d390 con 0x7f7eec0ffb00 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef2bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec19d4d0 con 0x7f7eec100cf0 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef2bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec19d660 con 0x7f7eec101680 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef099b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 0x7f7eec195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef099b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 0x7f7eec195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47008/0 (socket says 172.21.15.66:47008) 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef099b700 1 -- 172.21.15.66:0/932824432 learned_addr learned my addr 172.21.15.66:0/932824432 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef119c700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7eec101680 0x7f7eec19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7eebfff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7eec100cf0 0x7f7eec195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.549+0000 7f7ef099b700 1 -- 172.21.15.66:0/932824432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7eec101680 msgr2=0x7f7eec19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef099b700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7eec101680 0x7f7eec19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef099b700 1 -- 172.21.15.66:0/932824432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7eec100cf0 msgr2=0x7f7eec195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef099b700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7eec100cf0 0x7f7eec195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef099b700 1 -- 172.21.15.66:0/932824432 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80095f0 con 0x7f7eec0ffb00 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef099b700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 0x7f7eec195810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000e970 tx=0x7f7ee000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0009790 con 0x7f7eec0ffb00 2023-07-21T22:19:37.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ee000c7e0 con 0x7f7eec0ffb00 2023-07-21T22:19:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0003c70 con 0x7f7eec0ffb00 2023-07-21T22:19:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec19d940 con 0x7f7eec0ffb00 2023-07-21T22:19:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.550+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec19de00 con 0x7f7eec0ffb00 2023-07-21T22:19:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.551+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec107050 con 0x7f7eec0ffb00 2023-07-21T22:19:37.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.553+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7ee000cc50 con 0x7f7eec0ffb00 2023-07-21T22:19:37.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.555+0000 7f7ee9ffb700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7edc0a12a0 0x7f7edc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:37.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.555+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ee0014070 con 0x7f7eec0ffb00 2023-07-21T22:19:37.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.555+0000 7f7eebfff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7edc0a12a0 0x7f7edc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:37.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.556+0000 7f7eebfff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7edc0a12a0 0x7f7edc0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b2e0 tx=0x7f7ed8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:37.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.558+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7ee00961a0 con 0x7f7eec0ffb00 2023-07-21T22:19:37.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: pgmap v289: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2023-07-21T22:19:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-21T22:19:37.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:37.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3117193975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:37.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T22:19:37.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:37 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4185567880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:37.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:37.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: pgmap v289: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-07-21T22:19:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3117193975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:37.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T22:19:37.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:37 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4185567880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:37.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:37.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: pgmap v289: 129 pgs: 129 active+clean; 583 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2023-07-21T22:19:37.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-21T22:19:37.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-07-21T22:19:37.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2023-07-21T22:19:37.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-07-21T22:19:37.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2023-07-21T22:19:37.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 10]}]': finished 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-21T22:19:37.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:19:37.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3117193975' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:19:37.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T22:19:37.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:37 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4185567880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:37.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.766+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7eec060950 con 0x7f7edc0a12a0 2023-07-21T22:19:37.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.781+0000 7f7ee9ffb700 1 -- 172.21.15.66:0/932824432 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422104 (secure 0 0 0) 0x7f7eec060950 con 0x7f7edc0a12a0 2023-07-21T22:19:37.831 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:37.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7edc0a12a0 msgr2=0x7f7edc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7edc0a12a0 0x7f7edc0a3750 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800b2e0 tx=0x7f7ed8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 msgr2=0x7f7eec195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 0x7f7eec195810 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000e970 tx=0x7f7ee000ec80 comp rx=0 tx=0).stop 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 shutdown_connections 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7edc0a12a0 0x7f7edc0a3750 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7eec101680 0x7f7eec19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7eec0ffb00 0x7f7eec195810 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 --2- 172.21.15.66:0/932824432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7eec100cf0 0x7f7eec195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 >> 172.21.15.66:0/932824432 conn(0x7f7eec0fb0e0 msgr2=0x7f7eec104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.786+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 shutdown_connections 2023-07-21T22:19:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:37.787+0000 7f7ef2bff700 1 -- 172.21.15.66:0/932824432 wait complete. 2023-07-21T22:19:37.836 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:19:37.851 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-07-21T22:19:37.303403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598456,"num_objects":59,"num_object_clones":0,"num_object_copies":177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":159,"num_read_kb":137,"num_write":307,"num_write_kb":2983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":248,"ondisk_log_size":248,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335028,"kb_used_data":11604,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606796,"statfs":{"total":1151940427776,"available":1151597359104,"internally_reserved":0,"allocated":11882496,"data_stored":7482162,"data_compressed":250073,"data_compressed_allocated":335872,"data_compressed_original":2170880,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3637,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":78,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.122572"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635357+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:19:12.635357+0000","last_peered":"2023-07-21T22:19:12.635357+0000","last_clean":"2023-07-21T22:19:12.635357+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:19:12.635357+0000","last_undegraded":"2023-07-21T22:19:12.635357+0000","last_fullsized":"2023-07-21T22:19:12.635357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:43:30.586945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054535+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:19:13.054535+0000","last_peered":"2023-07-21T22:19:13.054535+0000","last_clean":"2023-07-21T22:19:13.054535+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:19:13.054535+0000","last_undegraded":"2023-07-21T22:19:13.054535+0000","last_fullsized":"2023-07-21T22:19:13.054535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:58:15.473345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054499+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:19:13.054499+0000","last_peered":"2023-07-21T22:19:13.054499+0000","last_clean":"2023-07-21T22:19:13.054499+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:19:13.054499+0000","last_undegraded":"2023-07-21T22:19:13.054499+0000","last_fullsized":"2023-07-21T22:19:13.054499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:22:25.508998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054820+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:19:13.054820+0000","last_peered":"2023-07-21T22:19:13.054820+0000","last_clean":"2023-07-21T22:19:13.054820+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:19:13.054820+0000","last_undegraded":"2023-07-21T22:19:13.054820+0000","last_fullsized":"2023-07-21T22:19:13.054820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:37:05.611733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054111+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:19:13.054111+0000","last_peered":"2023-07-21T22:19:13.054111+0000","last_clean":"2023-07-21T22:19:13.054111+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:19:13.054111+0000","last_undegraded":"2023-07-21T22:19:13.054111+0000","last_fullsized":"2023-07-21T22:19:13.054111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:43:30.768314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391309+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:19:12.391309+0000","last_peered":"2023-07-21T22:19:12.391309+0000","last_clean":"2023-07-21T22:19:12.391309+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:19:12.391309+0000","last_undegraded":"2023-07-21T22:19:12.391309+0000","last_fullsized":"2023-07-21T22:19:12.391309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:46:12.527665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769784+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:19:12.769784+0000","last_peered":"2023-07-21T22:19:12.769784+0000","last_clean":"2023-07-21T22:19:12.769784+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:19:12.769784+0000","last_undegraded":"2023-07-21T22:19:12.769784+0000","last_fullsized":"2023-07-21T22:19:12.769784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:44.363556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054369+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:19:13.054369+0000","last_peered":"2023-07-21T22:19:13.054369+0000","last_clean":"2023-07-21T22:19:13.054369+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:19:13.054369+0000","last_undegraded":"2023-07-21T22:19:13.054369+0000","last_fullsized":"2023-07-21T22:19:13.054369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:49:33.535575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054539+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:19:13.054539+0000","last_peered":"2023-07-21T22:19:13.054539+0000","last_clean":"2023-07-21T22:19:13.054539+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:19:13.054539+0000","last_undegraded":"2023-07-21T22:19:13.054539+0000","last_fullsized":"2023-07-21T22:19:13.054539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:02:21.910119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391720+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:19:12.391720+0000","last_peered":"2023-07-21T22:19:12.391720+0000","last_clean":"2023-07-21T22:19:12.391720+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:19:12.391720+0000","last_undegraded":"2023-07-21T22:19:12.391720+0000","last_fullsized":"2023-07-21T22:19:12.391720+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:01:14.480144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770333+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:19:12.770333+0000","last_peered":"2023-07-21T22:19:12.770333+0000","last_clean":"2023-07-21T22:19:12.770333+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:19:12.770333+0000","last_undegraded":"2023-07-21T22:19:12.770333+0000","last_fullsized":"2023-07-21T22:19:12.770333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:11.614161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054708+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:19:13.054708+0000","last_peered":"2023-07-21T22:19:13.054708+0000","last_clean":"2023-07-21T22:19:13.054708+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:19:13.054708+0000","last_undegraded":"2023-07-21T22:19:13.054708+0000","last_fullsized":"2023-07-21T22:19:13.054708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:05:24.932631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055253+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:19:13.055253+0000","last_peered":"2023-07-21T22:19:13.055253+0000","last_clean":"2023-07-21T22:19:13.055253+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:19:13.055253+0000","last_undegraded":"2023-07-21T22:19:13.055253+0000","last_fullsized":"2023-07-21T22:19:13.055253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:26:26.315690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054077+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:19:13.054077+0000","last_peered":"2023-07-21T22:19:13.054077+0000","last_clean":"2023-07-21T22:19:13.054077+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:19:13.054077+0000","last_undegraded":"2023-07-21T22:19:13.054077+0000","last_fullsized":"2023-07-21T22:19:13.054077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:19:52.027627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054172+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:19:13.054172+0000","last_peered":"2023-07-21T22:19:13.054172+0000","last_clean":"2023-07-21T22:19:13.054172+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:19:13.054172+0000","last_undegraded":"2023-07-21T22:19:13.054172+0000","last_fullsized":"2023-07-21T22:19:13.054172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:29:41.950899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053639+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:19:13.053639+0000","last_peered":"2023-07-21T22:19:13.053639+0000","last_clean":"2023-07-21T22:19:13.053639+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:19:13.053639+0000","last_undegraded":"2023-07-21T22:19:13.053639+0000","last_fullsized":"2023-07-21T22:19:13.053639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:41.706094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635220+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:19:12.635220+0000","last_peered":"2023-07-21T22:19:12.635220+0000","last_clean":"2023-07-21T22:19:12.635220+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:19:12.635220+0000","last_undegraded":"2023-07-21T22:19:12.635220+0000","last_fullsized":"2023-07-21T22:19:12.635220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:45:57.291197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390640+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:19:12.390640+0000","last_peered":"2023-07-21T22:19:12.390640+0000","last_clean":"2023-07-21T22:19:12.390640+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:19:12.390640+0000","last_undegraded":"2023-07-21T22:19:12.390640+0000","last_fullsized":"2023-07-21T22:19:12.390640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:35:56.391831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391012+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:19:12.391012+0000","last_peered":"2023-07-21T22:19:12.391012+0000","last_clean":"2023-07-21T22:19:12.391012+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:19:12.391012+0000","last_undegraded":"2023-07-21T22:19:12.391012+0000","last_fullsized":"2023-07-21T22:19:12.391012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:51:04.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390711+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:19:12.390711+0000","last_peered":"2023-07-21T22:19:12.390711+0000","last_clean":"2023-07-21T22:19:12.390711+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:19:12.390711+0000","last_undegraded":"2023-07-21T22:19:12.390711+0000","last_fullsized":"2023-07-21T22:19:12.390711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:58.482547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.303878+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:19:33.303878+0000","last_peered":"2023-07-21T22:19:33.303878+0000","last_clean":"2023-07-21T22:19:33.303878+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:19:33.303878+0000","last_undegraded":"2023-07-21T22:19:33.303878+0000","last_fullsized":"2023-07-21T22:19:33.303878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:32:17.121216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055956+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:19:13.055956+0000","last_peered":"2023-07-21T22:19:13.055956+0000","last_clean":"2023-07-21T22:19:13.055956+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:19:13.055956+0000","last_undegraded":"2023-07-21T22:19:13.055956+0000","last_fullsized":"2023-07-21T22:19:13.055956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:42:47.152694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053650+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:19:13.053650+0000","last_peered":"2023-07-21T22:19:13.053650+0000","last_clean":"2023-07-21T22:19:13.053650+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:19:13.053650+0000","last_undegraded":"2023-07-21T22:19:13.053650+0000","last_fullsized":"2023-07-21T22:19:13.053650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:15:19.457080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.056271+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:19:13.056271+0000","last_peered":"2023-07-21T22:19:13.056271+0000","last_clean":"2023-07-21T22:19:13.056271+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:19:13.056271+0000","last_undegraded":"2023-07-21T22:19:13.056271+0000","last_fullsized":"2023-07-21T22:19:13.056271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:19:59.319714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055599+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:19:13.055599+0000","last_peered":"2023-07-21T22:19:13.055599+0000","last_clean":"2023-07-21T22:19:13.055599+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:19:13.055599+0000","last_undegraded":"2023-07-21T22:19:13.055599+0000","last_fullsized":"2023-07-21T22:19:13.055599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:05:46.461070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054710+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:19:13.054710+0000","last_peered":"2023-07-21T22:19:13.054710+0000","last_clean":"2023-07-21T22:19:13.054710+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:19:13.054710+0000","last_undegraded":"2023-07-21T22:19:13.054710+0000","last_fullsized":"2023-07-21T22:19:13.054710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:59.853903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.300206+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:19:33.300206+0000","last_peered":"2023-07-21T22:19:33.300206+0000","last_clean":"2023-07-21T22:19:33.300206+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:19:33.300206+0000","last_undegraded":"2023-07-21T22:19:33.300206+0000","last_fullsized":"2023-07-21T22:19:33.300206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:03:04.510706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053961+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:19:13.053961+0000","last_peered":"2023-07-21T22:19:13.053961+0000","last_clean":"2023-07-21T22:19:13.053961+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:19:13.053961+0000","last_undegraded":"2023-07-21T22:19:13.053961+0000","last_fullsized":"2023-07-21T22:19:13.053961+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:39:51.923885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054061+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:19:13.054061+0000","last_peered":"2023-07-21T22:19:13.054061+0000","last_clean":"2023-07-21T22:19:13.054061+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:19:13.054061+0000","last_undegraded":"2023-07-21T22:19:13.054061+0000","last_fullsized":"2023-07-21T22:19:13.054061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:03:23.370924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054319+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:19:13.054319+0000","last_peered":"2023-07-21T22:19:13.054319+0000","last_clean":"2023-07-21T22:19:13.054319+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:19:13.054319+0000","last_undegraded":"2023-07-21T22:19:13.054319+0000","last_fullsized":"2023-07-21T22:19:13.054319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:01:46.744100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305422+0000","last_change":"2023-07-21T22:19:07.373299+0000","last_active":"2023-07-21T22:19:36.305422+0000","last_peered":"2023-07-21T22:19:36.305422+0000","last_clean":"2023-07-21T22:19:36.305422+0000","last_became_active":"2023-07-21T22:19:07.372944+0000","last_became_peered":"2023-07-21T22:19:07.372944+0000","last_unstale":"2023-07-21T22:19:36.305422+0000","last_undegraded":"2023-07-21T22:19:36.305422+0000","last_fullsized":"2023-07-21T22:19:36.305422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:05:06.676917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391017+0000","last_change":"2023-07-21T22:19:11.387155+0000","last_active":"2023-07-21T22:19:12.391017+0000","last_peered":"2023-07-21T22:19:12.391017+0000","last_clean":"2023-07-21T22:19:12.391017+0000","last_became_active":"2023-07-21T22:19:11.386756+0000","last_became_peered":"2023-07-21T22:19:11.386756+0000","last_unstale":"2023-07-21T22:19:12.391017+0000","last_undegraded":"2023-07-21T22:19:12.391017+0000","last_fullsized":"2023-07-21T22:19:12.391017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:08:36.187925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304737+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:19:36.304737+0000","last_peered":"2023-07-21T22:19:36.304737+0000","last_clean":"2023-07-21T22:19:36.304737+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:19:36.304737+0000","last_undegraded":"2023-07-21T22:19:36.304737+0000","last_fullsized":"2023-07-21T22:19:36.304737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:27:22.286359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054124+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:19:13.054124+0000","last_peered":"2023-07-21T22:19:13.054124+0000","last_clean":"2023-07-21T22:19:13.054124+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:19:13.054124+0000","last_undegraded":"2023-07-21T22:19:13.054124+0000","last_fullsized":"2023-07-21T22:19:13.054124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:57:28.616635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390704+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:19:12.390704+0000","last_peered":"2023-07-21T22:19:12.390704+0000","last_clean":"2023-07-21T22:19:12.390704+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:19:12.390704+0000","last_undegraded":"2023-07-21T22:19:12.390704+0000","last_fullsized":"2023-07-21T22:19:12.390704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:50:05.930511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391226+0000","last_change":"2023-07-21T22:19:06.374311+0000","last_active":"2023-07-21T22:19:12.391226+0000","last_peered":"2023-07-21T22:19:12.391226+0000","last_clean":"2023-07-21T22:19:12.391226+0000","last_became_active":"2023-07-21T22:19:06.374002+0000","last_became_peered":"2023-07-21T22:19:06.374002+0000","last_unstale":"2023-07-21T22:19:12.391226+0000","last_undegraded":"2023-07-21T22:19:12.391226+0000","last_fullsized":"2023-07-21T22:19:12.391226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:33:49.680310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304829+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:19:36.304829+0000","last_peered":"2023-07-21T22:19:36.304829+0000","last_clean":"2023-07-21T22:19:36.304829+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:19:36.304829+0000","last_undegraded":"2023-07-21T22:19:36.304829+0000","last_fullsized":"2023-07-21T22:19:36.304829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:18:41.783365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635249+0000","last_change":"2023-07-21T22:19:11.382069+0000","last_active":"2023-07-21T22:19:12.635249+0000","last_peered":"2023-07-21T22:19:12.635249+0000","last_clean":"2023-07-21T22:19:12.635249+0000","last_became_active":"2023-07-21T22:19:11.381316+0000","last_became_peered":"2023-07-21T22:19:11.381316+0000","last_unstale":"2023-07-21T22:19:12.635249+0000","last_undegraded":"2023-07-21T22:19:12.635249+0000","last_fullsized":"2023-07-21T22:19:12.635249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:08:47.464945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.302925+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:19:33.302925+0000","last_peered":"2023-07-21T22:19:33.302925+0000","last_clean":"2023-07-21T22:19:33.302925+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:19:33.302925+0000","last_undegraded":"2023-07-21T22:19:33.302925+0000","last_fullsized":"2023-07-21T22:19:33.302925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:03:56.587172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054249+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:19:13.054249+0000","last_peered":"2023-07-21T22:19:13.054249+0000","last_clean":"2023-07-21T22:19:13.054249+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:19:13.054249+0000","last_undegraded":"2023-07-21T22:19:13.054249+0000","last_fullsized":"2023-07-21T22:19:13.054249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:38.969371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054444+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:19:13.054444+0000","last_peered":"2023-07-21T22:19:13.054444+0000","last_clean":"2023-07-21T22:19:13.054444+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:19:13.054444+0000","last_undegraded":"2023-07-21T22:19:13.054444+0000","last_fullsized":"2023-07-21T22:19:13.054444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:57.047349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304370+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:19:36.304370+0000","last_peered":"2023-07-21T22:19:36.304370+0000","last_clean":"2023-07-21T22:19:36.304370+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:19:36.304370+0000","last_undegraded":"2023-07-21T22:19:36.304370+0000","last_fullsized":"2023-07-21T22:19:36.304370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:04.271052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390382+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:19:12.390382+0000","last_peered":"2023-07-21T22:19:12.390382+0000","last_clean":"2023-07-21T22:19:12.390382+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:19:12.390382+0000","last_undegraded":"2023-07-21T22:19:12.390382+0000","last_fullsized":"2023-07-21T22:19:12.390382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:20:02.329601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054721+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:19:13.054721+0000","last_peered":"2023-07-21T22:19:13.054721+0000","last_clean":"2023-07-21T22:19:13.054721+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:19:13.054721+0000","last_undegraded":"2023-07-21T22:19:13.054721+0000","last_fullsized":"2023-07-21T22:19:13.054721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:20:42.835815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.290682+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:19:32.290682+0000","last_peered":"2023-07-21T22:19:32.290682+0000","last_clean":"2023-07-21T22:19:32.290682+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:19:32.290682+0000","last_undegraded":"2023-07-21T22:19:32.290682+0000","last_fullsized":"2023-07-21T22:19:32.290682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:19:26.358252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054255+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:19:13.054255+0000","last_peered":"2023-07-21T22:19:13.054255+0000","last_clean":"2023-07-21T22:19:13.054255+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:19:13.054255+0000","last_undegraded":"2023-07-21T22:19:13.054255+0000","last_fullsized":"2023-07-21T22:19:13.054255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:35:16.956227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390110+0000","last_change":"2023-07-21T22:19:11.382224+0000","last_active":"2023-07-21T22:19:12.390110+0000","last_peered":"2023-07-21T22:19:12.390110+0000","last_clean":"2023-07-21T22:19:12.390110+0000","last_became_active":"2023-07-21T22:19:11.381787+0000","last_became_peered":"2023-07-21T22:19:11.381787+0000","last_unstale":"2023-07-21T22:19:12.390110+0000","last_undegraded":"2023-07-21T22:19:12.390110+0000","last_fullsized":"2023-07-21T22:19:12.390110+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:28:40.534736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.293677+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:19:32.293677+0000","last_peered":"2023-07-21T22:19:32.293677+0000","last_clean":"2023-07-21T22:19:32.293677+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:19:32.293677+0000","last_undegraded":"2023-07-21T22:19:32.293677+0000","last_fullsized":"2023-07-21T22:19:32.293677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:54:02.828429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055718+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:19:13.055718+0000","last_peered":"2023-07-21T22:19:13.055718+0000","last_clean":"2023-07-21T22:19:13.055718+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:19:13.055718+0000","last_undegraded":"2023-07-21T22:19:13.055718+0000","last_fullsized":"2023-07-21T22:19:13.055718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:52.030405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054925+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:19:13.054925+0000","last_peered":"2023-07-21T22:19:13.054925+0000","last_clean":"2023-07-21T22:19:13.054925+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:19:13.054925+0000","last_undegraded":"2023-07-21T22:19:13.054925+0000","last_fullsized":"2023-07-21T22:19:13.054925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:41:33.631869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305147+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:19:36.305147+0000","last_peered":"2023-07-21T22:19:36.305147+0000","last_clean":"2023-07-21T22:19:36.305147+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:19:36.305147+0000","last_undegraded":"2023-07-21T22:19:36.305147+0000","last_fullsized":"2023-07-21T22:19:36.305147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:17.946189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054803+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:19:13.054803+0000","last_peered":"2023-07-21T22:19:13.054803+0000","last_clean":"2023-07-21T22:19:13.054803+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:19:13.054803+0000","last_undegraded":"2023-07-21T22:19:13.054803+0000","last_fullsized":"2023-07-21T22:19:13.054803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:32:53.912282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390479+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:19:12.390479+0000","last_peered":"2023-07-21T22:19:12.390479+0000","last_clean":"2023-07-21T22:19:12.390479+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:19:12.390479+0000","last_undegraded":"2023-07-21T22:19:12.390479+0000","last_fullsized":"2023-07-21T22:19:12.390479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:05.071713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'12","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.290465+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:19:32.290465+0000","last_peered":"2023-07-21T22:19:32.290465+0000","last_clean":"2023-07-21T22:19:32.290465+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:19:32.290465+0000","last_undegraded":"2023-07-21T22:19:32.290465+0000","last_fullsized":"2023-07-21T22:19:32.290465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:58:53.295309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3794,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054380+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:19:13.054380+0000","last_peered":"2023-07-21T22:19:13.054380+0000","last_clean":"2023-07-21T22:19:13.054380+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:19:13.054380+0000","last_undegraded":"2023-07-21T22:19:13.054380+0000","last_fullsized":"2023-07-21T22:19:13.054380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:45:18.050114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305755+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:19:36.305755+0000","last_peered":"2023-07-21T22:19:36.305755+0000","last_clean":"2023-07-21T22:19:36.305755+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:19:36.305755+0000","last_undegraded":"2023-07-21T22:19:36.305755+0000","last_fullsized":"2023-07-21T22:19:36.305755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.622432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.299636+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:19:33.299636+0000","last_peered":"2023-07-21T22:19:33.299636+0000","last_clean":"2023-07-21T22:19:33.299636+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:19:33.299636+0000","last_undegraded":"2023-07-21T22:19:33.299636+0000","last_fullsized":"2023-07-21T22:19:33.299636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:44:22.810764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054829+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:19:13.054829+0000","last_peered":"2023-07-21T22:19:13.054829+0000","last_clean":"2023-07-21T22:19:13.054829+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:19:13.054829+0000","last_undegraded":"2023-07-21T22:19:13.054829+0000","last_fullsized":"2023-07-21T22:19:13.054829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:28:45.448195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634964+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:19:12.634964+0000","last_peered":"2023-07-21T22:19:12.634964+0000","last_clean":"2023-07-21T22:19:12.634964+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:19:12.634964+0000","last_undegraded":"2023-07-21T22:19:12.634964+0000","last_fullsized":"2023-07-21T22:19:12.634964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:16.588357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304505+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:19:36.304505+0000","last_peered":"2023-07-21T22:19:36.304505+0000","last_clean":"2023-07-21T22:19:36.304505+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:19:36.304505+0000","last_undegraded":"2023-07-21T22:19:36.304505+0000","last_fullsized":"2023-07-21T22:19:36.304505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:00:45.003028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769962+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:19:12.769962+0000","last_peered":"2023-07-21T22:19:12.769962+0000","last_clean":"2023-07-21T22:19:12.769962+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:19:12.769962+0000","last_undegraded":"2023-07-21T22:19:12.769962+0000","last_fullsized":"2023-07-21T22:19:12.769962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:05.872357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390541+0000","last_change":"2023-07-21T22:19:11.394298+0000","last_active":"2023-07-21T22:19:12.390541+0000","last_peered":"2023-07-21T22:19:12.390541+0000","last_clean":"2023-07-21T22:19:12.390541+0000","last_became_active":"2023-07-21T22:19:11.390666+0000","last_became_peered":"2023-07-21T22:19:11.390666+0000","last_unstale":"2023-07-21T22:19:12.390541+0000","last_undegraded":"2023-07-21T22:19:12.390541+0000","last_fullsized":"2023-07-21T22:19:12.390541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:35.316183+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:19:35.316183+0000","last_peered":"2023-07-21T22:19:35.316183+0000","last_clean":"2023-07-21T22:19:35.316183+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:19:35.316183+0000","last_undegraded":"2023-07-21T22:19:35.316183+0000","last_fullsized":"2023-07-21T22:19:35.316183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:19.859228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769881+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:19:12.769881+0000","last_peered":"2023-07-21T22:19:12.769881+0000","last_clean":"2023-07-21T22:19:12.769881+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:19:12.769881+0000","last_undegraded":"2023-07-21T22:19:12.769881+0000","last_fullsized":"2023-07-21T22:19:12.769881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:29:38.425684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054739+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:19:13.054739+0000","last_peered":"2023-07-21T22:19:13.054739+0000","last_clean":"2023-07-21T22:19:13.054739+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:19:13.054739+0000","last_undegraded":"2023-07-21T22:19:13.054739+0000","last_fullsized":"2023-07-21T22:19:13.054739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:23:55.867317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.302776+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:19:33.302776+0000","last_peered":"2023-07-21T22:19:33.302776+0000","last_clean":"2023-07-21T22:19:33.302776+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:19:33.302776+0000","last_undegraded":"2023-07-21T22:19:33.302776+0000","last_fullsized":"2023-07-21T22:19:33.302776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:11.608929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304559+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:19:36.304559+0000","last_peered":"2023-07-21T22:19:36.304559+0000","last_clean":"2023-07-21T22:19:36.304559+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:19:36.304559+0000","last_undegraded":"2023-07-21T22:19:36.304559+0000","last_fullsized":"2023-07-21T22:19:36.304559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:47.513193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304416+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:19:36.304416+0000","last_peered":"2023-07-21T22:19:36.304416+0000","last_clean":"2023-07-21T22:19:36.304416+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:19:36.304416+0000","last_undegraded":"2023-07-21T22:19:36.304416+0000","last_fullsized":"2023-07-21T22:19:36.304416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:30.723357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054561+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:19:13.054561+0000","last_peered":"2023-07-21T22:19:13.054561+0000","last_clean":"2023-07-21T22:19:13.054561+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:19:13.054561+0000","last_undegraded":"2023-07-21T22:19:13.054561+0000","last_fullsized":"2023-07-21T22:19:13.054561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:12.419549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635103+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:19:12.635103+0000","last_peered":"2023-07-21T22:19:12.635103+0000","last_clean":"2023-07-21T22:19:12.635103+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:19:12.635103+0000","last_undegraded":"2023-07-21T22:19:12.635103+0000","last_fullsized":"2023-07-21T22:19:12.635103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:33:45.036406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054185+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:19:13.054185+0000","last_peered":"2023-07-21T22:19:13.054185+0000","last_clean":"2023-07-21T22:19:13.054185+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:19:13.054185+0000","last_undegraded":"2023-07-21T22:19:13.054185+0000","last_fullsized":"2023-07-21T22:19:13.054185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:36:15.202635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390628+0000","last_change":"2023-07-21T22:19:06.370251+0000","last_active":"2023-07-21T22:19:12.390628+0000","last_peered":"2023-07-21T22:19:12.390628+0000","last_clean":"2023-07-21T22:19:12.390628+0000","last_became_active":"2023-07-21T22:19:06.369838+0000","last_became_peered":"2023-07-21T22:19:06.369838+0000","last_unstale":"2023-07-21T22:19:12.390628+0000","last_undegraded":"2023-07-21T22:19:12.390628+0000","last_fullsized":"2023-07-21T22:19:12.390628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:50.497452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391657+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:19:12.391657+0000","last_peered":"2023-07-21T22:19:12.391657+0000","last_clean":"2023-07-21T22:19:12.391657+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:19:12.391657+0000","last_undegraded":"2023-07-21T22:19:12.391657+0000","last_fullsized":"2023-07-21T22:19:12.391657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:02:21.094817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055475+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:19:13.055475+0000","last_peered":"2023-07-21T22:19:13.055475+0000","last_clean":"2023-07-21T22:19:13.055475+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:19:13.055475+0000","last_undegraded":"2023-07-21T22:19:13.055475+0000","last_fullsized":"2023-07-21T22:19:13.055475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:42:33.190178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769967+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:19:12.769967+0000","last_peered":"2023-07-21T22:19:12.769967+0000","last_clean":"2023-07-21T22:19:12.769967+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:19:12.769967+0000","last_undegraded":"2023-07-21T22:19:12.769967+0000","last_fullsized":"2023-07-21T22:19:12.769967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:56:54.780234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391262+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:19:12.391262+0000","last_peered":"2023-07-21T22:19:12.391262+0000","last_clean":"2023-07-21T22:19:12.391262+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:19:12.391262+0000","last_undegraded":"2023-07-21T22:19:12.391262+0000","last_fullsized":"2023-07-21T22:19:12.391262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:44:54.733609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054867+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:19:13.054867+0000","last_peered":"2023-07-21T22:19:13.054867+0000","last_clean":"2023-07-21T22:19:13.054867+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:19:13.054867+0000","last_undegraded":"2023-07-21T22:19:13.054867+0000","last_fullsized":"2023-07-21T22:19:13.054867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:46.115924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305359+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:19:36.305359+0000","last_peered":"2023-07-21T22:19:36.305359+0000","last_clean":"2023-07-21T22:19:36.305359+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:19:36.305359+0000","last_undegraded":"2023-07-21T22:19:36.305359+0000","last_fullsized":"2023-07-21T22:19:36.305359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:10:05.495352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.303339+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:19:33.303339+0000","last_peered":"2023-07-21T22:19:33.303339+0000","last_clean":"2023-07-21T22:19:33.303339+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:19:33.303339+0000","last_undegraded":"2023-07-21T22:19:33.303339+0000","last_fullsized":"2023-07-21T22:19:33.303339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:47.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770270+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:19:12.770270+0000","last_peered":"2023-07-21T22:19:12.770270+0000","last_clean":"2023-07-21T22:19:12.770270+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:19:12.770270+0000","last_undegraded":"2023-07-21T22:19:12.770270+0000","last_fullsized":"2023-07-21T22:19:12.770270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:52:08.464869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055281+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:19:13.055281+0000","last_peered":"2023-07-21T22:19:13.055281+0000","last_clean":"2023-07-21T22:19:13.055281+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:19:13.055281+0000","last_undegraded":"2023-07-21T22:19:13.055281+0000","last_fullsized":"2023-07-21T22:19:13.055281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:12:18.699993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:34.307478+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:19:34.307478+0000","last_peered":"2023-07-21T22:19:34.307478+0000","last_clean":"2023-07-21T22:19:34.307478+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:19:34.307478+0000","last_undegraded":"2023-07-21T22:19:34.307478+0000","last_fullsized":"2023-07-21T22:19:34.307478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:25.582023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769709+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:19:12.769709+0000","last_peered":"2023-07-21T22:19:12.769709+0000","last_clean":"2023-07-21T22:19:12.769709+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:19:12.769709+0000","last_undegraded":"2023-07-21T22:19:12.769709+0000","last_fullsized":"2023-07-21T22:19:12.769709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:37.375418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304854+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:19:36.304854+0000","last_peered":"2023-07-21T22:19:36.304854+0000","last_clean":"2023-07-21T22:19:36.304854+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:19:36.304854+0000","last_undegraded":"2023-07-21T22:19:36.304854+0000","last_fullsized":"2023-07-21T22:19:36.304854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:26.951333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054386+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:19:13.054386+0000","last_peered":"2023-07-21T22:19:13.054386+0000","last_clean":"2023-07-21T22:19:13.054386+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:19:13.054386+0000","last_undegraded":"2023-07-21T22:19:13.054386+0000","last_fullsized":"2023-07-21T22:19:13.054386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:08:44.365646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.300052+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:19:33.300052+0000","last_peered":"2023-07-21T22:19:33.300052+0000","last_clean":"2023-07-21T22:19:33.300052+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:19:33.300052+0000","last_undegraded":"2023-07-21T22:19:33.300052+0000","last_fullsized":"2023-07-21T22:19:33.300052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:54.851783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634906+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:19:12.634906+0000","last_peered":"2023-07-21T22:19:12.634906+0000","last_clean":"2023-07-21T22:19:12.634906+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:19:12.634906+0000","last_undegraded":"2023-07-21T22:19:12.634906+0000","last_fullsized":"2023-07-21T22:19:12.634906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:14:49.440660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.294135+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:19:32.294135+0000","last_peered":"2023-07-21T22:19:32.294135+0000","last_clean":"2023-07-21T22:19:32.294135+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:19:32.294135+0000","last_undegraded":"2023-07-21T22:19:32.294135+0000","last_fullsized":"2023-07-21T22:19:32.294135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:11.723720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055557+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:19:13.055557+0000","last_peered":"2023-07-21T22:19:13.055557+0000","last_clean":"2023-07-21T22:19:13.055557+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:19:13.055557+0000","last_undegraded":"2023-07-21T22:19:13.055557+0000","last_fullsized":"2023-07-21T22:19:13.055557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:51.326201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054165+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:19:13.054165+0000","last_peered":"2023-07-21T22:19:13.054165+0000","last_clean":"2023-07-21T22:19:13.054165+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:19:13.054165+0000","last_undegraded":"2023-07-21T22:19:13.054165+0000","last_fullsized":"2023-07-21T22:19:13.054165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:29:36.603496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.301079+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:19:33.301079+0000","last_peered":"2023-07-21T22:19:33.301079+0000","last_clean":"2023-07-21T22:19:33.301079+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:19:33.301079+0000","last_undegraded":"2023-07-21T22:19:33.301079+0000","last_fullsized":"2023-07-21T22:19:33.301079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:19:02.491269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.056053+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:19:13.056053+0000","last_peered":"2023-07-21T22:19:13.056053+0000","last_clean":"2023-07-21T22:19:13.056053+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:19:13.056053+0000","last_undegraded":"2023-07-21T22:19:13.056053+0000","last_fullsized":"2023-07-21T22:19:13.056053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:30:00.515628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390408+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:19:12.390408+0000","last_peered":"2023-07-21T22:19:12.390408+0000","last_clean":"2023-07-21T22:19:12.390408+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:19:12.390408+0000","last_undegraded":"2023-07-21T22:19:12.390408+0000","last_fullsized":"2023-07-21T22:19:12.390408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:44:50.086359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.289823+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:19:32.289823+0000","last_peered":"2023-07-21T22:19:32.289823+0000","last_clean":"2023-07-21T22:19:32.289823+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:19:32.289823+0000","last_undegraded":"2023-07-21T22:19:32.289823+0000","last_fullsized":"2023-07-21T22:19:32.289823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:00:41.255335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054658+0000","last_change":"2023-07-21T22:19:07.381395+0000","last_active":"2023-07-21T22:19:13.054658+0000","last_peered":"2023-07-21T22:19:13.054658+0000","last_clean":"2023-07-21T22:19:13.054658+0000","last_became_active":"2023-07-21T22:19:07.381077+0000","last_became_peered":"2023-07-21T22:19:07.381077+0000","last_unstale":"2023-07-21T22:19:13.054658+0000","last_undegraded":"2023-07-21T22:19:13.054658+0000","last_fullsized":"2023-07-21T22:19:13.054658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:40.595370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054182+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:19:13.054182+0000","last_peered":"2023-07-21T22:19:13.054182+0000","last_clean":"2023-07-21T22:19:13.054182+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:19:13.054182+0000","last_undegraded":"2023-07-21T22:19:13.054182+0000","last_fullsized":"2023-07-21T22:19:13.054182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:05.648258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.294204+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:19:32.294204+0000","last_peered":"2023-07-21T22:19:32.294204+0000","last_clean":"2023-07-21T22:19:32.294204+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:19:32.294204+0000","last_undegraded":"2023-07-21T22:19:32.294204+0000","last_fullsized":"2023-07-21T22:19:32.294204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:03.976195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053966+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:19:13.053966+0000","last_peered":"2023-07-21T22:19:13.053966+0000","last_clean":"2023-07-21T22:19:13.053966+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:19:13.053966+0000","last_undegraded":"2023-07-21T22:19:13.053966+0000","last_fullsized":"2023-07-21T22:19:13.053966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:52:35.548078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390653+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:19:12.390653+0000","last_peered":"2023-07-21T22:19:12.390653+0000","last_clean":"2023-07-21T22:19:12.390653+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:19:12.390653+0000","last_undegraded":"2023-07-21T22:19:12.390653+0000","last_fullsized":"2023-07-21T22:19:12.390653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:28:55.634783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.290626+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:19:32.290626+0000","last_peered":"2023-07-21T22:19:32.290626+0000","last_clean":"2023-07-21T22:19:32.290626+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:19:32.290626+0000","last_undegraded":"2023-07-21T22:19:32.290626+0000","last_fullsized":"2023-07-21T22:19:32.290626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:00:38.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634593+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:19:12.634593+0000","last_peered":"2023-07-21T22:19:12.634593+0000","last_clean":"2023-07-21T22:19:12.634593+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:19:12.634593+0000","last_undegraded":"2023-07-21T22:19:12.634593+0000","last_fullsized":"2023-07-21T22:19:12.634593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:09:57.461798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054351+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:19:13.054351+0000","last_peered":"2023-07-21T22:19:13.054351+0000","last_clean":"2023-07-21T22:19:13.054351+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:19:13.054351+0000","last_undegraded":"2023-07-21T22:19:13.054351+0000","last_fullsized":"2023-07-21T22:19:13.054351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:39:31.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390329+0000","last_change":"2023-07-21T22:19:06.370025+0000","last_active":"2023-07-21T22:19:12.390329+0000","last_peered":"2023-07-21T22:19:12.390329+0000","last_clean":"2023-07-21T22:19:12.390329+0000","last_became_active":"2023-07-21T22:19:06.369722+0000","last_became_peered":"2023-07-21T22:19:06.369722+0000","last_unstale":"2023-07-21T22:19:12.390329+0000","last_undegraded":"2023-07-21T22:19:12.390329+0000","last_fullsized":"2023-07-21T22:19:12.390329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:27:56.521833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055418+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:19:13.055418+0000","last_peered":"2023-07-21T22:19:13.055418+0000","last_clean":"2023-07-21T22:19:13.055418+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:19:13.055418+0000","last_undegraded":"2023-07-21T22:19:13.055418+0000","last_fullsized":"2023-07-21T22:19:13.055418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:56:10.176571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054652+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:19:13.054652+0000","last_peered":"2023-07-21T22:19:13.054652+0000","last_clean":"2023-07-21T22:19:13.054652+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:19:13.054652+0000","last_undegraded":"2023-07-21T22:19:13.054652+0000","last_fullsized":"2023-07-21T22:19:13.054652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:41:56.117032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.294090+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:19:32.294090+0000","last_peered":"2023-07-21T22:19:32.294090+0000","last_clean":"2023-07-21T22:19:32.294090+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:19:32.294090+0000","last_undegraded":"2023-07-21T22:19:32.294090+0000","last_fullsized":"2023-07-21T22:19:32.294090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:10.462016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770062+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:19:12.770062+0000","last_peered":"2023-07-21T22:19:12.770062+0000","last_clean":"2023-07-21T22:19:12.770062+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:19:12.770062+0000","last_undegraded":"2023-07-21T22:19:12.770062+0000","last_fullsized":"2023-07-21T22:19:12.770062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:20.580549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053801+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:19:13.053801+0000","last_peered":"2023-07-21T22:19:13.053801+0000","last_clean":"2023-07-21T22:19:13.053801+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:19:13.053801+0000","last_undegraded":"2023-07-21T22:19:13.053801+0000","last_fullsized":"2023-07-21T22:19:13.053801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:11.576796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.302834+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:19:33.302834+0000","last_peered":"2023-07-21T22:19:33.302834+0000","last_clean":"2023-07-21T22:19:33.302834+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:19:33.302834+0000","last_undegraded":"2023-07-21T22:19:33.302834+0000","last_fullsized":"2023-07-21T22:19:33.302834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:52.750263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770434+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:19:12.770434+0000","last_peered":"2023-07-21T22:19:12.770434+0000","last_clean":"2023-07-21T22:19:12.770434+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:19:12.770434+0000","last_undegraded":"2023-07-21T22:19:12.770434+0000","last_fullsized":"2023-07-21T22:19:12.770434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:07:54.175591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054312+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:19:13.054312+0000","last_peered":"2023-07-21T22:19:13.054312+0000","last_clean":"2023-07-21T22:19:13.054312+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:19:13.054312+0000","last_undegraded":"2023-07-21T22:19:13.054312+0000","last_fullsized":"2023-07-21T22:19:13.054312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:39.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305532+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:19:36.305532+0000","last_peered":"2023-07-21T22:19:36.305532+0000","last_clean":"2023-07-21T22:19:36.305532+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:19:36.305532+0000","last_undegraded":"2023-07-21T22:19:36.305532+0000","last_fullsized":"2023-07-21T22:19:36.305532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:12.092223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770202+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:19:12.770202+0000","last_peered":"2023-07-21T22:19:12.770202+0000","last_clean":"2023-07-21T22:19:12.770202+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:19:12.770202+0000","last_undegraded":"2023-07-21T22:19:12.770202+0000","last_fullsized":"2023-07-21T22:19:12.770202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:08:01.010451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304086+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:19:36.304086+0000","last_peered":"2023-07-21T22:19:36.304086+0000","last_clean":"2023-07-21T22:19:36.304086+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:19:36.304086+0000","last_undegraded":"2023-07-21T22:19:36.304086+0000","last_fullsized":"2023-07-21T22:19:36.304086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:34.722629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054845+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:19:13.054845+0000","last_peered":"2023-07-21T22:19:13.054845+0000","last_clean":"2023-07-21T22:19:13.054845+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:19:13.054845+0000","last_undegraded":"2023-07-21T22:19:13.054845+0000","last_fullsized":"2023-07-21T22:19:13.054845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:01:21.549206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770435+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:19:12.770435+0000","last_peered":"2023-07-21T22:19:12.770435+0000","last_clean":"2023-07-21T22:19:12.770435+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:19:12.770435+0000","last_undegraded":"2023-07-21T22:19:12.770435+0000","last_fullsized":"2023-07-21T22:19:12.770435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:14.503720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634680+0000","last_change":"2023-07-21T22:19:07.372870+0000","last_active":"2023-07-21T22:19:12.634680+0000","last_peered":"2023-07-21T22:19:12.634680+0000","last_clean":"2023-07-21T22:19:12.634680+0000","last_became_active":"2023-07-21T22:19:07.372558+0000","last_became_peered":"2023-07-21T22:19:07.372558+0000","last_unstale":"2023-07-21T22:19:12.634680+0000","last_undegraded":"2023-07-21T22:19:12.634680+0000","last_fullsized":"2023-07-21T22:19:12.634680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:13.533905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390926+0000","last_change":"2023-07-21T22:19:07.374976+0000","last_active":"2023-07-21T22:19:12.390926+0000","last_peered":"2023-07-21T22:19:12.390926+0000","last_clean":"2023-07-21T22:19:12.390926+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:19:12.390926+0000","last_undegraded":"2023-07-21T22:19:12.390926+0000","last_fullsized":"2023-07-21T22:19:12.390926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:00.488258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390498+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:19:12.390498+0000","last_peered":"2023-07-21T22:19:12.390498+0000","last_clean":"2023-07-21T22:19:12.390498+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:19:12.390498+0000","last_undegraded":"2023-07-21T22:19:12.390498+0000","last_fullsized":"2023-07-21T22:19:12.390498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:12:14.641612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054622+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:19:13.054622+0000","last_peered":"2023-07-21T22:19:13.054622+0000","last_clean":"2023-07-21T22:19:13.054622+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:19:13.054622+0000","last_undegraded":"2023-07-21T22:19:13.054622+0000","last_fullsized":"2023-07-21T22:19:13.054622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:13:37.517124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055264+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:19:13.055264+0000","last_peered":"2023-07-21T22:19:13.055264+0000","last_clean":"2023-07-21T22:19:13.055264+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:19:13.055264+0000","last_undegraded":"2023-07-21T22:19:13.055264+0000","last_fullsized":"2023-07-21T22:19:13.055264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:29:13.974954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635182+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:19:12.635182+0000","last_peered":"2023-07-21T22:19:12.635182+0000","last_clean":"2023-07-21T22:19:12.635182+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:19:12.635182+0000","last_undegraded":"2023-07-21T22:19:12.635182+0000","last_fullsized":"2023-07-21T22:19:12.635182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:11:16.616530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390906+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:19:12.390906+0000","last_peered":"2023-07-21T22:19:12.390906+0000","last_clean":"2023-07-21T22:19:12.390906+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:19:12.390906+0000","last_undegraded":"2023-07-21T22:19:12.390906+0000","last_fullsized":"2023-07-21T22:19:12.390906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:18.494870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.056170+0000","last_change":"2023-07-21T22:19:07.382715+0000","last_active":"2023-07-21T22:19:13.056170+0000","last_peered":"2023-07-21T22:19:13.056170+0000","last_clean":"2023-07-21T22:19:13.056170+0000","last_became_active":"2023-07-21T22:19:07.382208+0000","last_became_peered":"2023-07-21T22:19:07.382208+0000","last_unstale":"2023-07-21T22:19:13.056170+0000","last_undegraded":"2023-07-21T22:19:13.056170+0000","last_fullsized":"2023-07-21T22:19:13.056170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:49:38.138026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305396+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:19:36.305396+0000","last_peered":"2023-07-21T22:19:36.305396+0000","last_clean":"2023-07-21T22:19:36.305396+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:19:36.305396+0000","last_undegraded":"2023-07-21T22:19:36.305396+0000","last_fullsized":"2023-07-21T22:19:36.305396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:56:57.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054928+0000","last_change":"2023-07-21T22:19:07.381609+0000","last_active":"2023-07-21T22:19:13.054928+0000","last_peered":"2023-07-21T22:19:13.054928+0000","last_clean":"2023-07-21T22:19:13.054928+0000","last_became_active":"2023-07-21T22:19:07.381272+0000","last_became_peered":"2023-07-21T22:19:07.381272+0000","last_unstale":"2023-07-21T22:19:13.054928+0000","last_undegraded":"2023-07-21T22:19:13.054928+0000","last_fullsized":"2023-07-21T22:19:13.054928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:09.205823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054247+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:19:13.054247+0000","last_peered":"2023-07-21T22:19:13.054247+0000","last_clean":"2023-07-21T22:19:13.054247+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:19:13.054247+0000","last_undegraded":"2023-07-21T22:19:13.054247+0000","last_fullsized":"2023-07-21T22:19:13.054247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:08:08.505369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055378+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:19:13.055378+0000","last_peered":"2023-07-21T22:19:13.055378+0000","last_clean":"2023-07-21T22:19:13.055378+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:19:13.055378+0000","last_undegraded":"2023-07-21T22:19:13.055378+0000","last_fullsized":"2023-07-21T22:19:13.055378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:52.683920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635042+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:19:12.635042+0000","last_peered":"2023-07-21T22:19:12.635042+0000","last_clean":"2023-07-21T22:19:12.635042+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:19:12.635042+0000","last_undegraded":"2023-07-21T22:19:12.635042+0000","last_fullsized":"2023-07-21T22:19:12.635042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:37.145657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8088,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":126,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":22130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":473302,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.752},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.78600000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612628,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":1044480,"data_stored":1067838,"data_compressed":47429,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.5}]},{"osd":2,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.83799999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.79400000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.88800000000000001}]}]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":477634,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.496}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.67700000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.58899999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.74399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":473458,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.747,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.45700000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77100000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73299999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81000000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82999999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.438}]},{"osd":5,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.751,"15min":0.751},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75600000000000001,"5min":1.083,"15min":1.083},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94799999999999995,"5min":1.111,"15min":1.111},"last":0.627}]},{"osd":9,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72399999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1064244,"data_compressed":47429,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.442,"15min":0.443},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65000000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.623},"min":{"1min":0.434,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.68999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.53800000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.81200000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":472720,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.438},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73599999999999999,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2210000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92800000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.56599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0389999999999999,"5min":1.145,"15min":1.145},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1910000000000001,"5min":1.284,"15min":1.284},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.495},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1140000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.081,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80600000000000005,"15min":0.79800000000000004},"min":{"1min":0.58599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3240000000000001,"5min":1.369,"15min":1.369},"last":0.86599999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.63,"5min":0.622,"15min":0.622},"max":{"1min":1.1970000000000001,"5min":1.599,"15min":1.599},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.90800000000000003},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3049999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:18:40 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.876}]},{"osd":10,"last update":"Fri Jul 21 22:18:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":476514,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.31,"15min":0.31},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.878,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.149,"15min":1.149},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.37,"15min":0.37},"max":{"1min":0.85699999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.29299999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.443,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.627,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.022,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97699999999999998,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Jul 21 22:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60799999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.84399999999999997}]},{"osd":9,"last update":"Fri Jul 21 22:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.628},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.65300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":474198,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57299999999999995,"15min":0.51600000000000001},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88800000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.63700000000000001,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Jul 21 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70099999999999996,"15min":0.63800000000000001},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0609999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67800000000000005,"15min":0.63},"min":{"1min":0.623,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":1.151,"15min":1.151},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0289999999999999,"5min":1.095,"15min":1.095},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.55100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.98699999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71999999999999997,"15min":0.69799999999999995},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.101,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93700000000000006,"5min":1.081,"15min":1.081},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.74099999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.752},{"interface":"front","average":{"1min":0.879,"5min":0.90900000000000003,"15min":0.91400000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1850000000000001,"5min":1.335,"15min":1.335},"last":0.60599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.84199999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":477122,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.317,"15min":0.317},"max":{"1min":1.9530000000000001,"5min":3.988,"15min":3.988},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.54500000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93100000000000005,"5min":1.984,"15min":1.984},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.69899999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7889999999999999,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.628}]},{"osd":5,"last update":"Fri Jul 21 22:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45400000000000001,"15min":0.441},"min":{"1min":0.311,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47399999999999998,"15min":0.45600000000000002},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.70099999999999996,"15min":0.68799999999999994},"min":{"1min":0.628,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.871}]},{"osd":8,"last update":"Fri Jul 21 22:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.88800000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.96299999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.044},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.022}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":472914,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73399999999999999,"5min":0.877,"15min":0.877},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.41399999999999998},"min":{"1min":0.39900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.748}]},{"osd":2,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0289999999999999,"5min":1.139,"15min":1.139},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":1.02,"15min":1.02},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.64500000000000002,"15min":0.61199999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.64100000000000001},"min":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":1.002,"15min":1.002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.69199999999999995},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.375,"15min":0.375},"max":{"1min":0.99399999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.56699999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0529999999999999,"5min":1.139,"15min":1.139},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.155,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.873}]},{"osd":8,"last update":"Fri Jul 21 22:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.018},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":473706,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.375},"min":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71799999999999997,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.38}]},{"osd":2,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.46500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88400000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.53000000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88100000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.75},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.49,"15min":1.49},"last":0.87}]},{"osd":7,"last update":"Fri Jul 21 22:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.50800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77900000000000003,"15min":0.77600000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3740000000000001,"5min":2.202,"15min":2.202},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.62}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":1052672,"data_stored":1078512,"data_compressed":50383,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.41399999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.95499999999999996,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.44,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.374}]},{"osd":3,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63700000000000001,"15min":0.59699999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0980000000000001,"5min":1.647,"15min":1.647},"last":0.627},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98799999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.871,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70499999999999996,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.796,"15min":1.796},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.53200000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97699999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.749},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67200000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:19:37.863 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:19:38.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 -- 172.21.15.66:0/3927306805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc0100cf0 msgr2=0x7f2cc0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:38.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/3927306805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc0100cf0 0x7f2cc0101140 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009990 tx=0x7f2cb0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 -- 172.21.15.66:0/3927306805 shutdown_connections 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/3927306805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cc0101680 0x7f2cc0107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/3927306805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc00ffb00 0x7f2cc00fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/3927306805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc0100cf0 0x7f2cc0101140 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 -- 172.21.15.66:0/3927306805 >> 172.21.15.66:0/3927306805 conn(0x7f2cc00fb0e0 msgr2=0x7f2cc00fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 -- 172.21.15.66:0/3927306805 shutdown_connections 2023-07-21T22:19:38.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.052+0000 7f2cc7fff700 1 -- 172.21.15.66:0/3927306805 wait complete. 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.054+0000 7f2cc7fff700 1 Processor -- start 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.054+0000 7f2cc7fff700 1 -- start start 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 0x7f2cc0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0100cf0 0x7f2cc0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cc0101680 0x7f2cc019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc7fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc019f590 con 0x7f2cc00ffb00 2023-07-21T22:19:38.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc7fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc019f6d0 con 0x7f2cc0101680 2023-07-21T22:19:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc7fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc019f860 con 0x7f2cc0100cf0 2023-07-21T22:19:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 0x7f2cc0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 0x7f2cc0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47030/0 (socket says 172.21.15.66:47030) 2023-07-21T22:19:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 -- 172.21.15.66:0/894838608 learned_addr learned my addr 172.21.15.66:0/894838608 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc559a700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0100cf0 0x7f2cc0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:38.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc659c700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cc0101680 0x7f2cc019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:38.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 -- 172.21.15.66:0/894838608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0100cf0 msgr2=0x7f2cc0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:38.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0100cf0 0x7f2cc0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 -- 172.21.15.66:0/894838608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cc0101680 msgr2=0x7f2cc019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cc0101680 0x7f2cc019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.055+0000 7f2cc5d9b700 1 -- 172.21.15.66:0/894838608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb00095f0 con 0x7f2cc00ffb00 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.056+0000 7f2cc5d9b700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 0x7f2cc0197a10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d980 tx=0x7f2cbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.056+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00f890 con 0x7f2cc00ffb00 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.056+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc019fb40 con 0x7f2cc00ffb00 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.056+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cbc010460 con 0x7f2cc00ffb00 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.056+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0195c0 con 0x7f2cc00ffb00 2023-07-21T22:19:38.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.056+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc01a0050 con 0x7f2cc00ffb00 2023-07-21T22:19:38.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.057+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc010b320 con 0x7f2cc00ffb00 2023-07-21T22:19:38.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.059+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f2cbc0105d0 con 0x7f2cc00ffb00 2023-07-21T22:19:38.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.060+0000 7f2cb6ffd700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2cac0a1220 0x7f2cac0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:38.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.061+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2cbc0ccb70 con 0x7f2cc00ffb00 2023-07-21T22:19:38.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.061+0000 7f2cc559a700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2cac0a1220 0x7f2cac0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:38.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.061+0000 7f2cc559a700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2cac0a1220 0x7f2cac0a36d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b2e0 tx=0x7f2cb0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:38.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.065+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2cbc095df0 con 0x7f2cc00ffb00 2023-07-21T22:19:38.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.273+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2cc0107cb0 con 0x7f2cac0a1220 2023-07-21T22:19:38.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.286+0000 7f2cb6ffd700 1 -- 172.21.15.66:0/894838608 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422017 (secure 0 0 0) 0x7f2cc0107cb0 con 0x7f2cac0a1220 2023-07-21T22:19:38.338 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:38.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2cac0a1220 msgr2=0x7f2cac0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2cac0a1220 0x7f2cac0a36d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000b2e0 tx=0x7f2cb0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 msgr2=0x7f2cc0197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:38.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 0x7f2cc0197a10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc00d980 tx=0x7f2cbc00dc90 comp rx=0 tx=0).stop 2023-07-21T22:19:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 shutdown_connections 2023-07-21T22:19:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2cac0a1220 0x7f2cac0a36d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cc0101680 0x7f2cc019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2cc00ffb00 0x7f2cc0197a10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 --2- 172.21.15.66:0/894838608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cc0100cf0 0x7f2cc0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 >> 172.21.15.66:0/894838608 conn(0x7f2cc00fb0e0 msgr2=0x7f2cc0106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 shutdown_connections 2023-07-21T22:19:38.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:38.291+0000 7f2cc7fff700 1 -- 172.21.15.66:0/894838608 wait complete. 2023-07-21T22:19:38.345 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:19:38.355 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-07-21T22:19:38.183653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633177,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":175,"num_read_kb":184,"num_write":321,"num_write_kb":3021,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":261,"ondisk_log_size":261,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335144,"kb_used_data":11720,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606680,"statfs":{"total":1151940427776,"available":1151597240320,"internally_reserved":0,"allocated":12001280,"data_stored":7581154,"data_compressed":258935,"data_compressed_allocated":348160,"data_compressed_original":2195456,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38358,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":39,"num_read_kb":65,"num_write":92,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002300"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635357+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:19:12.635357+0000","last_peered":"2023-07-21T22:19:12.635357+0000","last_clean":"2023-07-21T22:19:12.635357+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:19:12.635357+0000","last_undegraded":"2023-07-21T22:19:12.635357+0000","last_fullsized":"2023-07-21T22:19:12.635357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:43:30.586945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054535+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:19:13.054535+0000","last_peered":"2023-07-21T22:19:13.054535+0000","last_clean":"2023-07-21T22:19:13.054535+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:19:13.054535+0000","last_undegraded":"2023-07-21T22:19:13.054535+0000","last_fullsized":"2023-07-21T22:19:13.054535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:58:15.473345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054499+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:19:13.054499+0000","last_peered":"2023-07-21T22:19:13.054499+0000","last_clean":"2023-07-21T22:19:13.054499+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:19:13.054499+0000","last_undegraded":"2023-07-21T22:19:13.054499+0000","last_fullsized":"2023-07-21T22:19:13.054499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:22:25.508998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054820+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:19:13.054820+0000","last_peered":"2023-07-21T22:19:13.054820+0000","last_clean":"2023-07-21T22:19:13.054820+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:19:13.054820+0000","last_undegraded":"2023-07-21T22:19:13.054820+0000","last_fullsized":"2023-07-21T22:19:13.054820+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:37:05.611733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054111+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:19:13.054111+0000","last_peered":"2023-07-21T22:19:13.054111+0000","last_clean":"2023-07-21T22:19:13.054111+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:19:13.054111+0000","last_undegraded":"2023-07-21T22:19:13.054111+0000","last_fullsized":"2023-07-21T22:19:13.054111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:43:30.768314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309427+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:19:37.309427+0000","last_peered":"2023-07-21T22:19:37.309427+0000","last_clean":"2023-07-21T22:19:37.309427+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:19:37.309427+0000","last_undegraded":"2023-07-21T22:19:37.309427+0000","last_fullsized":"2023-07-21T22:19:37.309427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:08.453027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769784+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:19:12.769784+0000","last_peered":"2023-07-21T22:19:12.769784+0000","last_clean":"2023-07-21T22:19:12.769784+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:19:12.769784+0000","last_undegraded":"2023-07-21T22:19:12.769784+0000","last_fullsized":"2023-07-21T22:19:12.769784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:44.363556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054369+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:19:13.054369+0000","last_peered":"2023-07-21T22:19:13.054369+0000","last_clean":"2023-07-21T22:19:13.054369+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:19:13.054369+0000","last_undegraded":"2023-07-21T22:19:13.054369+0000","last_fullsized":"2023-07-21T22:19:13.054369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:49:33.535575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310974+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:19:37.310974+0000","last_peered":"2023-07-21T22:19:37.310974+0000","last_clean":"2023-07-21T22:19:37.310974+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:19:37.310974+0000","last_undegraded":"2023-07-21T22:19:37.310974+0000","last_fullsized":"2023-07-21T22:19:37.310974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:05.850547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310192+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:19:37.310192+0000","last_peered":"2023-07-21T22:19:37.310192+0000","last_clean":"2023-07-21T22:19:37.310192+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:19:37.310192+0000","last_undegraded":"2023-07-21T22:19:37.310192+0000","last_fullsized":"2023-07-21T22:19:37.310192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:21:52.428612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770333+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:19:12.770333+0000","last_peered":"2023-07-21T22:19:12.770333+0000","last_clean":"2023-07-21T22:19:12.770333+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:19:12.770333+0000","last_undegraded":"2023-07-21T22:19:12.770333+0000","last_fullsized":"2023-07-21T22:19:12.770333+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:11.614161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054708+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:19:13.054708+0000","last_peered":"2023-07-21T22:19:13.054708+0000","last_clean":"2023-07-21T22:19:13.054708+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:19:13.054708+0000","last_undegraded":"2023-07-21T22:19:13.054708+0000","last_fullsized":"2023-07-21T22:19:13.054708+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:05:24.932631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055253+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:19:13.055253+0000","last_peered":"2023-07-21T22:19:13.055253+0000","last_clean":"2023-07-21T22:19:13.055253+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:19:13.055253+0000","last_undegraded":"2023-07-21T22:19:13.055253+0000","last_fullsized":"2023-07-21T22:19:13.055253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:26:26.315690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054077+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:19:13.054077+0000","last_peered":"2023-07-21T22:19:13.054077+0000","last_clean":"2023-07-21T22:19:13.054077+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:19:13.054077+0000","last_undegraded":"2023-07-21T22:19:13.054077+0000","last_fullsized":"2023-07-21T22:19:13.054077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:19:52.027627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054172+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:19:13.054172+0000","last_peered":"2023-07-21T22:19:13.054172+0000","last_clean":"2023-07-21T22:19:13.054172+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:19:13.054172+0000","last_undegraded":"2023-07-21T22:19:13.054172+0000","last_fullsized":"2023-07-21T22:19:13.054172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:29:41.950899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309872+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:19:37.309872+0000","last_peered":"2023-07-21T22:19:37.309872+0000","last_clean":"2023-07-21T22:19:37.309872+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:19:37.309872+0000","last_undegraded":"2023-07-21T22:19:37.309872+0000","last_fullsized":"2023-07-21T22:19:37.309872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:46:06.325343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635220+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:19:12.635220+0000","last_peered":"2023-07-21T22:19:12.635220+0000","last_clean":"2023-07-21T22:19:12.635220+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:19:12.635220+0000","last_undegraded":"2023-07-21T22:19:12.635220+0000","last_fullsized":"2023-07-21T22:19:12.635220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:45:57.291197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390640+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:19:12.390640+0000","last_peered":"2023-07-21T22:19:12.390640+0000","last_clean":"2023-07-21T22:19:12.390640+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:19:12.390640+0000","last_undegraded":"2023-07-21T22:19:12.390640+0000","last_fullsized":"2023-07-21T22:19:12.390640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:35:56.391831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391012+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:19:12.391012+0000","last_peered":"2023-07-21T22:19:12.391012+0000","last_clean":"2023-07-21T22:19:12.391012+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:19:12.391012+0000","last_undegraded":"2023-07-21T22:19:12.391012+0000","last_fullsized":"2023-07-21T22:19:12.391012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:51:04.808102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390711+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:19:12.390711+0000","last_peered":"2023-07-21T22:19:12.390711+0000","last_clean":"2023-07-21T22:19:12.390711+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:19:12.390711+0000","last_undegraded":"2023-07-21T22:19:12.390711+0000","last_fullsized":"2023-07-21T22:19:12.390711+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:45:58.482547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.303878+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:19:33.303878+0000","last_peered":"2023-07-21T22:19:33.303878+0000","last_clean":"2023-07-21T22:19:33.303878+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:19:33.303878+0000","last_undegraded":"2023-07-21T22:19:33.303878+0000","last_fullsized":"2023-07-21T22:19:33.303878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:32:17.121216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055956+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:19:13.055956+0000","last_peered":"2023-07-21T22:19:13.055956+0000","last_clean":"2023-07-21T22:19:13.055956+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:19:13.055956+0000","last_undegraded":"2023-07-21T22:19:13.055956+0000","last_fullsized":"2023-07-21T22:19:13.055956+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:42:47.152694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053650+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:19:13.053650+0000","last_peered":"2023-07-21T22:19:13.053650+0000","last_clean":"2023-07-21T22:19:13.053650+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:19:13.053650+0000","last_undegraded":"2023-07-21T22:19:13.053650+0000","last_fullsized":"2023-07-21T22:19:13.053650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:15:19.457080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.056271+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:19:13.056271+0000","last_peered":"2023-07-21T22:19:13.056271+0000","last_clean":"2023-07-21T22:19:13.056271+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:19:13.056271+0000","last_undegraded":"2023-07-21T22:19:13.056271+0000","last_fullsized":"2023-07-21T22:19:13.056271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:19:59.319714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055599+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:19:13.055599+0000","last_peered":"2023-07-21T22:19:13.055599+0000","last_clean":"2023-07-21T22:19:13.055599+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:19:13.055599+0000","last_undegraded":"2023-07-21T22:19:13.055599+0000","last_fullsized":"2023-07-21T22:19:13.055599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:05:46.461070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054710+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:19:13.054710+0000","last_peered":"2023-07-21T22:19:13.054710+0000","last_clean":"2023-07-21T22:19:13.054710+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:19:13.054710+0000","last_undegraded":"2023-07-21T22:19:13.054710+0000","last_fullsized":"2023-07-21T22:19:13.054710+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:47:59.853903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.300206+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:19:33.300206+0000","last_peered":"2023-07-21T22:19:33.300206+0000","last_clean":"2023-07-21T22:19:33.300206+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:19:33.300206+0000","last_undegraded":"2023-07-21T22:19:33.300206+0000","last_fullsized":"2023-07-21T22:19:33.300206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:03:04.510706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311511+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:19:37.311511+0000","last_peered":"2023-07-21T22:19:37.311511+0000","last_clean":"2023-07-21T22:19:37.311511+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:19:37.311511+0000","last_undegraded":"2023-07-21T22:19:37.311511+0000","last_fullsized":"2023-07-21T22:19:37.311511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:10:47.470299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054061+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:19:13.054061+0000","last_peered":"2023-07-21T22:19:13.054061+0000","last_clean":"2023-07-21T22:19:13.054061+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:19:13.054061+0000","last_undegraded":"2023-07-21T22:19:13.054061+0000","last_fullsized":"2023-07-21T22:19:13.054061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:03:23.370924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054319+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:19:13.054319+0000","last_peered":"2023-07-21T22:19:13.054319+0000","last_clean":"2023-07-21T22:19:13.054319+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:19:13.054319+0000","last_undegraded":"2023-07-21T22:19:13.054319+0000","last_fullsized":"2023-07-21T22:19:13.054319+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:01:46.744100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305422+0000","last_change":"2023-07-21T22:19:07.373299+0000","last_active":"2023-07-21T22:19:36.305422+0000","last_peered":"2023-07-21T22:19:36.305422+0000","last_clean":"2023-07-21T22:19:36.305422+0000","last_became_active":"2023-07-21T22:19:07.372944+0000","last_became_peered":"2023-07-21T22:19:07.372944+0000","last_unstale":"2023-07-21T22:19:36.305422+0000","last_undegraded":"2023-07-21T22:19:36.305422+0000","last_fullsized":"2023-07-21T22:19:36.305422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:05:06.676917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.391017+0000","last_change":"2023-07-21T22:19:11.387155+0000","last_active":"2023-07-21T22:19:12.391017+0000","last_peered":"2023-07-21T22:19:12.391017+0000","last_clean":"2023-07-21T22:19:12.391017+0000","last_became_active":"2023-07-21T22:19:11.386756+0000","last_became_peered":"2023-07-21T22:19:11.386756+0000","last_unstale":"2023-07-21T22:19:12.391017+0000","last_undegraded":"2023-07-21T22:19:12.391017+0000","last_fullsized":"2023-07-21T22:19:12.391017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:08:36.187925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304737+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:19:36.304737+0000","last_peered":"2023-07-21T22:19:36.304737+0000","last_clean":"2023-07-21T22:19:36.304737+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:19:36.304737+0000","last_undegraded":"2023-07-21T22:19:36.304737+0000","last_fullsized":"2023-07-21T22:19:36.304737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:27:22.286359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054124+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:19:13.054124+0000","last_peered":"2023-07-21T22:19:13.054124+0000","last_clean":"2023-07-21T22:19:13.054124+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:19:13.054124+0000","last_undegraded":"2023-07-21T22:19:13.054124+0000","last_fullsized":"2023-07-21T22:19:13.054124+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:57:28.616635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390704+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:19:12.390704+0000","last_peered":"2023-07-21T22:19:12.390704+0000","last_clean":"2023-07-21T22:19:12.390704+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:19:12.390704+0000","last_undegraded":"2023-07-21T22:19:12.390704+0000","last_fullsized":"2023-07-21T22:19:12.390704+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:50:05.930511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.328942+0000","last_change":"2023-07-21T22:19:37.328942+0000","last_active":"2023-07-21T22:19:37.328942+0000","last_peered":"2023-07-21T22:19:37.328942+0000","last_clean":"2023-07-21T22:19:37.328942+0000","last_became_active":"2023-07-21T22:19:37.325738+0000","last_became_peered":"2023-07-21T22:19:37.325738+0000","last_unstale":"2023-07-21T22:19:37.328942+0000","last_undegraded":"2023-07-21T22:19:37.328942+0000","last_fullsized":"2023-07-21T22:19:37.328942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:49:28.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304829+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:19:36.304829+0000","last_peered":"2023-07-21T22:19:36.304829+0000","last_clean":"2023-07-21T22:19:36.304829+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:19:36.304829+0000","last_undegraded":"2023-07-21T22:19:36.304829+0000","last_fullsized":"2023-07-21T22:19:36.304829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:18:41.783365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635249+0000","last_change":"2023-07-21T22:19:11.382069+0000","last_active":"2023-07-21T22:19:12.635249+0000","last_peered":"2023-07-21T22:19:12.635249+0000","last_clean":"2023-07-21T22:19:12.635249+0000","last_became_active":"2023-07-21T22:19:11.381316+0000","last_became_peered":"2023-07-21T22:19:11.381316+0000","last_unstale":"2023-07-21T22:19:12.635249+0000","last_undegraded":"2023-07-21T22:19:12.635249+0000","last_fullsized":"2023-07-21T22:19:12.635249+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:08:47.464945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.302925+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:19:33.302925+0000","last_peered":"2023-07-21T22:19:33.302925+0000","last_clean":"2023-07-21T22:19:33.302925+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:19:33.302925+0000","last_undegraded":"2023-07-21T22:19:33.302925+0000","last_fullsized":"2023-07-21T22:19:33.302925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:03:56.587172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054249+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:19:13.054249+0000","last_peered":"2023-07-21T22:19:13.054249+0000","last_clean":"2023-07-21T22:19:13.054249+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:19:13.054249+0000","last_undegraded":"2023-07-21T22:19:13.054249+0000","last_fullsized":"2023-07-21T22:19:13.054249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:38.969371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054444+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:19:13.054444+0000","last_peered":"2023-07-21T22:19:13.054444+0000","last_clean":"2023-07-21T22:19:13.054444+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:19:13.054444+0000","last_undegraded":"2023-07-21T22:19:13.054444+0000","last_fullsized":"2023-07-21T22:19:13.054444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:57.047349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304370+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:19:36.304370+0000","last_peered":"2023-07-21T22:19:36.304370+0000","last_clean":"2023-07-21T22:19:36.304370+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:19:36.304370+0000","last_undegraded":"2023-07-21T22:19:36.304370+0000","last_fullsized":"2023-07-21T22:19:36.304370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:04.271052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390382+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:19:12.390382+0000","last_peered":"2023-07-21T22:19:12.390382+0000","last_clean":"2023-07-21T22:19:12.390382+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:19:12.390382+0000","last_undegraded":"2023-07-21T22:19:12.390382+0000","last_fullsized":"2023-07-21T22:19:12.390382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:20:02.329601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054721+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:19:13.054721+0000","last_peered":"2023-07-21T22:19:13.054721+0000","last_clean":"2023-07-21T22:19:13.054721+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:19:13.054721+0000","last_undegraded":"2023-07-21T22:19:13.054721+0000","last_fullsized":"2023-07-21T22:19:13.054721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:20:42.835815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310421+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:19:37.310421+0000","last_peered":"2023-07-21T22:19:37.310421+0000","last_clean":"2023-07-21T22:19:37.310421+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:19:37.310421+0000","last_undegraded":"2023-07-21T22:19:37.310421+0000","last_fullsized":"2023-07-21T22:19:37.310421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:26:25.437396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311461+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:19:37.311461+0000","last_peered":"2023-07-21T22:19:37.311461+0000","last_clean":"2023-07-21T22:19:37.311461+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:19:37.311461+0000","last_undegraded":"2023-07-21T22:19:37.311461+0000","last_fullsized":"2023-07-21T22:19:37.311461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:04:20.100122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326929+0000","last_change":"2023-07-21T22:19:37.326317+0000","last_active":"2023-07-21T22:19:37.326929+0000","last_peered":"2023-07-21T22:19:37.326929+0000","last_clean":"2023-07-21T22:19:37.326929+0000","last_became_active":"2023-07-21T22:19:37.325791+0000","last_became_peered":"2023-07-21T22:19:37.325791+0000","last_unstale":"2023-07-21T22:19:37.326929+0000","last_undegraded":"2023-07-21T22:19:37.326929+0000","last_fullsized":"2023-07-21T22:19:37.326929+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:11:24.661772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.293677+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:19:32.293677+0000","last_peered":"2023-07-21T22:19:32.293677+0000","last_clean":"2023-07-21T22:19:32.293677+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:19:32.293677+0000","last_undegraded":"2023-07-21T22:19:32.293677+0000","last_fullsized":"2023-07-21T22:19:32.293677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:54:02.828429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055718+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:19:13.055718+0000","last_peered":"2023-07-21T22:19:13.055718+0000","last_clean":"2023-07-21T22:19:13.055718+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:19:13.055718+0000","last_undegraded":"2023-07-21T22:19:13.055718+0000","last_fullsized":"2023-07-21T22:19:13.055718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:52.030405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054925+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:19:13.054925+0000","last_peered":"2023-07-21T22:19:13.054925+0000","last_clean":"2023-07-21T22:19:13.054925+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:19:13.054925+0000","last_undegraded":"2023-07-21T22:19:13.054925+0000","last_fullsized":"2023-07-21T22:19:13.054925+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:41:33.631869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305147+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:19:36.305147+0000","last_peered":"2023-07-21T22:19:36.305147+0000","last_clean":"2023-07-21T22:19:36.305147+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:19:36.305147+0000","last_undegraded":"2023-07-21T22:19:36.305147+0000","last_fullsized":"2023-07-21T22:19:36.305147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:17.946189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311593+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:19:37.311593+0000","last_peered":"2023-07-21T22:19:37.311593+0000","last_clean":"2023-07-21T22:19:37.311593+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:19:37.311593+0000","last_undegraded":"2023-07-21T22:19:37.311593+0000","last_fullsized":"2023-07-21T22:19:37.311593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:07:46.707479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390479+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:19:12.390479+0000","last_peered":"2023-07-21T22:19:12.390479+0000","last_clean":"2023-07-21T22:19:12.390479+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:19:12.390479+0000","last_undegraded":"2023-07-21T22:19:12.390479+0000","last_fullsized":"2023-07-21T22:19:12.390479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:05.071713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.321109+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:19:37.321109+0000","last_peered":"2023-07-21T22:19:37.321109+0000","last_clean":"2023-07-21T22:19:37.321109+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:19:37.321109+0000","last_undegraded":"2023-07-21T22:19:37.321109+0000","last_fullsized":"2023-07-21T22:19:37.321109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:08.163210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":12,"num_read_kb":43,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054380+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:19:13.054380+0000","last_peered":"2023-07-21T22:19:13.054380+0000","last_clean":"2023-07-21T22:19:13.054380+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:19:13.054380+0000","last_undegraded":"2023-07-21T22:19:13.054380+0000","last_fullsized":"2023-07-21T22:19:13.054380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:45:18.050114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305755+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:19:36.305755+0000","last_peered":"2023-07-21T22:19:36.305755+0000","last_clean":"2023-07-21T22:19:36.305755+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:19:36.305755+0000","last_undegraded":"2023-07-21T22:19:36.305755+0000","last_fullsized":"2023-07-21T22:19:36.305755+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.622432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.299636+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:19:33.299636+0000","last_peered":"2023-07-21T22:19:33.299636+0000","last_clean":"2023-07-21T22:19:33.299636+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:19:33.299636+0000","last_undegraded":"2023-07-21T22:19:33.299636+0000","last_fullsized":"2023-07-21T22:19:33.299636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:44:22.810764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054829+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:19:13.054829+0000","last_peered":"2023-07-21T22:19:13.054829+0000","last_clean":"2023-07-21T22:19:13.054829+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:19:13.054829+0000","last_undegraded":"2023-07-21T22:19:13.054829+0000","last_fullsized":"2023-07-21T22:19:13.054829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:28:45.448195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634964+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:19:12.634964+0000","last_peered":"2023-07-21T22:19:12.634964+0000","last_clean":"2023-07-21T22:19:12.634964+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:19:12.634964+0000","last_undegraded":"2023-07-21T22:19:12.634964+0000","last_fullsized":"2023-07-21T22:19:12.634964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:16.588357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304505+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:19:36.304505+0000","last_peered":"2023-07-21T22:19:36.304505+0000","last_clean":"2023-07-21T22:19:36.304505+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:19:36.304505+0000","last_undegraded":"2023-07-21T22:19:36.304505+0000","last_fullsized":"2023-07-21T22:19:36.304505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:00:45.003028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769962+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:19:12.769962+0000","last_peered":"2023-07-21T22:19:12.769962+0000","last_clean":"2023-07-21T22:19:12.769962+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:19:12.769962+0000","last_undegraded":"2023-07-21T22:19:12.769962+0000","last_fullsized":"2023-07-21T22:19:12.769962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:34:05.872357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390541+0000","last_change":"2023-07-21T22:19:11.394298+0000","last_active":"2023-07-21T22:19:12.390541+0000","last_peered":"2023-07-21T22:19:12.390541+0000","last_clean":"2023-07-21T22:19:12.390541+0000","last_became_active":"2023-07-21T22:19:11.390666+0000","last_became_peered":"2023-07-21T22:19:11.390666+0000","last_unstale":"2023-07-21T22:19:12.390541+0000","last_undegraded":"2023-07-21T22:19:12.390541+0000","last_fullsized":"2023-07-21T22:19:12.390541+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:19:17.395604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:35.316183+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:19:35.316183+0000","last_peered":"2023-07-21T22:19:35.316183+0000","last_clean":"2023-07-21T22:19:35.316183+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:19:35.316183+0000","last_undegraded":"2023-07-21T22:19:35.316183+0000","last_fullsized":"2023-07-21T22:19:35.316183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:19.859228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769881+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:19:12.769881+0000","last_peered":"2023-07-21T22:19:12.769881+0000","last_clean":"2023-07-21T22:19:12.769881+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:19:12.769881+0000","last_undegraded":"2023-07-21T22:19:12.769881+0000","last_fullsized":"2023-07-21T22:19:12.769881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:29:38.425684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054739+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:19:13.054739+0000","last_peered":"2023-07-21T22:19:13.054739+0000","last_clean":"2023-07-21T22:19:13.054739+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:19:13.054739+0000","last_undegraded":"2023-07-21T22:19:13.054739+0000","last_fullsized":"2023-07-21T22:19:13.054739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:23:55.867317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.302776+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:19:33.302776+0000","last_peered":"2023-07-21T22:19:33.302776+0000","last_clean":"2023-07-21T22:19:33.302776+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:19:33.302776+0000","last_undegraded":"2023-07-21T22:19:33.302776+0000","last_fullsized":"2023-07-21T22:19:33.302776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:11.608929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304559+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:19:36.304559+0000","last_peered":"2023-07-21T22:19:36.304559+0000","last_clean":"2023-07-21T22:19:36.304559+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:19:36.304559+0000","last_undegraded":"2023-07-21T22:19:36.304559+0000","last_fullsized":"2023-07-21T22:19:36.304559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:47.513193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304416+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:19:36.304416+0000","last_peered":"2023-07-21T22:19:36.304416+0000","last_clean":"2023-07-21T22:19:36.304416+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:19:36.304416+0000","last_undegraded":"2023-07-21T22:19:36.304416+0000","last_fullsized":"2023-07-21T22:19:36.304416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:30.723357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054561+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:19:13.054561+0000","last_peered":"2023-07-21T22:19:13.054561+0000","last_clean":"2023-07-21T22:19:13.054561+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:19:13.054561+0000","last_undegraded":"2023-07-21T22:19:13.054561+0000","last_fullsized":"2023-07-21T22:19:13.054561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:10:12.419549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635103+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:19:12.635103+0000","last_peered":"2023-07-21T22:19:12.635103+0000","last_clean":"2023-07-21T22:19:12.635103+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:19:12.635103+0000","last_undegraded":"2023-07-21T22:19:12.635103+0000","last_fullsized":"2023-07-21T22:19:12.635103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:33:45.036406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054185+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:19:13.054185+0000","last_peered":"2023-07-21T22:19:13.054185+0000","last_clean":"2023-07-21T22:19:13.054185+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:19:13.054185+0000","last_undegraded":"2023-07-21T22:19:13.054185+0000","last_fullsized":"2023-07-21T22:19:13.054185+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:36:15.202635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390628+0000","last_change":"2023-07-21T22:19:06.370251+0000","last_active":"2023-07-21T22:19:12.390628+0000","last_peered":"2023-07-21T22:19:12.390628+0000","last_clean":"2023-07-21T22:19:12.390628+0000","last_became_active":"2023-07-21T22:19:06.369838+0000","last_became_peered":"2023-07-21T22:19:06.369838+0000","last_unstale":"2023-07-21T22:19:12.390628+0000","last_undegraded":"2023-07-21T22:19:12.390628+0000","last_fullsized":"2023-07-21T22:19:12.390628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:50.497452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310127+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:19:37.310127+0000","last_peered":"2023-07-21T22:19:37.310127+0000","last_clean":"2023-07-21T22:19:37.310127+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:19:37.310127+0000","last_undegraded":"2023-07-21T22:19:37.310127+0000","last_fullsized":"2023-07-21T22:19:37.310127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:22:26.812058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055475+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:19:13.055475+0000","last_peered":"2023-07-21T22:19:13.055475+0000","last_clean":"2023-07-21T22:19:13.055475+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:19:13.055475+0000","last_undegraded":"2023-07-21T22:19:13.055475+0000","last_fullsized":"2023-07-21T22:19:13.055475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:42:33.190178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769967+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:19:12.769967+0000","last_peered":"2023-07-21T22:19:12.769967+0000","last_clean":"2023-07-21T22:19:12.769967+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:19:12.769967+0000","last_undegraded":"2023-07-21T22:19:12.769967+0000","last_fullsized":"2023-07-21T22:19:12.769967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:56:54.780234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309432+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:19:37.309432+0000","last_peered":"2023-07-21T22:19:37.309432+0000","last_clean":"2023-07-21T22:19:37.309432+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:19:37.309432+0000","last_undegraded":"2023-07-21T22:19:37.309432+0000","last_fullsized":"2023-07-21T22:19:37.309432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:25:52.984944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054867+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:19:13.054867+0000","last_peered":"2023-07-21T22:19:13.054867+0000","last_clean":"2023-07-21T22:19:13.054867+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:19:13.054867+0000","last_undegraded":"2023-07-21T22:19:13.054867+0000","last_fullsized":"2023-07-21T22:19:13.054867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:46.115924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305359+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:19:36.305359+0000","last_peered":"2023-07-21T22:19:36.305359+0000","last_clean":"2023-07-21T22:19:36.305359+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:19:36.305359+0000","last_undegraded":"2023-07-21T22:19:36.305359+0000","last_fullsized":"2023-07-21T22:19:36.305359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:10:05.495352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.303339+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:19:33.303339+0000","last_peered":"2023-07-21T22:19:33.303339+0000","last_clean":"2023-07-21T22:19:33.303339+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:19:33.303339+0000","last_undegraded":"2023-07-21T22:19:33.303339+0000","last_fullsized":"2023-07-21T22:19:33.303339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:47.217683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770270+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:19:12.770270+0000","last_peered":"2023-07-21T22:19:12.770270+0000","last_clean":"2023-07-21T22:19:12.770270+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:19:12.770270+0000","last_undegraded":"2023-07-21T22:19:12.770270+0000","last_fullsized":"2023-07-21T22:19:12.770270+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:52:08.464869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055281+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:19:13.055281+0000","last_peered":"2023-07-21T22:19:13.055281+0000","last_clean":"2023-07-21T22:19:13.055281+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:19:13.055281+0000","last_undegraded":"2023-07-21T22:19:13.055281+0000","last_fullsized":"2023-07-21T22:19:13.055281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:12:18.699993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:34.307478+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:19:34.307478+0000","last_peered":"2023-07-21T22:19:34.307478+0000","last_clean":"2023-07-21T22:19:34.307478+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:19:34.307478+0000","last_undegraded":"2023-07-21T22:19:34.307478+0000","last_fullsized":"2023-07-21T22:19:34.307478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:25.582023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":354,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.769709+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:19:12.769709+0000","last_peered":"2023-07-21T22:19:12.769709+0000","last_clean":"2023-07-21T22:19:12.769709+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:19:12.769709+0000","last_undegraded":"2023-07-21T22:19:12.769709+0000","last_fullsized":"2023-07-21T22:19:12.769709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:37.375418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304854+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:19:36.304854+0000","last_peered":"2023-07-21T22:19:36.304854+0000","last_clean":"2023-07-21T22:19:36.304854+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:19:36.304854+0000","last_undegraded":"2023-07-21T22:19:36.304854+0000","last_fullsized":"2023-07-21T22:19:36.304854+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:09:26.951333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054386+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:19:13.054386+0000","last_peered":"2023-07-21T22:19:13.054386+0000","last_clean":"2023-07-21T22:19:13.054386+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:19:13.054386+0000","last_undegraded":"2023-07-21T22:19:13.054386+0000","last_fullsized":"2023-07-21T22:19:13.054386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:08:44.365646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.300052+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:19:33.300052+0000","last_peered":"2023-07-21T22:19:33.300052+0000","last_clean":"2023-07-21T22:19:33.300052+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:19:33.300052+0000","last_undegraded":"2023-07-21T22:19:33.300052+0000","last_fullsized":"2023-07-21T22:19:33.300052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:54.851783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634906+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:19:12.634906+0000","last_peered":"2023-07-21T22:19:12.634906+0000","last_clean":"2023-07-21T22:19:12.634906+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:19:12.634906+0000","last_undegraded":"2023-07-21T22:19:12.634906+0000","last_fullsized":"2023-07-21T22:19:12.634906+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:14:49.440660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.294135+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:19:32.294135+0000","last_peered":"2023-07-21T22:19:32.294135+0000","last_clean":"2023-07-21T22:19:32.294135+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:19:32.294135+0000","last_undegraded":"2023-07-21T22:19:32.294135+0000","last_fullsized":"2023-07-21T22:19:32.294135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:11.723720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055557+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:19:13.055557+0000","last_peered":"2023-07-21T22:19:13.055557+0000","last_clean":"2023-07-21T22:19:13.055557+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:19:13.055557+0000","last_undegraded":"2023-07-21T22:19:13.055557+0000","last_fullsized":"2023-07-21T22:19:13.055557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:51.326201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054165+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:19:13.054165+0000","last_peered":"2023-07-21T22:19:13.054165+0000","last_clean":"2023-07-21T22:19:13.054165+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:19:13.054165+0000","last_undegraded":"2023-07-21T22:19:13.054165+0000","last_fullsized":"2023-07-21T22:19:13.054165+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:29:36.603496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.301079+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:19:33.301079+0000","last_peered":"2023-07-21T22:19:33.301079+0000","last_clean":"2023-07-21T22:19:33.301079+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:19:33.301079+0000","last_undegraded":"2023-07-21T22:19:33.301079+0000","last_fullsized":"2023-07-21T22:19:33.301079+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:19:02.491269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.056053+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:19:13.056053+0000","last_peered":"2023-07-21T22:19:13.056053+0000","last_clean":"2023-07-21T22:19:13.056053+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:19:13.056053+0000","last_undegraded":"2023-07-21T22:19:13.056053+0000","last_fullsized":"2023-07-21T22:19:13.056053+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:30:00.515628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390408+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:19:12.390408+0000","last_peered":"2023-07-21T22:19:12.390408+0000","last_clean":"2023-07-21T22:19:12.390408+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:19:12.390408+0000","last_undegraded":"2023-07-21T22:19:12.390408+0000","last_fullsized":"2023-07-21T22:19:12.390408+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:44:50.086359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.289823+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:19:32.289823+0000","last_peered":"2023-07-21T22:19:32.289823+0000","last_clean":"2023-07-21T22:19:32.289823+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:19:32.289823+0000","last_undegraded":"2023-07-21T22:19:32.289823+0000","last_fullsized":"2023-07-21T22:19:32.289823+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:00:41.255335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054658+0000","last_change":"2023-07-21T22:19:07.381395+0000","last_active":"2023-07-21T22:19:13.054658+0000","last_peered":"2023-07-21T22:19:13.054658+0000","last_clean":"2023-07-21T22:19:13.054658+0000","last_became_active":"2023-07-21T22:19:07.381077+0000","last_became_peered":"2023-07-21T22:19:07.381077+0000","last_unstale":"2023-07-21T22:19:13.054658+0000","last_undegraded":"2023-07-21T22:19:13.054658+0000","last_fullsized":"2023-07-21T22:19:13.054658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:40.595370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054182+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:19:13.054182+0000","last_peered":"2023-07-21T22:19:13.054182+0000","last_clean":"2023-07-21T22:19:13.054182+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:19:13.054182+0000","last_undegraded":"2023-07-21T22:19:13.054182+0000","last_fullsized":"2023-07-21T22:19:13.054182+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:34:05.648258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310235+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:19:37.310235+0000","last_peered":"2023-07-21T22:19:37.310235+0000","last_clean":"2023-07-21T22:19:37.310235+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:19:37.310235+0000","last_undegraded":"2023-07-21T22:19:37.310235+0000","last_fullsized":"2023-07-21T22:19:37.310235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:17:04.984129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.053966+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:19:13.053966+0000","last_peered":"2023-07-21T22:19:13.053966+0000","last_clean":"2023-07-21T22:19:13.053966+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:19:13.053966+0000","last_undegraded":"2023-07-21T22:19:13.053966+0000","last_fullsized":"2023-07-21T22:19:13.053966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:52:35.548078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390653+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:19:12.390653+0000","last_peered":"2023-07-21T22:19:12.390653+0000","last_clean":"2023-07-21T22:19:12.390653+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:19:12.390653+0000","last_undegraded":"2023-07-21T22:19:12.390653+0000","last_fullsized":"2023-07-21T22:19:12.390653+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:28:55.634783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.290626+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:19:32.290626+0000","last_peered":"2023-07-21T22:19:32.290626+0000","last_clean":"2023-07-21T22:19:32.290626+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:19:32.290626+0000","last_undegraded":"2023-07-21T22:19:32.290626+0000","last_fullsized":"2023-07-21T22:19:32.290626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:00:38.698345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634593+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:19:12.634593+0000","last_peered":"2023-07-21T22:19:12.634593+0000","last_clean":"2023-07-21T22:19:12.634593+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:19:12.634593+0000","last_undegraded":"2023-07-21T22:19:12.634593+0000","last_fullsized":"2023-07-21T22:19:12.634593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:09:57.461798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054351+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:19:13.054351+0000","last_peered":"2023-07-21T22:19:13.054351+0000","last_clean":"2023-07-21T22:19:13.054351+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:19:13.054351+0000","last_undegraded":"2023-07-21T22:19:13.054351+0000","last_fullsized":"2023-07-21T22:19:13.054351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:39:31.892673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390329+0000","last_change":"2023-07-21T22:19:06.370025+0000","last_active":"2023-07-21T22:19:12.390329+0000","last_peered":"2023-07-21T22:19:12.390329+0000","last_clean":"2023-07-21T22:19:12.390329+0000","last_became_active":"2023-07-21T22:19:06.369722+0000","last_became_peered":"2023-07-21T22:19:06.369722+0000","last_unstale":"2023-07-21T22:19:12.390329+0000","last_undegraded":"2023-07-21T22:19:12.390329+0000","last_fullsized":"2023-07-21T22:19:12.390329+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:27:56.521833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055418+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:19:13.055418+0000","last_peered":"2023-07-21T22:19:13.055418+0000","last_clean":"2023-07-21T22:19:13.055418+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:19:13.055418+0000","last_undegraded":"2023-07-21T22:19:13.055418+0000","last_fullsized":"2023-07-21T22:19:13.055418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:56:10.176571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054652+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:19:13.054652+0000","last_peered":"2023-07-21T22:19:13.054652+0000","last_clean":"2023-07-21T22:19:13.054652+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:19:13.054652+0000","last_undegraded":"2023-07-21T22:19:13.054652+0000","last_fullsized":"2023-07-21T22:19:13.054652+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:41:56.117032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:32.294090+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:19:32.294090+0000","last_peered":"2023-07-21T22:19:32.294090+0000","last_clean":"2023-07-21T22:19:32.294090+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:19:32.294090+0000","last_undegraded":"2023-07-21T22:19:32.294090+0000","last_fullsized":"2023-07-21T22:19:32.294090+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:22:10.462016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770062+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:19:12.770062+0000","last_peered":"2023-07-21T22:19:12.770062+0000","last_clean":"2023-07-21T22:19:12.770062+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:19:12.770062+0000","last_undegraded":"2023-07-21T22:19:12.770062+0000","last_fullsized":"2023-07-21T22:19:12.770062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:41:20.580549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310346+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:19:37.310346+0000","last_peered":"2023-07-21T22:19:37.310346+0000","last_clean":"2023-07-21T22:19:37.310346+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:19:37.310346+0000","last_undegraded":"2023-07-21T22:19:37.310346+0000","last_fullsized":"2023-07-21T22:19:37.310346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:46.183044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:33.302834+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:19:33.302834+0000","last_peered":"2023-07-21T22:19:33.302834+0000","last_clean":"2023-07-21T22:19:33.302834+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:19:33.302834+0000","last_undegraded":"2023-07-21T22:19:33.302834+0000","last_fullsized":"2023-07-21T22:19:33.302834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:52.750263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770434+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:19:12.770434+0000","last_peered":"2023-07-21T22:19:12.770434+0000","last_clean":"2023-07-21T22:19:12.770434+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:19:12.770434+0000","last_undegraded":"2023-07-21T22:19:12.770434+0000","last_fullsized":"2023-07-21T22:19:12.770434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:07:54.175591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054312+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:19:13.054312+0000","last_peered":"2023-07-21T22:19:13.054312+0000","last_clean":"2023-07-21T22:19:13.054312+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:19:13.054312+0000","last_undegraded":"2023-07-21T22:19:13.054312+0000","last_fullsized":"2023-07-21T22:19:13.054312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:39.191463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305532+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:19:36.305532+0000","last_peered":"2023-07-21T22:19:36.305532+0000","last_clean":"2023-07-21T22:19:36.305532+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:19:36.305532+0000","last_undegraded":"2023-07-21T22:19:36.305532+0000","last_fullsized":"2023-07-21T22:19:36.305532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:12.092223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770202+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:19:12.770202+0000","last_peered":"2023-07-21T22:19:12.770202+0000","last_clean":"2023-07-21T22:19:12.770202+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:19:12.770202+0000","last_undegraded":"2023-07-21T22:19:12.770202+0000","last_fullsized":"2023-07-21T22:19:12.770202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:08:01.010451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.304086+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:19:36.304086+0000","last_peered":"2023-07-21T22:19:36.304086+0000","last_clean":"2023-07-21T22:19:36.304086+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:19:36.304086+0000","last_undegraded":"2023-07-21T22:19:36.304086+0000","last_fullsized":"2023-07-21T22:19:36.304086+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:34.722629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054845+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:19:13.054845+0000","last_peered":"2023-07-21T22:19:13.054845+0000","last_clean":"2023-07-21T22:19:13.054845+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:19:13.054845+0000","last_undegraded":"2023-07-21T22:19:13.054845+0000","last_fullsized":"2023-07-21T22:19:13.054845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:01:21.549206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.770435+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:19:12.770435+0000","last_peered":"2023-07-21T22:19:12.770435+0000","last_clean":"2023-07-21T22:19:12.770435+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:19:12.770435+0000","last_undegraded":"2023-07-21T22:19:12.770435+0000","last_fullsized":"2023-07-21T22:19:12.770435+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:31:14.503720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.634680+0000","last_change":"2023-07-21T22:19:07.372870+0000","last_active":"2023-07-21T22:19:12.634680+0000","last_peered":"2023-07-21T22:19:12.634680+0000","last_clean":"2023-07-21T22:19:12.634680+0000","last_became_active":"2023-07-21T22:19:07.372558+0000","last_became_peered":"2023-07-21T22:19:07.372558+0000","last_unstale":"2023-07-21T22:19:12.634680+0000","last_undegraded":"2023-07-21T22:19:12.634680+0000","last_fullsized":"2023-07-21T22:19:12.634680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:13.533905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390926+0000","last_change":"2023-07-21T22:19:07.374976+0000","last_active":"2023-07-21T22:19:12.390926+0000","last_peered":"2023-07-21T22:19:12.390926+0000","last_clean":"2023-07-21T22:19:12.390926+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:19:12.390926+0000","last_undegraded":"2023-07-21T22:19:12.390926+0000","last_fullsized":"2023-07-21T22:19:12.390926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:00.488258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390498+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:19:12.390498+0000","last_peered":"2023-07-21T22:19:12.390498+0000","last_clean":"2023-07-21T22:19:12.390498+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:19:12.390498+0000","last_undegraded":"2023-07-21T22:19:12.390498+0000","last_fullsized":"2023-07-21T22:19:12.390498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:12:14.641612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054622+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:19:13.054622+0000","last_peered":"2023-07-21T22:19:13.054622+0000","last_clean":"2023-07-21T22:19:13.054622+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:19:13.054622+0000","last_undegraded":"2023-07-21T22:19:13.054622+0000","last_fullsized":"2023-07-21T22:19:13.054622+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:13:37.517124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055264+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:19:13.055264+0000","last_peered":"2023-07-21T22:19:13.055264+0000","last_clean":"2023-07-21T22:19:13.055264+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:19:13.055264+0000","last_undegraded":"2023-07-21T22:19:13.055264+0000","last_fullsized":"2023-07-21T22:19:13.055264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:29:13.974954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635182+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:19:12.635182+0000","last_peered":"2023-07-21T22:19:12.635182+0000","last_clean":"2023-07-21T22:19:12.635182+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:19:12.635182+0000","last_undegraded":"2023-07-21T22:19:12.635182+0000","last_fullsized":"2023-07-21T22:19:12.635182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:11:16.616530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.390906+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:19:12.390906+0000","last_peered":"2023-07-21T22:19:12.390906+0000","last_clean":"2023-07-21T22:19:12.390906+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:19:12.390906+0000","last_undegraded":"2023-07-21T22:19:12.390906+0000","last_fullsized":"2023-07-21T22:19:12.390906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:48:18.494870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.056170+0000","last_change":"2023-07-21T22:19:07.382715+0000","last_active":"2023-07-21T22:19:13.056170+0000","last_peered":"2023-07-21T22:19:13.056170+0000","last_clean":"2023-07-21T22:19:13.056170+0000","last_became_active":"2023-07-21T22:19:07.382208+0000","last_became_peered":"2023-07-21T22:19:07.382208+0000","last_unstale":"2023-07-21T22:19:13.056170+0000","last_undegraded":"2023-07-21T22:19:13.056170+0000","last_fullsized":"2023-07-21T22:19:13.056170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:49:38.138026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-07-21T22:19:36.305396+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:19:36.305396+0000","last_peered":"2023-07-21T22:19:36.305396+0000","last_clean":"2023-07-21T22:19:36.305396+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:19:36.305396+0000","last_undegraded":"2023-07-21T22:19:36.305396+0000","last_fullsized":"2023-07-21T22:19:36.305396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:56:57.994081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.054928+0000","last_change":"2023-07-21T22:19:07.381609+0000","last_active":"2023-07-21T22:19:13.054928+0000","last_peered":"2023-07-21T22:19:13.054928+0000","last_clean":"2023-07-21T22:19:13.054928+0000","last_became_active":"2023-07-21T22:19:07.381272+0000","last_became_peered":"2023-07-21T22:19:07.381272+0000","last_unstale":"2023-07-21T22:19:13.054928+0000","last_undegraded":"2023-07-21T22:19:13.054928+0000","last_fullsized":"2023-07-21T22:19:13.054928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:09.205823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310743+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:19:37.310743+0000","last_peered":"2023-07-21T22:19:37.310743+0000","last_clean":"2023-07-21T22:19:37.310743+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:19:37.310743+0000","last_undegraded":"2023-07-21T22:19:37.310743+0000","last_fullsized":"2023-07-21T22:19:37.310743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:47:16.929327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:13.055378+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:19:13.055378+0000","last_peered":"2023-07-21T22:19:13.055378+0000","last_clean":"2023-07-21T22:19:13.055378+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:19:13.055378+0000","last_undegraded":"2023-07-21T22:19:13.055378+0000","last_fullsized":"2023-07-21T22:19:13.055378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:10:52.683920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-07-21T22:19:12.635042+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:19:12.635042+0000","last_peered":"2023-07-21T22:19:12.635042+0000","last_clean":"2023-07-21T22:19:12.635042+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:19:12.635042+0000","last_undegraded":"2023-07-21T22:19:12.635042+0000","last_fullsized":"2023-07-21T22:19:12.635042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:37.145657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42809,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":39,"num_read_kb":65,"num_write":140,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":92536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":473302,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.49199999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.38200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.70299999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.627}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.752},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.83499999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.78600000000000003}]}]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1114112,"data_stored":1122599,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.71799999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.85399999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.80900000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.87}]},{"osd":11,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.93899999999999995}]}]},{"osd":9,"up_from":68,"seq":292057776153,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":477634,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.42499999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.496}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.67700000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.58899999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:01 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.74399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":473458,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79100000000000004,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.747,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.45700000000000002,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77100000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61799999999999999,"15min":0.60999999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.29099999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.34699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.40100000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73299999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81000000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82999999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.438}]},{"osd":5,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53500000000000003,"15min":0.53900000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.751,"15min":0.751},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73099999999999998,"15min":0.73699999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92400000000000004,"5min":1.002,"15min":1.002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71099999999999997,"15min":0.72199999999999998},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75600000000000001,"5min":1.083,"15min":1.083},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.44400000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69799999999999995,"15min":0.70199999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94799999999999995,"5min":1.111,"15min":1.111},"last":0.627}]},{"osd":9,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72399999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1064244,"data_compressed":47429,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.439,"15min":0.437},"min":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.442,"15min":0.443},"min":{"1min":0.30099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65000000000000002,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51100000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.38900000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.627,"15min":0.623},"min":{"1min":0.434,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64000000000000001,"15min":0.628},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.35499999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.32000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.70099999999999996,"5min":0.745,"15min":0.745},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.64000000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.68999999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.69699999999999995,"15min":0.68999999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69599999999999995,"15min":0.69999999999999996},"min":{"1min":0.53800000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.81200000000000006,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73999999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.49299999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80900000000000005}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.83799999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]}]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":472720,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.438},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73599999999999999,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2210000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92800000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.56599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0389999999999999,"5min":1.145,"15min":1.145},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1910000000000001,"5min":1.284,"15min":1.284},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.495},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1140000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77900000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.081,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.80600000000000005,"15min":0.79800000000000004},"min":{"1min":0.58599999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.879,"15min":0.86899999999999999},"min":{"1min":0.628,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3240000000000001,"5min":1.369,"15min":1.369},"last":0.86599999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.92800000000000005,"15min":0.93400000000000005},"min":{"1min":0.63,"5min":0.622,"15min":0.622},"max":{"1min":1.1970000000000001,"5min":1.599,"15min":1.599},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.90800000000000003},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3049999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.82799999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:18:40 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.876}]},{"osd":10,"last update":"Fri Jul 21 22:18:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.93500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":520745,"data_compressed":14602,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.317,"5min":0.31,"15min":0.31},"max":{"1min":0.89200000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.50600000000000001},"min":{"1min":0.36599999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.878,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Jul 21 22:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.55900000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77900000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.871},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.149,"15min":1.149},"last":0.751},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.37,"15min":0.37},"max":{"1min":0.85699999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Jul 21 22:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.42799999999999999,"15min":0.41999999999999998},"min":{"1min":0.29299999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.443,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40699999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.627,"15min":0.63500000000000001},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.022,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.97699999999999998,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.83199999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.68600000000000005,"15min":0.66600000000000004},"min":{"1min":0.52900000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.53500000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:18:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.40200000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.313,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.626}]},{"osd":8,"last update":"Fri Jul 21 22:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.45300000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.81699999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.84499999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.78800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":474198,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.57299999999999995,"15min":0.51600000000000001},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88800000000000001,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.014,"5min":0.63700000000000001,"15min":0.53200000000000003},"min":{"1min":0.438,"5min":0.379,"15min":0.379},"max":{"1min":5.4909999999999997,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Jul 21 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70099999999999996,"15min":0.63800000000000001},"min":{"1min":0.53500000000000003,"5min":0.436,"15min":0.436},"max":{"1min":1.0609999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.67800000000000005,"15min":0.63},"min":{"1min":0.623,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.121,"5min":1.151,"15min":1.151},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:17:25 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.503},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.54100000000000004,"15min":0.53400000000000003},"min":{"1min":0.38,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.48399999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.53200000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0289999999999999,"5min":1.095,"15min":1.095},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.55100000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.98699999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71999999999999997,"15min":0.69799999999999995},"min":{"1min":0.56599999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.101,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93700000000000006,"5min":1.081,"15min":1.081},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.48299999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.74099999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.752},{"interface":"front","average":{"1min":0.879,"5min":0.90900000000000003,"15min":0.91400000000000003},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.1850000000000001,"5min":1.335,"15min":1.335},"last":0.60599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.57899999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.84199999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.80700000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":477122,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.61899999999999999,"15min":0.64500000000000002},"min":{"1min":0.373,"5min":0.317,"15min":0.317},"max":{"1min":1.9530000000000001,"5min":3.988,"15min":3.988},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.40000000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57199999999999995,"15min":0.54500000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.56200000000000006,"15min":0.52200000000000002},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.50600000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93100000000000005,"5min":1.984,"15min":1.984},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.67200000000000004,"15min":0.69899999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.7889999999999999,"5min":4.0060000000000002,"15min":4.0060000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.629,"15min":0.61199999999999999},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.628}]},{"osd":5,"last update":"Fri Jul 21 22:19:03 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45400000000000001,"15min":0.441},"min":{"1min":0.311,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.47399999999999998,"15min":0.45600000000000002},"min":{"1min":0.34100000000000003,"5min":0.253,"15min":0.253},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:18:34 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.70099999999999996,"15min":0.68799999999999994},"min":{"1min":0.628,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.8360000000000001,"5min":1.8360000000000001,"15min":1.8360000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70399999999999996,"15min":0.69399999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.871}]},{"osd":8,"last update":"Fri Jul 21 22:19:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.88800000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":1},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.96299999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.044},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.022}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149749,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":472914,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73399999999999999,"5min":0.877,"15min":0.877},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.41399999999999998},"min":{"1min":0.39900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.748}]},{"osd":2,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0289999999999999,"5min":1.139,"15min":1.139},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.61099999999999999,"15min":0.58699999999999997},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":1.02,"15min":1.02},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.64500000000000002,"15min":0.61199999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:19:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65700000000000003,"15min":0.64100000000000001},"min":{"1min":0.48099999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.85699999999999998,"5min":1.002,"15min":1.002},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72099999999999997,"15min":0.70099999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.69199999999999995},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.375,"15min":0.375},"max":{"1min":0.99399999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.84999999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:18:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.70399999999999996,"15min":0.68500000000000005},"min":{"1min":0.56699999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0529999999999999,"5min":1.139,"15min":1.139},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.155,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.873}]},{"osd":8,"last update":"Fri Jul 21 22:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.78700000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:18:59 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.018},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.83499999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]},{"osd":4,"up_from":34,"seq":146028888113,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":473706,"data_compressed":11648,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.39500000000000002,"15min":0.375},"min":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.312,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71799999999999997,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.38}]},{"osd":2,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.60399999999999998,"15min":0.57899999999999996},"min":{"1min":0.432,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.58599999999999997,"15min":0.55800000000000005},"min":{"1min":0.46500000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88400000000000001,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.60099999999999998,"15min":0.57399999999999995},"min":{"1min":0.47199999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:19:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.996,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.64400000000000002,"15min":0.63200000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.53000000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88100000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.75},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.49,"15min":1.49},"last":0.87}]},{"osd":7,"last update":"Fri Jul 21 22:18:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.46300000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.50800000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51000000000000001}]},{"osd":8,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77900000000000003,"15min":0.77600000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3740000000000001,"5min":2.202,"15min":2.202},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65400000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.81200000000000006}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.62}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":1052672,"data_stored":1078512,"data_compressed":50383,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.56399999999999995,"15min":0.52400000000000002},"min":{"1min":0.41399999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.95499999999999996,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.53500000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.56899999999999995,"15min":0.53100000000000003},"min":{"1min":0.44,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":3.319,"5min":3.319,"15min":3.319},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.39000000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.876,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.39700000000000002},"min":{"1min":0.26800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69299999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.40899999999999997,"15min":0.38800000000000001},"min":{"1min":0.32600000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.374}]},{"osd":3,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63700000000000001,"15min":0.59699999999999998},"min":{"1min":0.44700000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0980000000000001,"5min":1.647,"15min":1.647},"last":0.627},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.62,"15min":0.59799999999999998},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.0329999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.46700000000000003,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.98799999999999999,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.871,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70499999999999996,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.796,"15min":1.796},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.53200000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97699999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.749},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.63200000000000001,"15min":0.61799999999999999},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67200000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.60099999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.879,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.79200000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:18:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:19:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.81000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":38515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:19:38.369 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-21T22:19:38.369 INFO:tasks.cephadm:Setting up client nodes... 2023-07-21T22:19:38.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-21T22:19:39.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:38 smithi104 ceph-mon[129021]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:19:39.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:38 smithi104 ceph-mon[129021]: pgmap v292: 129 pgs: 129 active+clean; 618 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:39.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:38 smithi104 ceph-mon[129021]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:19:39.140 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:38 smithi066 ceph-mon[126379]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:19:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:38 smithi066 ceph-mon[126379]: pgmap v292: 129 pgs: 129 active+clean; 618 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:39.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:38 smithi066 ceph-mon[126379]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:19:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:38 smithi149 ceph-mon[129268]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:19:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:38 smithi149 ceph-mon[129268]: pgmap v292: 129 pgs: 129 active+clean; 618 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.1 KiB/s wr, 16 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:38 smithi149 ceph-mon[129268]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:19:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 -- 172.21.15.66:0/3027660202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 msgr2=0x7f1b980fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:40.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/3027660202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b980fffc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009990 tx=0x7f1b88009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 -- 172.21.15.66:0/3027660202 shutdown_connections 2023-07-21T22:19:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/3027660202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98101740 0x7f1b98108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/3027660202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b980fffc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/3027660202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b98100db0 0x7f1b98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 -- 172.21.15.66:0/3027660202 >> 172.21.15.66:0/3027660202 conn(0x7f1b980fb200 msgr2=0x7f1b980fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:40.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 -- 172.21.15.66:0/3027660202 shutdown_connections 2023-07-21T22:19:40.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.526+0000 7f1b9dae3700 1 -- 172.21.15.66:0/3027660202 wait complete. 2023-07-21T22:19:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.527+0000 7f1b9dae3700 1 Processor -- start 2023-07-21T22:19:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.527+0000 7f1b9dae3700 1 -- start start 2023-07-21T22:19:40.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b9dae3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b981958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b9dae3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98100db0 0x7f1b98195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b9dae3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b98101740 0x7f1b9819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b9dae3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9819d470 con 0x7f1b980ffbb0 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b9dae3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9819d5b0 con 0x7f1b98101740 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b9dae3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9819d740 con 0x7f1b98100db0 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b977fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b981958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:40.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b977fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b981958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47060/0 (socket says 172.21.15.66:47060) 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b977fe700 1 -- 172.21.15.66:0/729294534 learned_addr learned my addr 172.21.15.66:0/729294534 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98100db0 0x7f1b98195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b97fff700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b98101740 0x7f1b9819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b96ffd700 1 -- 172.21.15.66:0/729294534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b98101740 msgr2=0x7f1b9819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b96ffd700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b98101740 0x7f1b9819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b96ffd700 1 -- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 msgr2=0x7f1b981958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:40.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b96ffd700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b981958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b96ffd700 1 -- 172.21.15.66:0/729294534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880095f0 con 0x7f1b98100db0 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.528+0000 7f1b977fe700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b981958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.529+0000 7f1b96ffd700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98100db0 0x7f1b98195e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00ea20 tx=0x7f1b8c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.529+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c00cbd0 con 0x7f1b98100db0 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.529+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9819da20 con 0x7f1b98100db0 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.529+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b8c00ee60 con 0x7f1b98100db0 2023-07-21T22:19:40.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.529+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c018680 con 0x7f1b98100db0 2023-07-21T22:19:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.529+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9819df70 con 0x7f1b98100db0 2023-07-21T22:19:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.530+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98107140 con 0x7f1b98100db0 2023-07-21T22:19:40.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.532+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1b8c010430 con 0x7f1b98100db0 2023-07-21T22:19:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.534+0000 7f1b94ff9700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1b800a1260 0x7f1b800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.534+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1b8c014070 con 0x7f1b98100db0 2023-07-21T22:19:40.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.534+0000 7f1b977fe700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1b800a1260 0x7f1b800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:40.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.535+0000 7f1b977fe700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1b800a1260 0x7f1b800a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b2e0 tx=0x7f1b88005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:40.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.538+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1b8c051770 con 0x7f1b98100db0 2023-07-21T22:19:40.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.826+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1b9802cb00 con 0x7f1b98100db0 2023-07-21T22:19:40.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.832+0000 7f1b94ff9700 1 -- 172.21.15.66:0/729294534 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1b8c096190 con 0x7f1b98100db0 2023-07-21T22:19:40.878 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2023-07-21T22:19:40.878 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQB8BLtkRyZmMRAA6z+NsvVj66UxEmtH7q8DOw== 2023-07-21T22:19:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1b800a1260 msgr2=0x7f1b800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1b800a1260 0x7f1b800a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b2e0 tx=0x7f1b88005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98100db0 msgr2=0x7f1b98195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98100db0 0x7f1b98195e20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00ea20 tx=0x7f1b8c00ed30 comp rx=0 tx=0).stop 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 shutdown_connections 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1b800a1260 0x7f1b800a3710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b98101740 0x7f1b9819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1b980ffbb0 0x7f1b981958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 --2- 172.21.15.66:0/729294534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1b98100db0 0x7f1b98195e20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:40.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 >> 172.21.15.66:0/729294534 conn(0x7f1b980fb200 msgr2=0x7f1b98104280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:40.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 shutdown_connections 2023-07-21T22:19:40.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:40.835+0000 7f1b9dae3700 1 -- 172.21.15.66:0/729294534 wait complete. 2023-07-21T22:19:41.343 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:19:41.343 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-21T22:19:41.343 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-21T22:19:41.454 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-21T22:19:41.455 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-21T22:19:41.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-21T22:19:41.556 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:41 smithi066 ceph-mon[126379]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:41.556 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:41 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/729294534' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:19:41.556 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:41 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:19:41.557 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:41 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:19:41.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:41 smithi104 ceph-mon[129021]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:41.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:41 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/729294534' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:19:41.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:41 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:19:41.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:41 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:19:41.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:41 smithi149 ceph-mon[129268]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 17 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:41.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:41 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/729294534' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:19:41.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:41 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T22:19:41.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:41 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T22:19:42.231 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:43.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:43 smithi104 ceph-mon[129021]: pgmap v294: 129 pgs: 129 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:43.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:43 smithi066 ceph-mon[126379]: pgmap v294: 129 pgs: 129 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:43.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:43 smithi149 ceph-mon[129268]: pgmap v294: 129 pgs: 129 active+clean; 619 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.9 KiB/s wr, 13 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:43.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 -- 172.21.15.66:0/3302642785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 msgr2=0x7feb801001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:43.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 --2- 172.21.15.66:0/3302642785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb801001d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feb700099c0 tx=0x7feb70009cd0 comp rx=0 tx=0).stop 2023-07-21T22:19:43.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 -- 172.21.15.66:0/3302642785 shutdown_connections 2023-07-21T22:19:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 --2- 172.21.15.66:0/3302642785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb80104ea0 0x7feb8010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 --2- 172.21.15.66:0/3302642785 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb801001d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 --2- 172.21.15.66:0/3302642785 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb801007a0 0x7feb80104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 -- 172.21.15.66:0/3302642785 >> 172.21.15.66:0/3302642785 conn(0x7feb800fb660 msgr2=0x7feb800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 -- 172.21.15.66:0/3302642785 shutdown_connections 2023-07-21T22:19:43.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.709+0000 7feb860c7700 1 -- 172.21.15.66:0/3302642785 wait complete. 2023-07-21T22:19:43.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.710+0000 7feb860c7700 1 Processor -- start 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.710+0000 7feb860c7700 1 -- start start 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb860c7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb80197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb860c7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb801007a0 0x7feb80198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb860c7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80104ea0 0x7feb8019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb860c7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8019cde0 con 0x7feb800ffe00 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb860c7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8019cf50 con 0x7feb801007a0 2023-07-21T22:19:43.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb860c7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8019d110 con 0x7feb80104ea0 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb80197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb80197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48816/0 (socket says 172.21.15.66:48816) 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7f7fe700 1 -- 172.21.15.66:0/1310360749 learned_addr learned my addr 172.21.15.66:0/1310360749 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7effd700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb801007a0 0x7feb80198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7ffff700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80104ea0 0x7feb8019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7f7fe700 1 -- 172.21.15.66:0/1310360749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80104ea0 msgr2=0x7feb8019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:43.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.711+0000 7feb7f7fe700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80104ea0 0x7feb8019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:43.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7f7fe700 1 -- 172.21.15.66:0/1310360749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb801007a0 msgr2=0x7feb80198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:43.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7f7fe700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb801007a0 0x7feb80198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:43.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7f7fe700 1 -- 172.21.15.66:0/1310360749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb700095f0 con 0x7feb800ffe00 2023-07-21T22:19:43.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7effd700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb801007a0 0x7feb80198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:19:43.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7f7fe700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb80197e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb7000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7001d070 con 0x7feb800ffe00 2023-07-21T22:19:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8019d390 con 0x7feb800ffe00 2023-07-21T22:19:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb70005510 con 0x7feb800ffe00 2023-07-21T22:19:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70004cb0 con 0x7feb800ffe00 2023-07-21T22:19:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.712+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb801a4c50 con 0x7feb800ffe00 2023-07-21T22:19:43.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.713+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8004e210 con 0x7feb800ffe00 2023-07-21T22:19:43.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.715+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7feb7000fc30 con 0x7feb800ffe00 2023-07-21T22:19:43.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.717+0000 7feb7cff9700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7feb680a11e0 0x7feb680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:43.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.717+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb700cdcb0 con 0x7feb800ffe00 2023-07-21T22:19:43.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.717+0000 7feb7effd700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7feb680a11e0 0x7feb680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:43.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.718+0000 7feb7effd700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7feb680a11e0 0x7feb680a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feb80199440 tx=0x7feb7400b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:43.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.721+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7feb70096fc0 con 0x7feb800ffe00 2023-07-21T22:19:44.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.985+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7feb801a4de0 con 0x7feb800ffe00 2023-07-21T22:19:44.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.991+0000 7feb7cff9700 1 -- 172.21.15.66:0/1310360749 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260126 (secure 0 0 0) 0x7feb7009a5e0 con 0x7feb800ffe00 2023-07-21T22:19:44.041 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:44.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7feb680a11e0 msgr2=0x7feb680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:44.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7feb680a11e0 0x7feb680a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feb80199440 tx=0x7feb7400b5f0 comp rx=0 tx=0).stop 2023-07-21T22:19:44.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 msgr2=0x7feb80197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:44.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb80197e20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb70000c00 tx=0x7feb7000f740 comp rx=0 tx=0).stop 2023-07-21T22:19:44.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 shutdown_connections 2023-07-21T22:19:44.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7feb680a11e0 0x7feb680a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:44.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feb80104ea0 0x7feb8019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feb800ffe00 0x7feb80197e20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 --2- 172.21.15.66:0/1310360749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb801007a0 0x7feb80198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 >> 172.21.15.66:0/1310360749 conn(0x7feb800fb660 msgr2=0x7feb8010bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.996+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 shutdown_connections 2023-07-21T22:19:44.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:43.997+0000 7feb860c7700 1 -- 172.21.15.66:0/1310360749 wait complete. 2023-07-21T22:19:44.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:44 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1310360749' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:19:44.532 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1262827329},{"type":"v1","addr":"172.21.15.66:6801","nonce":1262827329}]},"active_addr":"172.21.15.66:6801/1262827329","active_change":"2023-07-21T22:12:36.098717+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.66:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1497274919}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":4235245168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3171200415}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":4012457980}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":2222007456}]}]} 2023-07-21T22:19:44.539 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-21T22:19:44.539 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-21T22:19:44.539 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:19:44.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:44 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1310360749' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:19:44.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:44 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1310360749' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T22:19:45.281 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:45.580 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:45 smithi066 ceph-mon[126379]: pgmap v295: 129 pgs: 129 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:45.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:45 smithi104 ceph-mon[129021]: pgmap v295: 129 pgs: 129 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:45.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:45 smithi149 ceph-mon[129268]: pgmap v295: 129 pgs: 129 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:46.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.658+0000 7f8cd245a700 1 -- 172.21.15.66:0/2183279020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc0feab0 msgr2=0x7f8ccc0fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.658+0000 7f8cd245a700 1 --2- 172.21.15.66:0/2183279020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc0feab0 0x7f8ccc0fef20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009990 tx=0x7f8cc0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.658+0000 7f8cd245a700 1 -- 172.21.15.66:0/2183279020 shutdown_connections 2023-07-21T22:19:46.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.658+0000 7f8cd245a700 1 --2- 172.21.15.66:0/2183279020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.658+0000 7f8cd245a700 1 --2- 172.21.15.66:0/2183279020 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ccc104ab0 0x7f8ccc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.658+0000 7f8cd245a700 1 --2- 172.21.15.66:0/2183279020 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc0feab0 0x7f8ccc0fef20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.659+0000 7f8cd245a700 1 -- 172.21.15.66:0/2183279020 >> 172.21.15.66:0/2183279020 conn(0x7f8ccc075b70 msgr2=0x7f8ccc075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.659+0000 7f8cd245a700 1 -- 172.21.15.66:0/2183279020 shutdown_connections 2023-07-21T22:19:46.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.659+0000 7f8cd245a700 1 -- 172.21.15.66:0/2183279020 wait complete. 2023-07-21T22:19:46.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.660+0000 7f8cd245a700 1 Processor -- start 2023-07-21T22:19:46.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.660+0000 7f8cd245a700 1 -- start start 2023-07-21T22:19:46.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd245a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ccc0feab0 0x7f8ccc19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd245a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd245a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc104ab0 0x7f8ccc195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd245a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1960d0 con 0x7f8ccc104ab0 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd245a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc196240 con 0x7f8ccc0ff460 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd245a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc196400 con 0x7f8ccc0feab0 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8cd09f7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc104ab0 0x7f8ccc195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8ccb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57292/0 (socket says 172.21.15.66:57292) 2023-07-21T22:19:46.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.661+0000 7f8ccb7fe700 1 -- 172.21.15.66:0/3946099450 learned_addr learned my addr 172.21.15.66:0/3946099450 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:46.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccbfff700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ccc0feab0 0x7f8ccc19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccb7fe700 1 -- 172.21.15.66:0/3946099450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ccc0feab0 msgr2=0x7f8ccc19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccb7fe700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ccc0feab0 0x7f8ccc19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccb7fe700 1 -- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc104ab0 msgr2=0x7f8ccc195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccb7fe700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc104ab0 0x7f8ccc195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccb7fe700 1 -- 172.21.15.66:0/3946099450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00095f0 con 0x7f8ccc0ff460 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8ccb7fe700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc19bf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005910 tx=0x7f8cc0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc001d070 con 0x7f8ccc0ff460 2023-07-21T22:19:46.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.662+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cc0022470 con 0x7f8ccc0ff460 2023-07-21T22:19:46.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.663+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc196680 con 0x7f8ccc0ff460 2023-07-21T22:19:46.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.663+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0017650 con 0x7f8ccc0ff460 2023-07-21T22:19:46.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.663+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc1a29d0 con 0x7f8ccc0ff460 2023-07-21T22:19:46.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.664+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc04e210 con 0x7f8ccc0ff460 2023-07-21T22:19:46.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.666+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8cc00225e0 con 0x7f8ccc0ff460 2023-07-21T22:19:46.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.667+0000 7f8cc97fa700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cb40a11e0 0x7f8cb40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:46.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.668+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8cc00ce270 con 0x7f8ccc0ff460 2023-07-21T22:19:46.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.668+0000 7f8ccbfff700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cb40a11e0 0x7f8cb40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:46.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.668+0000 7f8ccbfff700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cb40a11e0 0x7f8cb40a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0ffbf0 tx=0x7f8cbc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:46.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.669+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8cc0097580 con 0x7f8ccc0ff460 2023-07-21T22:19:46.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.877+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ccc1a2b60 con 0x7f8ccc0ff460 2023-07-21T22:19:46.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.880+0000 7f8cc97fa700 1 -- 172.21.15.66:0/3946099450 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24442 (secure 0 0 0) 0x7f8cc009aba0 con 0x7f8ccc0ff460 2023-07-21T22:19:46.925 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:46.925 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:37.301201+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cb40a11e0 msgr2=0x7f8cb40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cb40a11e0 0x7f8cb40a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0ffbf0 tx=0x7f8cbc009260 comp rx=0 tx=0).stop 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 msgr2=0x7f8ccc19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc19bf20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0005910 tx=0x7f8cc0004710 comp rx=0 tx=0).stop 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 shutdown_connections 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cb40a11e0 0x7f8cb40a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ccc104ab0 0x7f8ccc195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ccc0feab0 0x7f8ccc19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 --2- 172.21.15.66:0/3946099450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ccc0ff460 0x7f8ccc19bf20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 >> 172.21.15.66:0/3946099450 conn(0x7f8ccc075b70 msgr2=0x7f8ccc102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 shutdown_connections 2023-07-21T22:19:46.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:46.882+0000 7f8cd245a700 1 -- 172.21.15.66:0/3946099450 wait complete. 2023-07-21T22:19:47.383 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-21T22:19:47.384 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T22:19:47.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:47 smithi104 ceph-mon[129021]: pgmap v296: 129 pgs: 129 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:47.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:47 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3946099450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:47 smithi066 ceph-mon[126379]: pgmap v296: 129 pgs: 129 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:47.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:47 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3946099450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:47.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:47 smithi149 ceph-mon[129268]: pgmap v296: 129 pgs: 129 active+clean; 628 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.0 KiB/s wr, 10 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:47.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:47 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3946099450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:48.130 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:49.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:48 smithi104 ceph-mon[129021]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:49.132 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:48 smithi066 ceph-mon[126379]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:49.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:48 smithi149 ceph-mon[129268]: pgmap v297: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 KiB/s wr, 9 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 -- 172.21.15.66:0/3933046566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200731c0 msgr2=0x7f022010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 --2- 172.21.15.66:0/3933046566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200731c0 0x7f022010bea0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0214009990 tx=0x7f0214009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 -- 172.21.15.66:0/3933046566 shutdown_connections 2023-07-21T22:19:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 --2- 172.21.15.66:0/3933046566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 0x7f022010e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 --2- 172.21.15.66:0/3933046566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02200728b0 0x7f0220072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 --2- 172.21.15.66:0/3933046566 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200731c0 0x7f022010bea0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.726+0000 7f0224c3e700 1 -- 172.21.15.66:0/3933046566 >> 172.21.15.66:0/3933046566 conn(0x7f02200fb660 msgr2=0x7f02200fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.727+0000 7f0224c3e700 1 -- 172.21.15.66:0/3933046566 shutdown_connections 2023-07-21T22:19:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.727+0000 7f0224c3e700 1 -- 172.21.15.66:0/3933046566 wait complete. 2023-07-21T22:19:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 Processor -- start 2023-07-21T22:19:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 -- start start 2023-07-21T22:19:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 0x7f0220197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02200731c0 0x7f02201982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 0x7f022019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022019cdf0 con 0x7f02200728b0 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022019cf60 con 0x7f022010c3e0 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f0224c3e700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022019d120 con 0x7f02200731c0 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f021e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 0x7f0220197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f021e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 0x7f0220197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48844/0 (socket says 172.21.15.66:48844) 2023-07-21T22:19:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f021e59c700 1 -- 172.21.15.66:0/2575452173 learned_addr learned my addr 172.21.15.66:0/2575452173 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f021ed9d700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 0x7f022019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.728+0000 7f021dd9b700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02200731c0 0x7f02201982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021e59c700 1 -- 172.21.15.66:0/2575452173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02200731c0 msgr2=0x7f02201982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021e59c700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02200731c0 0x7f02201982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021e59c700 1 -- 172.21.15.66:0/2575452173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 msgr2=0x7f022019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021e59c700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 0x7f022019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021e59c700 1 -- 172.21.15.66:0/2575452173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140095f0 con 0x7f02200728b0 2023-07-21T22:19:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021ed9d700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 0x7f022019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f021e59c700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 0x7f0220197da0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f021000e9a0 tx=0x7f021000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.729+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210009790 con 0x7f02200728b0 2023-07-21T22:19:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.730+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0210010430 con 0x7f02200728b0 2023-07-21T22:19:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.730+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022019d400 con 0x7f02200728b0 2023-07-21T22:19:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.730+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210018680 con 0x7f02200728b0 2023-07-21T22:19:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.730+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02201a4bd0 con 0x7f02200728b0 2023-07-21T22:19:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.731+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220106d00 con 0x7f02200728b0 2023-07-21T22:19:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.733+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f02100105a0 con 0x7f02200728b0 2023-07-21T22:19:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.734+0000 7f020f7fe700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02080a11e0 0x7f02080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.735+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0210014070 con 0x7f02200728b0 2023-07-21T22:19:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.735+0000 7f021dd9b700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02080a11e0 0x7f02080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.735+0000 7f021dd9b700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02080a11e0 0x7f02080a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02201993c0 tx=0x7f0214005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:49.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.737+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0210096440 con 0x7f02200728b0 2023-07-21T22:19:49.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.945+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f022002c850 con 0x7f02200728b0 2023-07-21T22:19:49.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.947+0000 7f020f7fe700 1 -- 172.21.15.66:0/2575452173 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24442 (secure 0 0 0) 0x7f0210026090 con 0x7f02200728b0 2023-07-21T22:19:49.993 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:19:49.993 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:37.301201+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:19:49.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02080a11e0 msgr2=0x7f02080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:49.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02080a11e0 0x7f02080a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f02201993c0 tx=0x7f0214005fd0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 msgr2=0x7f0220197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:49.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 0x7f0220197da0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f021000e9a0 tx=0x7f021000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 shutdown_connections 2023-07-21T22:19:49.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02080a11e0 0x7f02080a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f022010c3e0 0x7f022019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f02200728b0 0x7f0220197da0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 --2- 172.21.15.66:0/2575452173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02200731c0 0x7f02201982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:49.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 >> 172.21.15.66:0/2575452173 conn(0x7f02200fb660 msgr2=0x7f0220103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:49.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 shutdown_connections 2023-07-21T22:19:49.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:49.950+0000 7f0224c3e700 1 -- 172.21.15.66:0/2575452173 wait complete. 2023-07-21T22:19:50.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:50 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2575452173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:50.428 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:50 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2575452173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:50.432 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-21T22:19:50.432 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-21T22:19:50.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-21T22:19:50.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-21T22:19:50.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-21T22:19:50.434 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-21T22:19:50.435 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-21T22:19:50.435 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-21T22:19:50.436 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-21T22:19:50.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-21T22:19:50.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-21T22:19:50.438 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-21T22:19:50.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:50 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2575452173' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:19:51.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:51 smithi104 ceph-mon[129021]: pgmap v298: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:51.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:51 smithi066 ceph-mon[126379]: pgmap v298: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:51.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:51 smithi149 ceph-mon[129268]: pgmap v298: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2023-07-21T22:19:51.869 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.333 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.334 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.334 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.861 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.866 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.894 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:52.939 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:53.040 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:53.072 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:53.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:53 smithi104 ceph-mon[129021]: pgmap v299: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-07-21T22:19:53.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:53 smithi066 ceph-mon[126379]: pgmap v299: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-07-21T22:19:53.735 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:53.736 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:19:53.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:53 smithi149 ceph-mon[129268]: pgmap v299: 129 pgs: 129 active+clean; 636 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.1 KiB/s wr, 6 op/s 2023-07-21T22:19:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:55 smithi104 ceph-mon[129021]: pgmap v300: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-07-21T22:19:55.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:55 smithi066 ceph-mon[126379]: pgmap v300: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-07-21T22:19:55.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:55 smithi149 ceph-mon[129268]: pgmap v300: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2023-07-21T22:19:56.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 -- 172.21.15.66:0/1389319746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 msgr2=0x7fc780104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:56.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 --2- 172.21.15.66:0/1389319746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 0x7fc780104770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7740099c0 tx=0x7fc774009cd0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 -- 172.21.15.66:0/1389319746 shutdown_connections 2023-07-21T22:19:56.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 --2- 172.21.15.66:0/1389319746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7800ffe00 0x7fc7801001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 --2- 172.21.15.66:0/1389319746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc780104ea0 0x7fc78010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 --2- 172.21.15.66:0/1389319746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 0x7fc780104770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 -- 172.21.15.66:0/1389319746 >> 172.21.15.66:0/1389319746 conn(0x7fc7800fb660 msgr2=0x7fc7800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:56.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 -- 172.21.15.66:0/1389319746 shutdown_connections 2023-07-21T22:19:56.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.389+0000 7fc785015700 1 -- 172.21.15.66:0/1389319746 wait complete. 2023-07-21T22:19:56.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.390+0000 7fc785015700 1 Processor -- start 2023-07-21T22:19:56.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.390+0000 7fc785015700 1 -- start start 2023-07-21T22:19:56.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc785015700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7800ffe00 0x7fc78019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:56.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc785015700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 0x7fc78019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:56.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc785015700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc780104ea0 0x7fc780195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc785015700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801961b0 con 0x7fc780104ea0 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc785015700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc780196320 con 0x7fc7800ffe00 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc785015700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7801964e0 con 0x7fc7801007a0 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc77f59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc780104ea0 0x7fc780195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc77ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7800ffe00 0x7fc78019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc77ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7800ffe00 0x7fc78019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:59566/0 (socket says 172.21.15.66:59566) 2023-07-21T22:19:56.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc77ed9d700 1 -- 172.21.15.66:0/1567184646 learned_addr learned my addr 172.21.15.66:0/1567184646 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.391+0000 7fc77e59c700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 0x7fc78019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77f59e700 1 -- 172.21.15.66:0/1567184646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 msgr2=0x7fc78019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77f59e700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 0x7fc78019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77f59e700 1 -- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7800ffe00 msgr2=0x7fc78019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77f59e700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7800ffe00 0x7fc78019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77f59e700 1 -- 172.21.15.66:0/1567184646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7740095f0 con 0x7fc780104ea0 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77e59c700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7801007a0 0x7fc78019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc77f59e700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc780104ea0 0x7fc780195b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b330 tx=0x7fc77800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.392+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778015070 con 0x7fc780104ea0 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.393+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc780196760 con 0x7fc780104ea0 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.393+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc77800bdd0 con 0x7fc780104ea0 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.393+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778007c20 con 0x7fc780104ea0 2023-07-21T22:19:56.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.393+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780072910 con 0x7fc780104ea0 2023-07-21T22:19:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.394+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc78010fd20 con 0x7fc780104ea0 2023-07-21T22:19:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.396+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc778007500 con 0x7fc780104ea0 2023-07-21T22:19:56.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.398+0000 7fc767fff700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc7680a1260 0x7fc7680a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:56.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.398+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7780d0f30 con 0x7fc780104ea0 2023-07-21T22:19:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.398+0000 7fc77ed9d700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc7680a1260 0x7fc7680a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.398+0000 7fc767fff700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] conn(0x7fc7680ac270 0x7fc7680ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:56.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.398+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 --> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7680aedd0 con 0x7fc7680ac270 2023-07-21T22:19:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.398+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7780d1340 con 0x7fc780104ea0 2023-07-21T22:19:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.399+0000 7fc77ed9d700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc7680a1260 0x7fc7680a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc770009d20 tx=0x7fc770009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.399+0000 7fc77e59c700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] conn(0x7fc7680ac270 0x7fc7680ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.400+0000 7fc77e59c700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] conn(0x7fc7680ac270 0x7fc7680ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:56.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.400+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== osd.1 v2:172.21.15.104:6800/1799730222 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc7680aedd0 con 0x7fc7680ac270 2023-07-21T22:19:56.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.417+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 --> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc78004e210 con 0x7fc7680ac270 2023-07-21T22:19:56.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.418+0000 7fc767fff700 1 -- 172.21.15.66:0/1567184646 <== osd.1 v2:172.21.15.104:6800/1799730222 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc78004e210 con 0x7fc7680ac270 2023-07-21T22:19:56.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.418+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] conn(0x7fc7680ac270 msgr2=0x7fc7680ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:56.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.418+0000 7fc785015700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] conn(0x7fc7680ac270 0x7fc7680ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:56.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc7680a1260 msgr2=0x7fc7680a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:56.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc7680a1260 0x7fc7680a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc770009d20 tx=0x7fc770009260 comp rx=0 tx=0).stop 2023-07-21T22:19:56.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc780104ea0 msgr2=0x7fc780195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:56.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 --2- 172.21.15.66:0/1567184646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc780104ea0 0x7fc780195b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc77800b330 tx=0x7fc77800b640 comp rx=0 tx=0).stop 2023-07-21T22:19:56.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc77f59e700 1 -- 172.21.15.66:0/1567184646 reap_dead start 2023-07-21T22:19:56.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 shutdown_connections 2023-07-21T22:19:56.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 >> 172.21.15.66:0/1567184646 conn(0x7fc7800fb660 msgr2=0x7fc7800fc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:56.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 shutdown_connections 2023-07-21T22:19:56.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:56.419+0000 7fc785015700 1 -- 172.21.15.66:0/1567184646 wait complete. 2023-07-21T22:19:57.353 INFO:teuthology.orchestra.run.smithi066.stdout:55834574915 2023-07-21T22:19:57.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-21T22:19:57.512 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:57 smithi066 ceph-mon[126379]: pgmap v301: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-21T22:19:57.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:57 smithi104 ceph-mon[129021]: pgmap v301: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-21T22:19:57.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:57 smithi149 ceph-mon[129268]: pgmap v301: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2023-07-21T22:19:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 -- 172.21.15.66:0/1289177482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 msgr2=0x7f6274109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 --2- 172.21.15.66:0/1289177482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 0x7f6274109c00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6264009990 tx=0x7f6264009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 -- 172.21.15.66:0/1289177482 shutdown_connections 2023-07-21T22:19:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 --2- 172.21.15.66:0/1289177482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f627410a140 0x7f627410c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 --2- 172.21.15.66:0/1289177482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 0x7f62741011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 --2- 172.21.15.66:0/1289177482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 0x7f6274109c00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 -- 172.21.15.66:0/1289177482 >> 172.21.15.66:0/1289177482 conn(0x7f6274075b70 msgr2=0x7f6274075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 -- 172.21.15.66:0/1289177482 shutdown_connections 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.280+0000 7f627bc35700 1 -- 172.21.15.66:0/1289177482 wait complete. 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 Processor -- start 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 -- start start 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 0x7f6274197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 0x7f6274198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f627410a140 0x7f627419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627419ce00 con 0x7f6274100df0 2023-07-21T22:19:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627419cf70 con 0x7f627410a140 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.281+0000 7f627bc35700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627419d130 con 0x7f6274101700 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 0x7f6274197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62791d0700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 0x7f6274198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 0x7f6274197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35918/0 (socket says 172.21.15.66:35918) 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 -- 172.21.15.66:0/2192183100 learned_addr learned my addr 172.21.15.66:0/2192183100 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f627a1d2700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f627410a140 0x7f627419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 -- 172.21.15.66:0/2192183100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 msgr2=0x7f6274198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 0x7f6274198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 -- 172.21.15.66:0/2192183100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f627410a140 msgr2=0x7f627419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f627410a140 0x7f627419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 -- 172.21.15.66:0/2192183100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640095f0 con 0x7f6274100df0 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62791d0700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6274101700 0x7f6274198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f627a1d2700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f627410a140 0x7f627419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.282+0000 7f62799d1700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 0x7f6274197e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9a0 tx=0x7f627000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.283+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270009790 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.283+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6270010430 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.283+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627419d410 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.283+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270018680 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.283+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62741a4c70 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.284+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f627410fd40 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.286+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f62700105a0 con 0x7f6274100df0 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.287+0000 7f626affd700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f62600a11e0 0x7f62600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6270014070 con 0x7f6274100df0 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f62791d0700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f62600a11e0 0x7f62600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f626affd700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] conn(0x7f62600ac1f0 0x7f62600ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 --> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62600aed50 con 0x7f62600ac1f0 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62700cd610 con 0x7f6274100df0 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f627a1d2700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] conn(0x7f62600ac1f0 0x7f62600ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.288+0000 7f62791d0700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f62600a11e0 0x7f62600a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f626400b2e0 tx=0x7f6264005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.289+0000 7f627a1d2700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] conn(0x7f62600ac1f0 0x7f62600ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.289+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== osd.6 v2:172.21.15.66:6818/2608304075 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f62600aed50 con 0x7f62600ac1f0 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.306+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 --> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f627404e210 con 0x7f62600ac1f0 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f626affd700 1 -- 172.21.15.66:0/2192183100 <== osd.6 v2:172.21.15.66:6818/2608304075 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f627404e210 con 0x7f62600ac1f0 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] conn(0x7f62600ac1f0 msgr2=0x7f62600ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] conn(0x7f62600ac1f0 0x7f62600ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f62600a11e0 msgr2=0x7f62600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f62600a11e0 0x7f62600a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f626400b2e0 tx=0x7f6264005960 comp rx=0 tx=0).stop 2023-07-21T22:19:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 msgr2=0x7f6274197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 --2- 172.21.15.66:0/2192183100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6274100df0 0x7f6274197e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9a0 tx=0x7f627000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627a1d2700 1 -- 172.21.15.66:0/2192183100 reap_dead start 2023-07-21T22:19:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 shutdown_connections 2023-07-21T22:19:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 >> 172.21.15.66:0/2192183100 conn(0x7f6274075b70 msgr2=0x7f62740fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 shutdown_connections 2023-07-21T22:19:58.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:58.307+0000 7f627bc35700 1 -- 172.21.15.66:0/2192183100 wait complete. 2023-07-21T22:19:58.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:19:58 smithi066 ceph-mon[126379]: pgmap v302: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-21T22:19:59.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:19:58 smithi104 ceph-mon[129021]: pgmap v302: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-21T22:19:59.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:19:58 smithi149 ceph-mon[129268]: pgmap v302: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-21T22:19:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.971+0000 7f8cc7c73700 1 -- 172.21.15.66:0/4261126907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 msgr2=0x7f8cc0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.971+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/4261126907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc0104770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00099c0 tx=0x7f8cb0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.971+0000 7f8cc7c73700 1 -- 172.21.15.66:0/4261126907 shutdown_connections 2023-07-21T22:19:59.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.971+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/4261126907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cc0104ea0 0x7f8cc010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.971+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/4261126907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc00ffe00 0x7f8cc01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.972+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/4261126907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc0104770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.972+0000 7f8cc7c73700 1 -- 172.21.15.66:0/4261126907 >> 172.21.15.66:0/4261126907 conn(0x7f8cc00fb660 msgr2=0x7f8cc00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.972+0000 7f8cc7c73700 1 -- 172.21.15.66:0/4261126907 shutdown_connections 2023-07-21T22:19:59.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.972+0000 7f8cc7c73700 1 -- 172.21.15.66:0/4261126907 wait complete. 2023-07-21T22:19:59.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 Processor -- start 2023-07-21T22:19:59.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 -- start start 2023-07-21T22:19:59.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cc00ffe00 0x7f8cc019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc0104ea0 0x7f8cc0195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01961b0 con 0x7f8cc01007a0 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc0196320 con 0x7f8cc00ffe00 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.973+0000 7f8cc7c73700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc01964e0 con 0x7f8cc0104ea0 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc520e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cc00ffe00 0x7f8cc019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc520e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc019bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35914/0 (socket says 172.21.15.66:35914) 2023-07-21T22:19:59.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc520e700 1 -- 172.21.15.66:0/2552119242 learned_addr learned my addr 172.21.15.66:0/2552119242 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc6210700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc0104ea0 0x7f8cc0195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 -- 172.21.15.66:0/2552119242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc0104ea0 msgr2=0x7f8cc0195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cc0104ea0 0x7f8cc0195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 -- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 msgr2=0x7f8cc019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 -- 172.21.15.66:0/2552119242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00095f0 con 0x7f8cc00ffe00 2023-07-21T22:19:59.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc520e700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cc01007a0 0x7f8cc019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.974+0000 7f8cc5a0f700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cc00ffe00 0x7f8cc019ba20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00e9f0 tx=0x7f8cbc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.975+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc00c9c0 con 0x7f8cc00ffe00 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.975+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc01967c0 con 0x7f8cc00ffe00 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.975+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cbc010430 con 0x7f8cc00ffe00 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.975+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc018680 con 0x7f8cc00ffe00 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.975+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc01a2a10 con 0x7f8cc00ffe00 2023-07-21T22:19:59.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.976+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cc004e210 con 0x7f8cc00ffe00 2023-07-21T22:19:59.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.978+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8cbc010860 con 0x7f8cc00ffe00 2023-07-21T22:19:59.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.979+0000 7f8cb6ffd700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cac0a1260 0x7f8cac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8cbc014070 con 0x7f8cc00ffe00 2023-07-21T22:19:59.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cc520e700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cac0a1260 0x7f8cac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cb6ffd700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] conn(0x7f8cac0ac270 0x7f8cac0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 --> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8cac0aedd0 con 0x7f8cac0ac270 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cbc0cd4f0 con 0x7f8cc00ffe00 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cc6210700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] conn(0x7f8cac0ac270 0x7f8cac0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.980+0000 7f8cc520e700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cac0a1260 0x7f8cac0a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0106250 tx=0x7f8cb0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.981+0000 7f8cc6210700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] conn(0x7f8cac0ac270 0x7f8cac0ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.981+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== osd.3 v2:172.21.15.66:6810/4147082736 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8cac0aedd0 con 0x7f8cac0ac270 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.998+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 --> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cc010fd60 con 0x7f8cac0ac270 2023-07-21T22:19:59.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.998+0000 7f8cb6ffd700 1 -- 172.21.15.66:0/2552119242 <== osd.3 v2:172.21.15.66:6810/4147082736 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8cb800f040 con 0x7f8cac0ac270 2023-07-21T22:19:59.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] conn(0x7f8cac0ac270 msgr2=0x7f8cac0ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] conn(0x7f8cac0ac270 0x7f8cac0ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cac0a1260 msgr2=0x7f8cac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8cac0a1260 0x7f8cac0a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0106250 tx=0x7f8cb0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cc00ffe00 msgr2=0x7f8cc019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 --2- 172.21.15.66:0/2552119242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cc00ffe00 0x7f8cc019ba20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00e9f0 tx=0x7f8cbc00ed00 comp rx=0 tx=0).stop 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc6210700 1 -- 172.21.15.66:0/2552119242 reap_dead start 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 shutdown_connections 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 >> 172.21.15.66:0/2552119242 conn(0x7f8cc00fb660 msgr2=0x7f8cc00fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 shutdown_connections 2023-07-21T22:19:59.268 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:57.999+0000 7f8cc7c73700 1 -- 172.21.15.66:0/2552119242 wait complete. 2023-07-21T22:19:59.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.417+0000 7f50975e7700 1 -- 172.21.15.66:0/1555807941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 msgr2=0x7f5090107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.417+0000 7f50975e7700 1 --2- 172.21.15.66:0/1555807941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 0x7f5090107f50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f50840098f0 tx=0x7f5084009c00 comp rx=0 tx=0).stop 2023-07-21T22:19:59.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 -- 172.21.15.66:0/1555807941 shutdown_connections 2023-07-21T22:19:59.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 --2- 172.21.15.66:0/1555807941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 0x7f5090107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 --2- 172.21.15.66:0/1555807941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5090112930 0x7f5090114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 --2- 172.21.15.66:0/1555807941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 0x7f5090107f50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 -- 172.21.15.66:0/1555807941 >> 172.21.15.66:0/1555807941 conn(0x7f509006be40 msgr2=0x7f5090071020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 -- 172.21.15.66:0/1555807941 shutdown_connections 2023-07-21T22:19:59.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.418+0000 7f50975e7700 1 -- 172.21.15.66:0/1555807941 wait complete. 2023-07-21T22:19:59.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.419+0000 7f50975e7700 1 Processor -- start 2023-07-21T22:19:59.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.419+0000 7f50975e7700 1 -- start start 2023-07-21T22:19:59.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.420+0000 7f50975e7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 0x7f50901a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.420+0000 7f50975e7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 0x7f50901a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.420+0000 7f50975e7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5090112930 0x7f50901a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.420+0000 7f50975e7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50901a45c0 con 0x7f5090107ae0 2023-07-21T22:19:59.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.420+0000 7f50975e7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50901a4730 con 0x7f509006da70 2023-07-21T22:19:59.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.420+0000 7f50975e7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50901a48f0 con 0x7f5090112930 2023-07-21T22:19:59.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 0x7f50901a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 0x7f50901a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:59620/0 (socket says 172.21.15.66:59620) 2023-07-21T22:19:59.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 -- 172.21.15.66:0/1739559591 learned_addr learned my addr 172.21.15.66:0/1739559591 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:59.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095b84700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5090112930 0x7f50901a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5094b82700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 0x7f50901a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 -- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5090112930 msgr2=0x7f50901a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5090112930 0x7f50901a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 -- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 msgr2=0x7f50901a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5090107ae0 0x7f50901a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 -- 172.21.15.66:0/1739559591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880095f0 con 0x7f509006da70 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.421+0000 7f5095383700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 0x7f50901a06c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f508800ec10 tx=0x7f508800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.422+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800cd40 con 0x7f509006da70 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.422+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5084009520 con 0x7f509006da70 2023-07-21T22:19:59.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.422+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50880044e0 con 0x7f509006da70 2023-07-21T22:19:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.422+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50901ad350 con 0x7f509006da70 2023-07-21T22:19:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.422+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088010690 con 0x7f509006da70 2023-07-21T22:19:59.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.423+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f507c000ff0 con 0x7f509006da70 2023-07-21T22:19:59.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.426+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5088004020 con 0x7f509006da70 2023-07-21T22:19:59.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.427+0000 7f508e7fc700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50780a1260 0x7f50780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.427+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5088014070 con 0x7f509006da70 2023-07-21T22:19:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.428+0000 7f508e7fc700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] conn(0x7f50780ac270 0x7f50780ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.428+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 --> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50780aedd0 con 0x7f50780ac270 2023-07-21T22:19:59.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.428+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50880cf670 con 0x7f509006da70 2023-07-21T22:19:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.428+0000 7f5095b84700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] conn(0x7f50780ac270 0x7f50780ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.428+0000 7f5094b82700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50780a1260 0x7f50780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.429+0000 7f5095b84700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] conn(0x7f50780ac270 0x7f50780ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.429+0000 7f5094b82700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50780a1260 0x7f50780a3710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50901a1de0 tx=0x7f5084019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.430+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== osd.5 v2:172.21.15.149:6808/2736316265 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f50780aedd0 con 0x7f50780ac270 2023-07-21T22:19:59.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.448+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 --> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f507c002d70 con 0x7f50780ac270 2023-07-21T22:19:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.448+0000 7f508e7fc700 1 -- 172.21.15.66:0/1739559591 <== osd.5 v2:172.21.15.149:6808/2736316265 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f507c002d70 con 0x7f50780ac270 2023-07-21T22:19:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] conn(0x7f50780ac270 msgr2=0x7f50780ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] conn(0x7f50780ac270 0x7f50780ae700 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50780a1260 msgr2=0x7f50780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f50780a1260 0x7f50780a3710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f50901a1de0 tx=0x7f5084019040 comp rx=0 tx=0).stop 2023-07-21T22:19:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 msgr2=0x7f50901a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 --2- 172.21.15.66:0/1739559591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f509006da70 0x7f50901a06c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f508800ec10 tx=0x7f508800ef20 comp rx=0 tx=0).stop 2023-07-21T22:19:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f5095b84700 1 -- 172.21.15.66:0/1739559591 reap_dead start 2023-07-21T22:19:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 shutdown_connections 2023-07-21T22:19:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 >> 172.21.15.66:0/1739559591 conn(0x7f509006be40 msgr2=0x7f509010a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 shutdown_connections 2023-07-21T22:19:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.449+0000 7f50975e7700 1 -- 172.21.15.66:0/1739559591 wait complete. 2023-07-21T22:19:59.499 INFO:teuthology.orchestra.run.smithi066.stdout:201863462957 2023-07-21T22:19:59.499 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-21T22:19:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.582+0000 7f1cdd304700 1 -- 172.21.15.66:0/3796353007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd80728b0 msgr2=0x7f1cd8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.582+0000 7f1cdd304700 1 --2- 172.21.15.66:0/3796353007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd80728b0 0x7f1cd8072c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80098f0 tx=0x7f1cc8009c00 comp rx=0 tx=0).stop 2023-07-21T22:19:59.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 -- 172.21.15.66:0/3796353007 shutdown_connections 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 --2- 172.21.15.66:0/3796353007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd80728b0 0x7f1cd8072c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 --2- 172.21.15.66:0/3796353007 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd810c3e0 0x7f1cd810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 --2- 172.21.15.66:0/3796353007 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 0x7f1cd810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 -- 172.21.15.66:0/3796353007 >> 172.21.15.66:0/3796353007 conn(0x7f1cd80fb660 msgr2=0x7f1cd80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 -- 172.21.15.66:0/3796353007 shutdown_connections 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.583+0000 7f1cdd304700 1 -- 172.21.15.66:0/3796353007 wait complete. 2023-07-21T22:19:59.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.584+0000 7f1cdd304700 1 Processor -- start 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.584+0000 7f1cdd304700 1 -- start start 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cdd304700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd80728b0 0x7f1cd8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cdd304700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 0x7f1cd81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cdd304700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd810c3e0 0x7f1cd819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cdd304700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd819ceb0 con 0x7f1cd80728b0 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cdd304700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd819d020 con 0x7f1cd810c3e0 2023-07-21T22:19:59.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cdd304700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd819d1e0 con 0x7f1cd80731c0 2023-07-21T22:19:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 0x7f1cd81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 0x7f1cd81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47856/0 (socket says 172.21.15.66:47856) 2023-07-21T22:19:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 -- 172.21.15.66:0/4194152316 learned_addr learned my addr 172.21.15.66:0/4194152316 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd80728b0 0x7f1cd8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd77fe700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd810c3e0 0x7f1cd819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 -- 172.21.15.66:0/4194152316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd810c3e0 msgr2=0x7f1cd819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd810c3e0 0x7f1cd819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 -- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd80728b0 msgr2=0x7f1cd8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.585+0000 7f1cd67fc700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1cd80728b0 0x7f1cd8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cd67fc700 1 -- 172.21.15.66:0/4194152316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009d00 con 0x7f1cd80731c0 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cd77fe700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1cd810c3e0 0x7f1cd819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:19:59.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cd67fc700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 0x7f1cd81983a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00e9a0 tx=0x7f1ccc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc009790 con 0x7f1cd80731c0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ccc00cbd0 con 0x7f1cd80731c0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc8009520 con 0x7f1cd80731c0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc018680 con 0x7f1cd80731c0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.586+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd819d660 con 0x7f1cd80731c0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.587+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1cd804e210 con 0x7f1cd80731c0 2023-07-21T22:19:59.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.590+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f1ccc00cd40 con 0x7f1cd80731c0 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.591+0000 7f1cbffff700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1cc00a11e0 0x7f1cc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.591+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1ccc014070 con 0x7f1cd80731c0 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.591+0000 7f1cbffff700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] conn(0x7f1cc00ac1f0 0x7f1cc00ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.591+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 --> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cc00aed50 con 0x7f1cc00ac1f0 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.591+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ccc0cd7b0 con 0x7f1cd80731c0 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.592+0000 7f1cd6ffd700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1cc00a11e0 0x7f1cc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.592+0000 7f1cd77fe700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] conn(0x7f1cc00ac1f0 0x7f1cc00ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.593+0000 7f1cd6ffd700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1cc00a11e0 0x7f1cc00a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8005e90 tx=0x7f1cc8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.593+0000 7f1cd77fe700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] conn(0x7f1cc00ac1f0 0x7f1cc00ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.594+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== osd.0 v2:172.21.15.66:6802/1273732852 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1cc00aed50 con 0x7f1cc00ac1f0 2023-07-21T22:19:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.616+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 --> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1cd8111f70 con 0x7f1cc00ac1f0 2023-07-21T22:19:59.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cbffff700 1 -- 172.21.15.66:0/4194152316 <== osd.0 v2:172.21.15.66:6802/1273732852 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1cd0011040 con 0x7f1cc00ac1f0 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] conn(0x7f1cc00ac1f0 msgr2=0x7f1cc00ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] conn(0x7f1cc00ac1f0 0x7f1cc00ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1cc00a11e0 msgr2=0x7f1cc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f1cc00a11e0 0x7f1cc00a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8005e90 tx=0x7f1cc8019040 comp rx=0 tx=0).stop 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 msgr2=0x7f1cd81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 --2- 172.21.15.66:0/4194152316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1cd80731c0 0x7f1cd81983a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00e9a0 tx=0x7f1ccc00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cd77fe700 1 -- 172.21.15.66:0/4194152316 reap_dead start 2023-07-21T22:19:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 shutdown_connections 2023-07-21T22:19:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 >> 172.21.15.66:0/4194152316 conn(0x7f1cd80fb660 msgr2=0x7f1cd810e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 shutdown_connections 2023-07-21T22:19:59.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.617+0000 7f1cdd304700 1 -- 172.21.15.66:0/4194152316 wait complete. 2023-07-21T22:19:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 -- 172.21.15.66:0/3711034130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900731c0 msgr2=0x7f489010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 --2- 172.21.15.66:0/3711034130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900731c0 0x7f489010bea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4884009990 tx=0x7f4884009ca0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 -- 172.21.15.66:0/3711034130 shutdown_connections 2023-07-21T22:19:59.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 --2- 172.21.15.66:0/3711034130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f489010c3e0 0x7f489010e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 --2- 172.21.15.66:0/3711034130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f48900728b0 0x7f4890072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 --2- 172.21.15.66:0/3711034130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900731c0 0x7f489010bea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 -- 172.21.15.66:0/3711034130 >> 172.21.15.66:0/3711034130 conn(0x7f48900fb660 msgr2=0x7f48900fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 -- 172.21.15.66:0/3711034130 shutdown_connections 2023-07-21T22:19:59.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.504+0000 7f4895d31700 1 -- 172.21.15.66:0/3711034130 wait complete. 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.506+0000 7f4895d31700 1 Processor -- start 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.506+0000 7f4895d31700 1 -- start start 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f4895d31700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900728b0 0x7f4890197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f4895d31700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48900731c0 0x7f4890198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f4895d31700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f489010c3e0 0x7f489019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f4895d31700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019ce10 con 0x7f48900728b0 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f4895d31700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019cf80 con 0x7f48900731c0 2023-07-21T22:19:59.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f4895d31700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019d140 con 0x7f489010c3e0 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900728b0 0x7f4890197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f488f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900728b0 0x7f4890197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35972/0 (socket says 172.21.15.66:35972) 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f488effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48900731c0 0x7f4890198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f488f7fe700 1 -- 172.21.15.66:0/161914053 learned_addr learned my addr 172.21.15.66:0/161914053 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.507+0000 7f488ffff700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f489010c3e0 0x7f489019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488effd700 1 -- 172.21.15.66:0/161914053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f489010c3e0 msgr2=0x7f489019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488effd700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f489010c3e0 0x7f489019c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488effd700 1 -- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900728b0 msgr2=0x7f4890197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488effd700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f48900728b0 0x7f4890197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488effd700 1 -- 172.21.15.66:0/161914053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880009520 con 0x7f48900731c0 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488effd700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48900731c0 0x7f4890198300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4884009990 tx=0x7f4884009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488401d070 con 0x7f48900731c0 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4884021410 con 0x7f48900731c0 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4888009c40 con 0x7f48900731c0 2023-07-21T22:19:59.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884017650 con 0x7f48900731c0 2023-07-21T22:19:59.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.508+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48840095f0 con 0x7f48900731c0 2023-07-21T22:19:59.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.510+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f489004e210 con 0x7f48900731c0 2023-07-21T22:19:59.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.511+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f48840217a0 con 0x7f48900731c0 2023-07-21T22:19:59.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.513+0000 7f488cff9700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f48780a11e0 0x7f48780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.513+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4884021e40 con 0x7f48900731c0 2023-07-21T22:19:59.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.513+0000 7f488f7fe700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f48780a11e0 0x7f48780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.513+0000 7f488cff9700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] conn(0x7f48780ac1f0 0x7f48780ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:19:59.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.513+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 --> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48780aed50 con 0x7f48780ac1f0 2023-07-21T22:19:59.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.513+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48840d03c0 con 0x7f48900731c0 2023-07-21T22:19:59.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.514+0000 7f488ffff700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] conn(0x7f48780ac1f0 0x7f48780ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:19:59.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.514+0000 7f488f7fe700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f48780a11e0 0x7f48780a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4880009ef0 tx=0x7f4880009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.514+0000 7f488ffff700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] conn(0x7f48780ac1f0 0x7f48780ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:19:59.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.515+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== osd.4 v2:172.21.15.104:6808/3974235171 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f48780aed50 con 0x7f48780ac1f0 2023-07-21T22:19:59.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.534+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 --> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4890111f80 con 0x7f48780ac1f0 2023-07-21T22:19:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f488cff9700 1 -- 172.21.15.66:0/161914053 <== osd.4 v2:172.21.15.104:6808/3974235171 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f488800f040 con 0x7f48780ac1f0 2023-07-21T22:19:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] conn(0x7f48780ac1f0 msgr2=0x7f48780ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f4895d31700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] conn(0x7f48780ac1f0 0x7f48780ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:19:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f48780a11e0 msgr2=0x7f48780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f4895d31700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f48780a11e0 0x7f48780a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4880009ef0 tx=0x7f4880009260 comp rx=0 tx=0).stop 2023-07-21T22:19:59.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48900731c0 msgr2=0x7f4890198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:19:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.535+0000 7f4895d31700 1 --2- 172.21.15.66:0/161914053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48900731c0 0x7f4890198300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4884009990 tx=0x7f4884009580 comp rx=0 tx=0).stop 2023-07-21T22:19:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.536+0000 7f488ffff700 1 -- 172.21.15.66:0/161914053 reap_dead start 2023-07-21T22:19:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.536+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 shutdown_connections 2023-07-21T22:19:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.536+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 >> 172.21.15.66:0/161914053 conn(0x7f48900fb660 msgr2=0x7f4890103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:19:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.536+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 shutdown_connections 2023-07-21T22:19:59.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:19:59.536+0000 7f4895d31700 1 -- 172.21.15.66:0/161914053 wait complete. 2023-07-21T22:20:00.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:00 smithi104 ceph-mon[129021]: overall HEALTH_OK 2023-07-21T22:20:00.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:00 smithi066 ceph-mon[126379]: overall HEALTH_OK 2023-07-21T22:20:00.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:00 smithi149 ceph-mon[129268]: overall HEALTH_OK 2023-07-21T22:20:01.086 INFO:teuthology.orchestra.run.smithi066.stdout:34359738440 2023-07-21T22:20:01.086 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-21T22:20:01.142 INFO:teuthology.orchestra.run.smithi066.stdout:167503724595 2023-07-21T22:20:01.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-21T22:20:01.216 INFO:teuthology.orchestra.run.smithi066.stdout:146028888118 2023-07-21T22:20:01.217 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-21T22:20:01.285 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:01 smithi066 ceph-mon[126379]: pgmap v303: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-21T22:20:01.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:01 smithi104 ceph-mon[129021]: pgmap v303: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-21T22:20:01.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:01 smithi149 ceph-mon[129268]: pgmap v303: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-21T22:20:01.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 -- 172.21.15.66:0/2438990418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 msgr2=0x7f4b80072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2438990418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b80072c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009990 tx=0x7f4b7c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 -- 172.21.15.66:0/2438990418 shutdown_connections 2023-07-21T22:20:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2438990418 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b80110bb0 0x7f4b80112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2438990418 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b80072c80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2438990418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b800731c0 0x7f4b80110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 -- 172.21.15.66:0/2438990418 >> 172.21.15.66:0/2438990418 conn(0x7f4b800fb660 msgr2=0x7f4b800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 -- 172.21.15.66:0/2438990418 shutdown_connections 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.507+0000 7f4b874fd700 1 -- 172.21.15.66:0/2438990418 wait complete. 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.508+0000 7f4b874fd700 1 Processor -- start 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 -- start start 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b8019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b800731c0 0x7f4b8019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b80110bb0 0x7f4b801a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:01.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801a1250 con 0x7f4b800728b0 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801a13c0 con 0x7f4b80110bb0 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.509+0000 7f4b874fd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801a1580 con 0x7f4b800731c0 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b8019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b8019c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36004/0 (socket says 172.21.15.66:36004) 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 -- 172.21.15.66:0/2817133968 learned_addr learned my addr 172.21.15.66:0/2817133968 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b84a98700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b800731c0 0x7f4b8019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85a9a700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b80110bb0 0x7f4b801a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:01.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 -- 172.21.15.66:0/2817133968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b800731c0 msgr2=0x7f4b8019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b800731c0 0x7f4b8019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 -- 172.21.15.66:0/2817133968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b80110bb0 msgr2=0x7f4b801a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b80110bb0 0x7f4b801a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 -- 172.21.15.66:0/2817133968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c0095f0 con 0x7f4b800728b0 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.510+0000 7f4b85299700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b8019c290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c000c00 tx=0x7f4b7c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.511+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c01d070 con 0x7f4b800728b0 2023-07-21T22:20:01.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.511+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b7c00be40 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.511+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801a1800 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.511+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0173f0 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.511+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b801a90c0 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.512+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b80116320 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.514+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4b7c022470 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.515+0000 7f4b767fc700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b6c0a11e0 0x7f4b6c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.515+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4b7c0ce870 con 0x7f4b800728b0 2023-07-21T22:20:01.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.515+0000 7f4b84a98700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b6c0a11e0 0x7f4b6c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.515+0000 7f4b767fc700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] conn(0x7f4b6c0ac1f0 0x7f4b6c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.516+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 --> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b6c0aed50 con 0x7f4b6c0ac1f0 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.516+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b7c0cebe0 con 0x7f4b800728b0 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.516+0000 7f4b85a9a700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] conn(0x7f4b6c0ac1f0 0x7f4b6c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.516+0000 7f4b84a98700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b6c0a11e0 0x7f4b6c0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b8019d8b0 tx=0x7f4b70009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.516+0000 7f4b85a9a700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] conn(0x7f4b6c0ac1f0 0x7f4b6c0ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:01.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.517+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== osd.9 v2:172.21.15.66:6826/825018601 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4b6c0aed50 con 0x7f4b6c0ac1f0 2023-07-21T22:20:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.534+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 --> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b801a1ad0 con 0x7f4b6c0ac1f0 2023-07-21T22:20:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.534+0000 7f4b767fc700 1 -- 172.21.15.66:0/2817133968 <== osd.9 v2:172.21.15.66:6826/825018601 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4b7800f040 con 0x7f4b6c0ac1f0 2023-07-21T22:20:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.534+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] conn(0x7f4b6c0ac1f0 msgr2=0x7f4b6c0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:01.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.534+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] conn(0x7f4b6c0ac1f0 0x7f4b6c0ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b6c0a11e0 msgr2=0x7f4b6c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b6c0a11e0 0x7f4b6c0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4b8019d8b0 tx=0x7f4b70009260 comp rx=0 tx=0).stop 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 msgr2=0x7f4b8019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 --2- 172.21.15.66:0/2817133968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b800728b0 0x7f4b8019c290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c000c00 tx=0x7f4b7c00f690 comp rx=0 tx=0).stop 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b85a9a700 1 -- 172.21.15.66:0/2817133968 reap_dead start 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 shutdown_connections 2023-07-21T22:20:01.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 >> 172.21.15.66:0/2817133968 conn(0x7f4b800fb660 msgr2=0x7f4b800fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:01.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 shutdown_connections 2023-07-21T22:20:01.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:01.535+0000 7f4b874fd700 1 -- 172.21.15.66:0/2817133968 wait complete. 2023-07-21T22:20:01.927 INFO:teuthology.orchestra.run.smithi066.stdout:111669149754 2023-07-21T22:20:01.928 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-21T22:20:02.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 -- 172.21.15.66:0/1459825899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80731c0 msgr2=0x7fa8e810bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 --2- 172.21.15.66:0/1459825899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80731c0 0x7fa8e810bea0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009990 tx=0x7fa8d8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 -- 172.21.15.66:0/1459825899 shutdown_connections 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 --2- 172.21.15.66:0/1459825899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 0x7fa8e810e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 --2- 172.21.15.66:0/1459825899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e80728b0 0x7fa8e8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 --2- 172.21.15.66:0/1459825899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80731c0 0x7fa8e810bea0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 -- 172.21.15.66:0/1459825899 >> 172.21.15.66:0/1459825899 conn(0x7fa8e80fb660 msgr2=0x7fa8e80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:02.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 -- 172.21.15.66:0/1459825899 shutdown_connections 2023-07-21T22:20:02.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.328+0000 7fa8ef342700 1 -- 172.21.15.66:0/1459825899 wait complete. 2023-07-21T22:20:02.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 Processor -- start 2023-07-21T22:20:02.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 -- start start 2023-07-21T22:20:02.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80728b0 0x7fa8e8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e80731c0 0x7fa8e81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 0x7fa8e819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e819cdf0 con 0x7fa8e80728b0 2023-07-21T22:20:02.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e819cf60 con 0x7fa8e810c3e0 2023-07-21T22:20:02.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.330+0000 7fa8ef342700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e819d120 con 0x7fa8e80731c0 2023-07-21T22:20:02.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ec8dd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e80731c0 0x7fa8e81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 0x7fa8e819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 0x7fa8e819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38250/0 (socket says 172.21.15.66:38250) 2023-07-21T22:20:02.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ec8dd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e80731c0 0x7fa8e81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51276/0 (socket says 172.21.15.66:51276) 2023-07-21T22:20:02.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 -- 172.21.15.66:0/4261729467 learned_addr learned my addr 172.21.15.66:0/4261729467 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:02.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed0de700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80728b0 0x7fa8e8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 -- 172.21.15.66:0/4261729467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e80731c0 msgr2=0x7fa8e81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8e80731c0 0x7fa8e81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 -- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80728b0 msgr2=0x7fa8e8197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8e80728b0 0x7fa8e8197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.331+0000 7fa8ed8df700 1 -- 172.21.15.66:0/4261729467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d80095f0 con 0x7fa8e810c3e0 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.332+0000 7fa8ed8df700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 0x7fa8e819c750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0007ee0 tx=0x7fa8e000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.332+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0017070 con 0x7fa8e810c3e0 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.332+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8e000f040 con 0x7fa8e810c3e0 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.332+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0005740 con 0x7fa8e810c3e0 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.332+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e819d400 con 0x7fa8e810c3e0 2023-07-21T22:20:02.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.332+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e81a4bd0 con 0x7fa8e810c3e0 2023-07-21T22:20:02.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.333+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8e8111f50 con 0x7fa8e810c3e0 2023-07-21T22:20:02.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.335+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa8e000dc60 con 0x7fa8e810c3e0 2023-07-21T22:20:02.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.337+0000 7fa8de7fc700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8d40a11e0 0x7fa8d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.337+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa8e00ccc50 con 0x7fa8e810c3e0 2023-07-21T22:20:02.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.337+0000 7fa8ed0de700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8d40a11e0 0x7fa8d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.337+0000 7fa8de7fc700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] conn(0x7fa8d40ac1f0 0x7fa8d40ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.337+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 --> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8d40aed50 con 0x7fa8d40ac1f0 2023-07-21T22:20:02.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.337+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8e00ccfc0 con 0x7fa8e810c3e0 2023-07-21T22:20:02.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.338+0000 7fa8ed0de700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8d40a11e0 0x7fa8d40a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005950 tx=0x7fa8e40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.338+0000 7fa8ec8dd700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] conn(0x7fa8d40ac1f0 0x7fa8d40ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.338+0000 7fa8ec8dd700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] conn(0x7fa8d40ac1f0 0x7fa8d40ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.339+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== osd.7 v2:172.21.15.104:6816/1910018308 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa8d40aed50 con 0x7fa8d40ac1f0 2023-07-21T22:20:02.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.356+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 --> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8e804e210 con 0x7fa8d40ac1f0 2023-07-21T22:20:02.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.357+0000 7fa8de7fc700 1 -- 172.21.15.66:0/4261729467 <== osd.7 v2:172.21.15.104:6816/1910018308 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fa8e804e210 con 0x7fa8d40ac1f0 2023-07-21T22:20:02.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] conn(0x7fa8d40ac1f0 msgr2=0x7fa8d40ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] conn(0x7fa8d40ac1f0 0x7fa8d40ae680 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8d40a11e0 msgr2=0x7fa8d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fa8d40a11e0 0x7fa8d40a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4005950 tx=0x7fa8e40058e0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 msgr2=0x7fa8e819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 --2- 172.21.15.66:0/4261729467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa8e810c3e0 0x7fa8e819c750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0007ee0 tx=0x7fa8e000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ed8df700 1 -- 172.21.15.66:0/4261729467 reap_dead start 2023-07-21T22:20:02.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 shutdown_connections 2023-07-21T22:20:02.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 >> 172.21.15.66:0/4261729467 conn(0x7fa8e80fb660 msgr2=0x7fa8e8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:02.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 shutdown_connections 2023-07-21T22:20:02.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.358+0000 7fa8ef342700 1 -- 172.21.15.66:0/4261729467 wait complete. 2023-07-21T22:20:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.304+0000 7f5eaf636700 1 -- 172.21.15.66:0/3332794741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 msgr2=0x7f5ea810bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.304+0000 7f5eaf636700 1 --2- 172.21.15.66:0/3332794741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 0x7f5ea810bee0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009990 tx=0x7f5e98009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 -- 172.21.15.66:0/3332794741 shutdown_connections 2023-07-21T22:20:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 --2- 172.21.15.66:0/3332794741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ea8072870 0x7f5ea8072c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 --2- 172.21.15.66:0/3332794741 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea810c420 0x7f5ea810e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 --2- 172.21.15.66:0/3332794741 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 0x7f5ea810bee0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 -- 172.21.15.66:0/3332794741 >> 172.21.15.66:0/3332794741 conn(0x7f5ea80fb660 msgr2=0x7f5ea80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 -- 172.21.15.66:0/3332794741 shutdown_connections 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.305+0000 7f5eaf636700 1 -- 172.21.15.66:0/3332794741 wait complete. 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.306+0000 7f5eaf636700 1 Processor -- start 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 -- start start 2023-07-21T22:20:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8072870 0x7f5ea806aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 0x7f5ea806af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ea810c420 0x7f5ea81ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea81ac990 con 0x7f5ea8073180 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea81acb00 con 0x7f5ea810c420 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eaf636700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea81accc0 con 0x7f5ea8072870 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eacbd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 0x7f5ea806af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5ead3d2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8072870 0x7f5ea806aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eacbd1700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 0x7f5ea806af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44286/0 (socket says 172.21.15.66:44286) 2023-07-21T22:20:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5ead3d2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8072870 0x7f5ea806aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51262/0 (socket says 172.21.15.66:51262) 2023-07-21T22:20:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eacbd1700 1 -- 172.21.15.66:0/2278042128 learned_addr learned my addr 172.21.15.66:0/2278042128 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.307+0000 7f5eadbd3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ea810c420 0x7f5ea81ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.308+0000 7f5ead3d2700 1 -- 172.21.15.66:0/2278042128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ea810c420 msgr2=0x7f5ea81ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.308+0000 7f5ead3d2700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ea810c420 0x7f5ea81ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.308+0000 7f5ead3d2700 1 -- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 msgr2=0x7f5ea806af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.308+0000 7f5ead3d2700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ea8073180 0x7f5ea806af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.308+0000 7f5ead3d2700 1 -- 172.21.15.66:0/2278042128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980095f0 con 0x7f5ea8072870 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.308+0000 7f5ead3d2700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8072870 0x7f5ea806aa30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e9f0 tx=0x7f5ea400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.309+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400eee0 con 0x7f5ea8072870 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.309+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ea400cbd0 con 0x7f5ea8072870 2023-07-21T22:20:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.309+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea81acfa0 con 0x7f5ea8072870 2023-07-21T22:20:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.309+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4018680 con 0x7f5ea8072870 2023-07-21T22:20:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.309+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea81ad410 con 0x7f5ea8072870 2023-07-21T22:20:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.310+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ea8111ef0 con 0x7f5ea8072870 2023-07-21T22:20:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.312+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5ea400cd40 con 0x7f5ea8072870 2023-07-21T22:20:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.313+0000 7f5e9e7fc700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5e940a1210 0x7f5e940a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.314+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ea4014070 con 0x7f5ea8072870 2023-07-21T22:20:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.314+0000 7f5e9e7fc700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] conn(0x7f5e940ac220 0x7f5e940ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.314+0000 7f5eacbd1700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5e940a1210 0x7f5e940a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.314+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 --> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e940aed80 con 0x7f5e940ac220 2023-07-21T22:20:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.314+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ea40cf3e0 con 0x7f5ea8072870 2023-07-21T22:20:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.314+0000 7f5eadbd3700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] conn(0x7f5e940ac220 0x7f5e940ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.315+0000 7f5eacbd1700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5e940a1210 0x7f5e940a36c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5ea806c050 tx=0x7f5e9800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.315+0000 7f5eadbd3700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] conn(0x7f5e940ac220 0x7f5e940ae6b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.316+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== osd.8 v2:172.21.15.149:6816/1996550641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e940aed80 con 0x7f5e940ac220 2023-07-21T22:20:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.351+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 --> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ea804e210 con 0x7f5e940ac220 2023-07-21T22:20:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5e9e7fc700 1 -- 172.21.15.66:0/2278042128 <== osd.8 v2:172.21.15.149:6816/1996550641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5ea804e210 con 0x7f5e940ac220 2023-07-21T22:20:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] conn(0x7f5e940ac220 msgr2=0x7f5e940ae6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] conn(0x7f5e940ac220 0x7f5e940ae6b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5e940a1210 msgr2=0x7f5e940a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5e940a1210 0x7f5e940a36c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5ea806c050 tx=0x7f5e9800b260 comp rx=0 tx=0).stop 2023-07-21T22:20:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8072870 msgr2=0x7f5ea806aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 --2- 172.21.15.66:0/2278042128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ea8072870 0x7f5ea806aa30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5ea400e9f0 tx=0x7f5ea400ed00 comp rx=0 tx=0).stop 2023-07-21T22:20:02.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eadbd3700 1 -- 172.21.15.66:0/2278042128 reap_dead start 2023-07-21T22:20:02.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 shutdown_connections 2023-07-21T22:20:02.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.352+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 >> 172.21.15.66:0/2278042128 conn(0x7f5ea80fb660 msgr2=0x7f5ea810e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:02.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.353+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 shutdown_connections 2023-07-21T22:20:02.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.353+0000 7f5eaf636700 1 -- 172.21.15.66:0/2278042128 wait complete. 2023-07-21T22:20:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 -- 172.21.15.66:0/478171830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c0ffe00 msgr2=0x7f433c1001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 --2- 172.21.15.66:0/478171830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c0ffe00 0x7f433c1001d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4338009990 tx=0x7f4338009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 -- 172.21.15.66:0/478171830 shutdown_connections 2023-07-21T22:20:02.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 --2- 172.21.15.66:0/478171830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c0ffe00 0x7f433c1001d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 --2- 172.21.15.66:0/478171830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c104ea0 0x7f433c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 --2- 172.21.15.66:0/478171830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f433c1007a0 0x7f433c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 -- 172.21.15.66:0/478171830 >> 172.21.15.66:0/478171830 conn(0x7f433c0fb660 msgr2=0x7f433c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 -- 172.21.15.66:0/478171830 shutdown_connections 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.401+0000 7f4344055700 1 -- 172.21.15.66:0/478171830 wait complete. 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.402+0000 7f4344055700 1 Processor -- start 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.402+0000 7f4344055700 1 -- start start 2023-07-21T22:20:02.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4344055700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c0ffe00 0x7f433c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4344055700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c1007a0 0x7f433c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4344055700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f433c104ea0 0x7f433c195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4344055700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c196140 con 0x7f433c104ea0 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4344055700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c1962b0 con 0x7f433c1007a0 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4344055700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c196470 con 0x7f433c0ffe00 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4341df1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c0ffe00 0x7f433c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f43425f2700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f433c104ea0 0x7f433c195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4341df1700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c0ffe00 0x7f433c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51282/0 (socket says 172.21.15.66:51282) 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f4341df1700 1 -- 172.21.15.66:0/3890997601 learned_addr learned my addr 172.21.15.66:0/3890997601 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.403+0000 7f43415f0700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c1007a0 0x7f433c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4341df1700 1 -- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c1007a0 msgr2=0x7f433c19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4341df1700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f433c1007a0 0x7f433c19bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4341df1700 1 -- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f433c104ea0 msgr2=0x7f433c195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4341df1700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f433c104ea0 0x7f433c195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4341df1700 1 -- 172.21.15.66:0/3890997601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380095f0 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4341df1700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c0ffe00 0x7f433c19ba50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4338005910 tx=0x7f4338004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433801d070 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c1966f0 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4338022470 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338017650 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.404+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c1a2a40 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.405+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f433c10fdb0 con 0x7f433c0ffe00 2023-07-21T22:20:02.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.408+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f433800fad0 con 0x7f433c0ffe00 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.409+0000 7f4332ffd700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f43280a11e0 0x7f43280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.409+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43380ce990 con 0x7f433c0ffe00 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.409+0000 7f43415f0700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f43280a11e0 0x7f43280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.409+0000 7f4332ffd700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] conn(0x7f43280ac1f0 0x7f43280ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.409+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 --> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f43280aed50 con 0x7f43280ac1f0 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.409+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43380ced00 con 0x7f433c0ffe00 2023-07-21T22:20:02.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.410+0000 7f43425f2700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] conn(0x7f43280ac1f0 0x7f43280ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.410+0000 7f43415f0700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f43280a11e0 0x7f43280a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f433c0fcb00 tx=0x7f432c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.410+0000 7f43425f2700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] conn(0x7f43280ac1f0 0x7f43280ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.411+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== osd.10 v2:172.21.15.104:6824/3265864999 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f43280aed50 con 0x7f43280ac1f0 2023-07-21T22:20:02.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.428+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 --> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f433c1040a0 con 0x7f43280ac1f0 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.428+0000 7f4332ffd700 1 -- 172.21.15.66:0/3890997601 <== osd.10 v2:172.21.15.104:6824/3265864999 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f433c1040a0 con 0x7f43280ac1f0 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] conn(0x7f43280ac1f0 msgr2=0x7f43280ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] conn(0x7f43280ac1f0 0x7f43280ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f43280a11e0 msgr2=0x7f43280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f43280a11e0 0x7f43280a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f433c0fcb00 tx=0x7f432c00b410 comp rx=0 tx=0).stop 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c0ffe00 msgr2=0x7f433c19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:02.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 --2- 172.21.15.66:0/3890997601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f433c0ffe00 0x7f433c19ba50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4338005910 tx=0x7f4338004730 comp rx=0 tx=0).stop 2023-07-21T22:20:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f43425f2700 1 -- 172.21.15.66:0/3890997601 reap_dead start 2023-07-21T22:20:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 shutdown_connections 2023-07-21T22:20:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 >> 172.21.15.66:0/3890997601 conn(0x7f433c0fb660 msgr2=0x7f433c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 shutdown_connections 2023-07-21T22:20:02.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:02.429+0000 7f4344055700 1 -- 172.21.15.66:0/3890997601 wait complete. 2023-07-21T22:20:02.714 INFO:teuthology.orchestra.run.smithi066.stdout:292057776159 2023-07-21T22:20:02.714 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-21T22:20:03.224 INFO:teuthology.orchestra.run.smithi066.stdout:257698037797 2023-07-21T22:20:03.224 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-21T22:20:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.186+0000 7f701140c700 1 -- 172.21.15.66:0/3088169707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1029e0 msgr2=0x7f700c1069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.186+0000 7f701140c700 1 --2- 172.21.15.66:0/3088169707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1029e0 0x7f700c1069b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f70000099c0 tx=0x7f7000009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.186+0000 7f701140c700 1 -- 172.21.15.66:0/3088169707 shutdown_connections 2023-07-21T22:20:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.186+0000 7f701140c700 1 --2- 172.21.15.66:0/3088169707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f700c1070e0 0x7f700c10ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.187+0000 7f701140c700 1 --2- 172.21.15.66:0/3088169707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c102040 0x7f700c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.187+0000 7f701140c700 1 --2- 172.21.15.66:0/3088169707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1029e0 0x7f700c1069b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.187+0000 7f701140c700 1 -- 172.21.15.66:0/3088169707 >> 172.21.15.66:0/3088169707 conn(0x7f700c0fd900 msgr2=0x7f700c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:03.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.187+0000 7f701140c700 1 -- 172.21.15.66:0/3088169707 shutdown_connections 2023-07-21T22:20:03.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.187+0000 7f701140c700 1 -- 172.21.15.66:0/3088169707 wait complete. 2023-07-21T22:20:03.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 Processor -- start 2023-07-21T22:20:03.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 -- start start 2023-07-21T22:20:03.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f700c102040 0x7f700c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:03.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c1029e0 0x7f700c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:03.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1070e0 0x7f700c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:03.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19ce80 con 0x7f700c1070e0 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19cff0 con 0x7f700c102040 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.188+0000 7f701140c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19d1b0 con 0x7f700c1029e0 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1070e0 0x7f700c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c1029e0 0x7f700c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c1029e0 0x7f700c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51300/0 (socket says 172.21.15.66:51300) 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 -- 172.21.15.66:0/2428086082 learned_addr learned my addr 172.21.15.66:0/2428086082 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:03.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700affd700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f700c102040 0x7f700c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:03.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 -- 172.21.15.66:0/2428086082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f700c102040 msgr2=0x7f700c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:03.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f700c102040 0x7f700c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 -- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1070e0 msgr2=0x7f700c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f700c1070e0 0x7f700c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.189+0000 7f700a7fc700 1 -- 172.21.15.66:0/2428086082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000095f0 con 0x7f700c1029e0 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.190+0000 7f700a7fc700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c1029e0 0x7f700c198400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70000058e0 tx=0x7f7000005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.190+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700001d070 con 0x7f700c1029e0 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.190+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c19d430 con 0x7f700c1029e0 2023-07-21T22:20:03.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.190+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f700000f790 con 0x7f700c1029e0 2023-07-21T22:20:03.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.190+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000004a90 con 0x7f700c1029e0 2023-07-21T22:20:03.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.190+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1a4cf0 con 0x7f700c1029e0 2023-07-21T22:20:03.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.191+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f700c19d700 con 0x7f700c1029e0 2023-07-21T22:20:03.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.193+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f700000fb90 con 0x7f700c1029e0 2023-07-21T22:20:03.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.195+0000 7f6ff3fff700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6ff40a11e0 0x7f6ff40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:03.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.195+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f70000cdc60 con 0x7f700c1029e0 2023-07-21T22:20:03.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.195+0000 7f700affd700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6ff40a11e0 0x7f6ff40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:03.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.195+0000 7f6ff3fff700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] conn(0x7f6ff40ac1f0 0x7f6ff40ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:03.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.195+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 --> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ff40aed50 con 0x7f6ff40ac1f0 2023-07-21T22:20:03.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.195+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70000cdfd0 con 0x7f700c1029e0 2023-07-21T22:20:03.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.196+0000 7f700b7fe700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] conn(0x7f6ff40ac1f0 0x7f6ff40ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:03.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.196+0000 7f700affd700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6ff40a11e0 0x7f6ff40a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005950 tx=0x7f6ffc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:03.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.197+0000 7f700b7fe700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] conn(0x7f6ff40ac1f0 0x7f6ff40ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:03.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.197+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== osd.2 v2:172.21.15.149:6800/943388416 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6ff40aed50 con 0x7f6ff40ac1f0 2023-07-21T22:20:03.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.214+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 --> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f700c111fd0 con 0x7f6ff40ac1f0 2023-07-21T22:20:03.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.215+0000 7f6ff3fff700 1 -- 172.21.15.66:0/2428086082 <== osd.2 v2:172.21.15.149:6800/943388416 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f700400f040 con 0x7f6ff40ac1f0 2023-07-21T22:20:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] conn(0x7f6ff40ac1f0 msgr2=0x7f6ff40ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] conn(0x7f6ff40ac1f0 0x7f6ff40ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6ff40a11e0 msgr2=0x7f6ff40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6ff40a11e0 0x7f6ff40a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc005950 tx=0x7f6ffc00b410 comp rx=0 tx=0).stop 2023-07-21T22:20:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c1029e0 msgr2=0x7f700c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:03.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 --2- 172.21.15.66:0/2428086082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f700c1029e0 0x7f700c198400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70000058e0 tx=0x7f7000005f10 comp rx=0 tx=0).stop 2023-07-21T22:20:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f700b7fe700 1 -- 172.21.15.66:0/2428086082 reap_dead start 2023-07-21T22:20:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 shutdown_connections 2023-07-21T22:20:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 >> 172.21.15.66:0/2428086082 conn(0x7f700c0fd900 msgr2=0x7f700c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:03.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 shutdown_connections 2023-07-21T22:20:03.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:03.216+0000 7f701140c700 1 -- 172.21.15.66:0/2428086082 wait complete. 2023-07-21T22:20:03.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:03 smithi066 ceph-mon[126379]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-21T22:20:03.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:03 smithi104 ceph-mon[129021]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-21T22:20:03.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:03 smithi149 ceph-mon[129268]: pgmap v304: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-21T22:20:03.783 INFO:teuthology.orchestra.run.smithi066.stdout:223338299433 2023-07-21T22:20:03.783 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-21T22:20:04.046 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:04.084 INFO:teuthology.orchestra.run.smithi066.stdout:77309411392 2023-07-21T22:20:04.085 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-21T22:20:04.092 INFO:teuthology.orchestra.run.smithi066.stdout:313532612634 2023-07-21T22:20:04.092 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-21T22:20:04.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 -- 172.21.15.66:0/3914631432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e980febe0 msgr2=0x7f4e980fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:04.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 --2- 172.21.15.66:0/3914631432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e980febe0 0x7f4e980fefb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009990 tx=0x7f4e94009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 -- 172.21.15.66:0/3914631432 shutdown_connections 2023-07-21T22:20:04.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 --2- 172.21.15.66:0/3914631432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e980febe0 0x7f4e980fefb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 --2- 172.21.15.66:0/3914631432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e98103f10 0x7f4e98104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 --2- 172.21.15.66:0/3914631432 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e980ff4f0 0x7f4e981039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 -- 172.21.15.66:0/3914631432 >> 172.21.15.66:0/3914631432 conn(0x7f4e98075b70 msgr2=0x7f4e98075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:04.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 -- 172.21.15.66:0/3914631432 shutdown_connections 2023-07-21T22:20:04.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.514+0000 7f4e9f865700 1 -- 172.21.15.66:0/3914631432 wait complete. 2023-07-21T22:20:04.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.515+0000 7f4e9f865700 1 Processor -- start 2023-07-21T22:20:04.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 -- start start 2023-07-21T22:20:04.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e980febe0 0x7f4e9819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:04.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e980ff4f0 0x7f4e9819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:04.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e98103f10 0x7f4e981a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:04.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981a12a0 con 0x7f4e980febe0 2023-07-21T22:20:04.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981a1410 con 0x7f4e98103f10 2023-07-21T22:20:04.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9f865700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981a15d0 con 0x7f4e980ff4f0 2023-07-21T22:20:04.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9d601700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e980febe0 0x7f4e9819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:04.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9d601700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e980febe0 0x7f4e9819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44346/0 (socket says 172.21.15.66:44346) 2023-07-21T22:20:04.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9de02700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e98103f10 0x7f4e981a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:04.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.516+0000 7f4e9d601700 1 -- 172.21.15.66:0/1260337503 learned_addr learned my addr 172.21.15.66:0/1260337503 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:04.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9ce00700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e980ff4f0 0x7f4e9819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:04.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9de02700 1 -- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e980ff4f0 msgr2=0x7f4e9819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:04.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9de02700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e980ff4f0 0x7f4e9819c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9de02700 1 -- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e980febe0 msgr2=0x7f4e9819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:04.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9de02700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e980febe0 0x7f4e9819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9de02700 1 -- 172.21.15.66:0/1260337503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940095f0 con 0x7f4e98103f10 2023-07-21T22:20:04.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.517+0000 7f4e9de02700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e98103f10 0x7f4e981a0c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000efc0 tx=0x7f4e9000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:04.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.518+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90018070 con 0x7f4e98103f10 2023-07-21T22:20:04.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.518+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e9000f040 con 0x7f4e98103f10 2023-07-21T22:20:04.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.518+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e981a1850 con 0x7f4e98103f10 2023-07-21T22:20:04.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.518+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9000cc80 con 0x7f4e98103f10 2023-07-21T22:20:04.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.518+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e981a9110 con 0x7f4e98103f10 2023-07-21T22:20:04.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.519+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4e98116300 con 0x7f4e98103f10 2023-07-21T22:20:04.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.521+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4e90003a00 con 0x7f4e98103f10 2023-07-21T22:20:04.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.523+0000 7f4e8e7fc700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e840a1330 0x7f4e840a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:04.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.523+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4e900cd130 con 0x7f4e98103f10 2023-07-21T22:20:04.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.523+0000 7f4e9d601700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e840a1330 0x7f4e840a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:04.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.523+0000 7f4e8e7fc700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] conn(0x7f4e840ac340 0x7f4e840ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:04.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.523+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 --> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4e840aeea0 con 0x7f4e840ac340 2023-07-21T22:20:04.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.523+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e900cd4a0 con 0x7f4e98103f10 2023-07-21T22:20:04.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.524+0000 7f4e9ce00700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] conn(0x7f4e840ac340 0x7f4e840ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:04.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.524+0000 7f4e9d601700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e840a1330 0x7f4e840a37e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e94006010 tx=0x7f4e9400b610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:04.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.524+0000 7f4e9ce00700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] conn(0x7f4e840ac340 0x7f4e840ae7d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:04.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.525+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== osd.11 v2:172.21.15.149:6824/2237094047 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4e840aeea0 con 0x7f4e840ac340 2023-07-21T22:20:04.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.558+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 --> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4e9804e210 con 0x7f4e840ac340 2023-07-21T22:20:04.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.559+0000 7f4e8e7fc700 1 -- 172.21.15.66:0/1260337503 <== osd.11 v2:172.21.15.149:6824/2237094047 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4e9804e210 con 0x7f4e840ac340 2023-07-21T22:20:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.559+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] conn(0x7f4e840ac340 msgr2=0x7f4e840ae7d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.559+0000 7f4e9f865700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] conn(0x7f4e840ac340 0x7f4e840ae7d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e840a1330 msgr2=0x7f4e840a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4e840a1330 0x7f4e840a37e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e94006010 tx=0x7f4e9400b610 comp rx=0 tx=0).stop 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e98103f10 msgr2=0x7f4e981a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 --2- 172.21.15.66:0/1260337503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e98103f10 0x7f4e981a0c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e9000efc0 tx=0x7f4e9000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9de02700 1 -- 172.21.15.66:0/1260337503 reap_dead start 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 shutdown_connections 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 >> 172.21.15.66:0/1260337503 conn(0x7f4e98075b70 msgr2=0x7f4e981019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:04.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 shutdown_connections 2023-07-21T22:20:04.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:04.560+0000 7f4e9f865700 1 -- 172.21.15.66:0/1260337503 wait complete. 2023-07-21T22:20:04.986 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:05.417 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:05.475 INFO:teuthology.orchestra.run.smithi066.stdout:343597383705 2023-07-21T22:20:05.475 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-21T22:20:05.549 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:05 smithi066 ceph-mon[126379]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-21T22:20:05.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:05 smithi104 ceph-mon[129021]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-21T22:20:05.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:05 smithi149 ceph-mon[129268]: pgmap v305: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-21T22:20:05.939 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:06.848 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:07.595 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:07 smithi066 ceph-mon[126379]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:07.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:07 smithi104 ceph-mon[129021]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:07.699 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:07 smithi149 ceph-mon[129268]: pgmap v306: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:08.379 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:08.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.343+0000 7f7132943700 1 -- 172.21.15.66:0/2263816941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 msgr2=0x7f712c104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.343+0000 7f7132943700 1 --2- 172.21.15.66:0/2263816941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c104340 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7124009f90 tx=0x7f712400b330 comp rx=0 tx=0).stop 2023-07-21T22:20:08.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 -- 172.21.15.66:0/2263816941 shutdown_connections 2023-07-21T22:20:08.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 --2- 172.21.15.66:0/2263816941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c104340 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 --2- 172.21.15.66:0/2263816941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0febe0 0x7f712c0fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 --2- 172.21.15.66:0/2263816941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f712c0ff4f0 0x7f712c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 -- 172.21.15.66:0/2263816941 >> 172.21.15.66:0/2263816941 conn(0x7f712c075b70 msgr2=0x7f712c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:08.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 -- 172.21.15.66:0/2263816941 shutdown_connections 2023-07-21T22:20:08.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.345+0000 7f7132943700 1 -- 172.21.15.66:0/2263816941 wait complete. 2023-07-21T22:20:08.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.346+0000 7f7132943700 1 Processor -- start 2023-07-21T22:20:08.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.347+0000 7f7132943700 1 -- start start 2023-07-21T22:20:08.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f7132943700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f712c0febe0 0x7f712c10eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f7132943700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 0x7f712c10f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f7132943700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c1c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f7132943700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1c1de0 con 0x7f712c0febe0 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f7132943700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1c1f20 con 0x7f712c103f10 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f7132943700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1c20e0 con 0x7f712c0ff4f0 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 0x7f712c10f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 0x7f712c10f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51364/0 (socket says 172.21.15.66:51364) 2023-07-21T22:20:08.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.348+0000 7f712b7fe700 1 -- 172.21.15.66:0/1681954253 learned_addr learned my addr 172.21.15.66:0/1681954253 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:08.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f712b7fe700 1 -- 172.21.15.66:0/1681954253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 msgr2=0x7f712c1c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f7130ee0700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c1c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f712b7fe700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c1c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f712b7fe700 1 -- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f712c0febe0 msgr2=0x7f712c10eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f712b7fe700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f712c0febe0 0x7f712c10eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f712b7fe700 1 -- 172.21.15.66:0/1681954253 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712001a600 con 0x7f712c0ff4f0 2023-07-21T22:20:08.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f7130ee0700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c1c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:20:08.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.349+0000 7f712b7fe700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 0x7f712c10f0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f712001cbe0 tx=0x7f712000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:08.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.351+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712001aa20 con 0x7f712c0ff4f0 2023-07-21T22:20:08.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.352+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71200044e0 con 0x7f712c0ff4f0 2023-07-21T22:20:08.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.352+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712000f460 con 0x7f712c0ff4f0 2023-07-21T22:20:08.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.352+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7124009c40 con 0x7f712c0ff4f0 2023-07-21T22:20:08.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.352+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c1c26e0 con 0x7f712c0ff4f0 2023-07-21T22:20:08.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.354+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c04e210 con 0x7f712c0ff4f0 2023-07-21T22:20:08.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.357+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7120003fd0 con 0x7f712c0ff4f0 2023-07-21T22:20:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.359+0000 7f71297fa700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f71140a13b0 0x7f71140a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.359+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f71200cd0f0 con 0x7f712c0ff4f0 2023-07-21T22:20:08.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.360+0000 7f712bfff700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f71140a13b0 0x7f71140a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.361+0000 7f712bfff700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f71140a13b0 0x7f71140a3860 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f711c005950 tx=0x7f711c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:08.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.363+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7120096370 con 0x7f712c0ff4f0 2023-07-21T22:20:08.456 INFO:teuthology.orchestra.run.smithi066.stdout:34359738442 2023-07-21T22:20:08.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 -- 172.21.15.66:0/2457833666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff158104ea0 msgr2=0x7ff15810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 --2- 172.21.15.66:0/2457833666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff158104ea0 0x7ff15810ca20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff15000b0d0 tx=0x7ff15000b3e0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 -- 172.21.15.66:0/2457833666 shutdown_connections 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 --2- 172.21.15.66:0/2457833666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1580ffe00 0x7ff1581001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 --2- 172.21.15.66:0/2457833666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff158104ea0 0x7ff15810ca20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 --2- 172.21.15.66:0/2457833666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1581007a0 0x7ff158104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 -- 172.21.15.66:0/2457833666 >> 172.21.15.66:0/2457833666 conn(0x7ff1580fb660 msgr2=0x7ff1580fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 -- 172.21.15.66:0/2457833666 shutdown_connections 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.009+0000 7ff15d45a700 1 -- 172.21.15.66:0/2457833666 wait complete. 2023-07-21T22:20:08.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.011+0000 7ff15d45a700 1 Processor -- start 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.011+0000 7ff15d45a700 1 -- start start 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff15d45a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 0x7ff15806f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff15d45a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1581007a0 0x7ff15806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff15d45a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff158104ea0 0x7ff15806a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff15d45a700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15806afa0 con 0x7ff1580ffe00 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff15d45a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15806b110 con 0x7ff158104ea0 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff15d45a700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15806b2d0 con 0x7ff1581007a0 2023-07-21T22:20:08.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff156ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 0x7ff15806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff156ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 0x7ff15806f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44358/0 (socket says 172.21.15.66:44358) 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff156ffd700 1 -- 172.21.15.66:0/69719814 learned_addr learned my addr 172.21.15.66:0/69719814 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff1567fc700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1581007a0 0x7ff15806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.012+0000 7ff1577fe700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff158104ea0 0x7ff15806a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1567fc700 1 -- 172.21.15.66:0/69719814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff158104ea0 msgr2=0x7ff15806a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1567fc700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff158104ea0 0x7ff15806a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1567fc700 1 -- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 msgr2=0x7ff15806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1567fc700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 0x7ff15806f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1567fc700 1 -- 172.21.15.66:0/69719814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150009d00 con 0x7ff1581007a0 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1577fe700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff158104ea0 0x7ff15806a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff156ffd700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 0x7ff15806f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff1567fc700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1581007a0 0x7ff15806fcc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff158069c30 tx=0x7ff14c00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00f890 con 0x7ff1581007a0 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15806b5b0 con 0x7ff1581007a0 2023-07-21T22:20:08.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff14c010460 con 0x7ff1581007a0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.013+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00e610 con 0x7ff1581007a0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.014+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15806e710 con 0x7ff1581007a0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.015+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15804e210 con 0x7ff1581007a0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.017+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff14c010840 con 0x7ff1581007a0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.019+0000 7ff13ffff700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff1400a1260 0x7ff1400a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.019+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff14c0ccf30 con 0x7ff1581007a0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.020+0000 7ff156ffd700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff1400a1260 0x7ff1400a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:08.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.020+0000 7ff156ffd700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff1400a1260 0x7ff1400a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff148011550 tx=0x7ff148009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.021+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff14c0f9770 con 0x7ff1581007a0 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.254+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff15806b740 con 0x7ff1581007a0 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.255+0000 7ff13ffff700 1 -- 172.21.15.66:0/69719814 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff14c016090 con 0x7ff1581007a0 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff1400a1260 msgr2=0x7ff1400a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff1400a1260 0x7ff1400a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff148011550 tx=0x7ff148009260 comp rx=0 tx=0).stop 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1581007a0 msgr2=0x7ff15806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1581007a0 0x7ff15806fcc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff158069c30 tx=0x7ff14c00dc30 comp rx=0 tx=0).stop 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 shutdown_connections 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff1400a1260 0x7ff1400a3710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff158104ea0 0x7ff15806a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1580ffe00 0x7ff15806f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 --2- 172.21.15.66:0/69719814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1581007a0 0x7ff15806fcc0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 >> 172.21.15.66:0/69719814 conn(0x7ff1580fb660 msgr2=0x7ff1580fc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 shutdown_connections 2023-07-21T22:20:08.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.258+0000 7ff15d45a700 1 -- 172.21.15.66:0/69719814 wait complete. 2023-07-21T22:20:08.653 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:08.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.666+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f712c10fec0 con 0x7f712c0ff4f0 2023-07-21T22:20:08.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.667+0000 7f71297fa700 1 -- 172.21.15.66:0/1681954253 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7120099990 con 0x7f712c0ff4f0 2023-07-21T22:20:08.711 INFO:teuthology.orchestra.run.smithi066.stdout:55834574917 2023-07-21T22:20:08.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f71140a13b0 msgr2=0x7f71140a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f71140a13b0 0x7f71140a3860 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f711c005950 tx=0x7f711c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 msgr2=0x7f712c10f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:08.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 0x7f712c10f0a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f712001cbe0 tx=0x7f712000b260 comp rx=0 tx=0).stop 2023-07-21T22:20:08.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 shutdown_connections 2023-07-21T22:20:08.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f71140a13b0 0x7f71140a3860 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f712c103f10 0x7f712c1c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f712c0febe0 0x7f712c10eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 --2- 172.21.15.66:0/1681954253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f712c0ff4f0 0x7f712c10f0a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:08.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.669+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 >> 172.21.15.66:0/1681954253 conn(0x7f712c075b70 msgr2=0x7f712c101970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:08.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.670+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 shutdown_connections 2023-07-21T22:20:08.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:08.670+0000 7f7132943700 1 -- 172.21.15.66:0/1681954253 wait complete. 2023-07-21T22:20:09.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:08 smithi104 ceph-mon[129021]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:09.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:08 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/69719814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:20:09.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:08 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1681954253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:20:09.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:08 smithi066 ceph-mon[126379]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:09.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:08 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/69719814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:20:09.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:08 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1681954253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:20:09.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:08 smithi149 ceph-mon[129268]: pgmap v307: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:09.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:08 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/69719814' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T22:20:09.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:08 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1681954253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T22:20:09.300 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738440 got 34359738442 for osd.0 2023-07-21T22:20:09.300 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:09.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.625+0000 7f9d9c68c700 1 -- 172.21.15.66:0/882855790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d940ffe00 msgr2=0x7f9d941001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:09.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.625+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/882855790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d940ffe00 0x7f9d941001d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009990 tx=0x7f9d90009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 -- 172.21.15.66:0/882855790 shutdown_connections 2023-07-21T22:20:09.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/882855790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d940ffe00 0x7f9d941001d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/882855790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d94104ea0 0x7f9d9410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/882855790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d941007a0 0x7f9d94104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 -- 172.21.15.66:0/882855790 >> 172.21.15.66:0/882855790 conn(0x7f9d940fb660 msgr2=0x7f9d940fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:09.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 -- 172.21.15.66:0/882855790 shutdown_connections 2023-07-21T22:20:09.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.626+0000 7f9d9c68c700 1 -- 172.21.15.66:0/882855790 wait complete. 2023-07-21T22:20:09.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.627+0000 7f9d9c68c700 1 Processor -- start 2023-07-21T22:20:09.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 -- start start 2023-07-21T22:20:09.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d940ffe00 0x7f9d9406bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d941007a0 0x7f9d9406c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 0x7f9d94070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94070b20 con 0x7f9d940ffe00 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94070c90 con 0x7f9d94104ea0 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.628+0000 7f9d9c68c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94070e50 con 0x7f9d941007a0 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d9ac29700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 0x7f9d94070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d99c27700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d941007a0 0x7f9d9406c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:09.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d9ac29700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 0x7f9d94070550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38350/0 (socket says 172.21.15.66:38350) 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d9ac29700 1 -- 172.21.15.66:0/163694273 learned_addr learned my addr 172.21.15.66:0/163694273 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d9a428700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d940ffe00 0x7f9d9406bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d99c27700 1 -- 172.21.15.66:0/163694273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 msgr2=0x7f9d94070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d99c27700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 0x7f9d94070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d99c27700 1 -- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d940ffe00 msgr2=0x7f9d9406bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d99c27700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d940ffe00 0x7f9d9406bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d99c27700 1 -- 172.21.15.66:0/163694273 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900095f0 con 0x7f9d941007a0 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.629+0000 7f9d9ac29700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 0x7f9d94070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.630+0000 7f9d99c27700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d941007a0 0x7f9d9406c0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d94069c30 tx=0x7f9d8400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.631+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84009930 con 0x7f9d941007a0 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.631+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d84010460 con 0x7f9d941007a0 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.631+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400b5d0 con 0x7f9d941007a0 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.631+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d94071130 con 0x7f9d941007a0 2023-07-21T22:20:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.631+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d941b18f0 con 0x7f9d941007a0 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.633+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9404e210 con 0x7f9d941007a0 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.637+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9d84010890 con 0x7f9d941007a0 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.639+0000 7f9d8b7fe700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9d800a1330 0x7f9d800a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.639+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9d840cc850 con 0x7f9d941007a0 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.640+0000 7f9d9a428700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9d800a1330 0x7f9d800a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.640+0000 7f9d9a428700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9d800a1330 0x7f9d800a37e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000b5c0 tx=0x7f9d90005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.643+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9d84095b30 con 0x7f9d941007a0 2023-07-21T22:20:09.866 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:09 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:20:09.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.897+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f9d9406cf10 con 0x7f9d941007a0 2023-07-21T22:20:09.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.897+0000 7f9d8b7fe700 1 -- 172.21.15.66:0/163694273 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d84017090 con 0x7f9d941007a0 2023-07-21T22:20:09.942 INFO:teuthology.orchestra.run.smithi066.stdout:201863462959 2023-07-21T22:20:09.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9d800a1330 msgr2=0x7f9d800a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9d800a1330 0x7f9d800a37e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000b5c0 tx=0x7f9d90005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d941007a0 msgr2=0x7f9d9406c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:09.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d941007a0 0x7f9d9406c0f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9d94069c30 tx=0x7f9d8400dc30 comp rx=0 tx=0).stop 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 shutdown_connections 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9d800a1330 0x7f9d800a37e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d94104ea0 0x7f9d94070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d940ffe00 0x7f9d9406bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 --2- 172.21.15.66:0/163694273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d941007a0 0x7f9d9406c0f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 >> 172.21.15.66:0/163694273 conn(0x7f9d940fb660 msgr2=0x7f9d940fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:09.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 shutdown_connections 2023-07-21T22:20:09.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:09.900+0000 7f9d9c68c700 1 -- 172.21.15.66:0/163694273 wait complete. 2023-07-21T22:20:10.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:09 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:20:10.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 -- 172.21.15.66:0/4054051865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08841007a0 msgr2=0x7f0884104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 --2- 172.21.15.66:0/4054051865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08841007a0 0x7f0884104770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f08740099c0 tx=0x7f0874009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 -- 172.21.15.66:0/4054051865 shutdown_connections 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 --2- 172.21.15.66:0/4054051865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08840ffe00 0x7f08841001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 --2- 172.21.15.66:0/4054051865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0884104ea0 0x7f088410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 --2- 172.21.15.66:0/4054051865 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08841007a0 0x7f0884104770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.016+0000 7f088b627700 1 -- 172.21.15.66:0/4054051865 >> 172.21.15.66:0/4054051865 conn(0x7f08840fb660 msgr2=0x7f08840fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.017+0000 7f088b627700 1 -- 172.21.15.66:0/4054051865 shutdown_connections 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.017+0000 7f088b627700 1 -- 172.21.15.66:0/4054051865 wait complete. 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 Processor -- start 2023-07-21T22:20:10.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 -- start start 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08840ffe00 0x7f0884197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 0x7f0884198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 0x7f088419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088419cec0 con 0x7f08840ffe00 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088419d030 con 0x7f0884104ea0 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.018+0000 7f088b627700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088419d1f0 con 0x7f08841007a0 2023-07-21T22:20:10.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f08893c3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08840ffe00 0x7f0884197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0888bc2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 0x7f0884198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 0x7f088419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 0x7f088419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38368/0 (socket says 172.21.15.66:38368) 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 -- 172.21.15.66:0/1248252351 learned_addr learned my addr 172.21.15.66:0/1248252351 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0888bc2700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 0x7f0884198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51390/0 (socket says 172.21.15.66:51390) 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 -- 172.21.15.66:0/1248252351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 msgr2=0x7f0884198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 0x7f0884198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 -- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08840ffe00 msgr2=0x7f0884197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08840ffe00 0x7f0884197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 -- 172.21.15.66:0/1248252351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08740095f0 con 0x7f0884104ea0 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0888bc2700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 0x7f0884198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.019+0000 7f0889bc4700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 0x7f088419c8b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f087c00ef30 tx=0x7f087c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.020+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c018070 con 0x7f0884104ea0 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.020+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f087c00f040 con 0x7f0884104ea0 2023-07-21T22:20:10.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.020+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088419d470 con 0x7f0884104ea0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.020+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c00cd00 con 0x7f0884104ea0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.020+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088419d720 con 0x7f0884104ea0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.021+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088404e210 con 0x7f0884104ea0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.023+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f087c003a00 con 0x7f0884104ea0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.024+0000 7f087a7fc700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f08700a11e0 0x7f08700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.025+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f087c0cc900 con 0x7f0884104ea0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.025+0000 7f08893c3700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f08700a11e0 0x7f08700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.025+0000 7f08893c3700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f08700a11e0 0x7f08700a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0880007be0 tx=0x7f08800073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:10.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.026+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f087c095c10 con 0x7f0884104ea0 2023-07-21T22:20:10.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:09 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:20:10.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.274+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f088419d660 con 0x7f0884104ea0 2023-07-21T22:20:10.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.275+0000 7f087a7fc700 1 -- 172.21.15.66:0/1248252351 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f087c099230 con 0x7f0884104ea0 2023-07-21T22:20:10.320 INFO:teuthology.orchestra.run.smithi066.stdout:146028888120 2023-07-21T22:20:10.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.278+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f08700a11e0 msgr2=0x7f08700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.278+0000 7f088b627700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f08700a11e0 0x7f08700a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0880007be0 tx=0x7f08800073d0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.278+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 msgr2=0x7f088419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.278+0000 7f088b627700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 0x7f088419c8b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f087c00ef30 tx=0x7f087c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 shutdown_connections 2023-07-21T22:20:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f08700a11e0 0x7f08700a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0884104ea0 0x7f088419c8b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08840ffe00 0x7f0884197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 --2- 172.21.15.66:0/1248252351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08841007a0 0x7f0884198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 >> 172.21.15.66:0/1248252351 conn(0x7f08840fb660 msgr2=0x7f08840fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:10.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 shutdown_connections 2023-07-21T22:20:10.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.279+0000 7f088b627700 1 -- 172.21.15.66:0/1248252351 wait complete. 2023-07-21T22:20:10.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.347+0000 7fee77890700 1 -- 172.21.15.66:0/234143051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100d60 msgr2=0x7fee70101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.347+0000 7fee77890700 1 --2- 172.21.15.66:0/234143051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100d60 0x7fee70101130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009990 tx=0x7fee6c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 -- 172.21.15.66:0/234143051 shutdown_connections 2023-07-21T22:20:10.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 --2- 172.21.15.66:0/234143051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee70100d60 0x7fee70101130 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 --2- 172.21.15.66:0/234143051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee7010a140 0x7fee7010c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 --2- 172.21.15.66:0/234143051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70101670 0x7fee70109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 -- 172.21.15.66:0/234143051 >> 172.21.15.66:0/234143051 conn(0x7fee700fa5a0 msgr2=0x7fee700fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:10.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 -- 172.21.15.66:0/234143051 shutdown_connections 2023-07-21T22:20:10.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.348+0000 7fee77890700 1 -- 172.21.15.66:0/234143051 wait complete. 2023-07-21T22:20:10.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 Processor -- start 2023-07-21T22:20:10.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 -- start start 2023-07-21T22:20:10.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 0x7fee70195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee70101670 0x7fee70196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee7010a140 0x7fee7019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7019ad10 con 0x7fee70101670 2023-07-21T22:20:10.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7019ae80 con 0x7fee7010a140 2023-07-21T22:20:10.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.349+0000 7fee77890700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7019b040 con 0x7fee70100d60 2023-07-21T22:20:10.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 0x7fee70195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee75e2d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee7010a140 0x7fee7019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 0x7fee70195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51406/0 (socket says 172.21.15.66:51406) 2023-07-21T22:20:10.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 -- 172.21.15.66:0/523821134 learned_addr learned my addr 172.21.15.66:0/523821134 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:10.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 -- 172.21.15.66:0/523821134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee7010a140 msgr2=0x7fee7019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee7010a140 0x7fee7019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 -- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee70101670 msgr2=0x7fee70196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee70101670 0x7fee70196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 -- 172.21.15.66:0/523821134 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0095f0 con 0x7fee70100d60 2023-07-21T22:20:10.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee75e2d700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee7010a140 0x7fee7019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:10.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.350+0000 7fee7562c700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 0x7fee70195d50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee6c005910 tx=0x7fee6c004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:10.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.351+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c01d070 con 0x7fee70100d60 2023-07-21T22:20:10.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.351+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee6c022470 con 0x7fee70100d60 2023-07-21T22:20:10.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.351+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c017650 con 0x7fee70100d60 2023-07-21T22:20:10.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.351+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7019b2c0 con 0x7fee70100d60 2023-07-21T22:20:10.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.351+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee701a29b0 con 0x7fee70100d60 2023-07-21T22:20:10.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.352+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7004e210 con 0x7fee70100d60 2023-07-21T22:20:10.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.354+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fee6c0225e0 con 0x7fee70100d60 2023-07-21T22:20:10.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.355+0000 7fee667fc700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fee5c0a11e0 0x7fee5c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:10.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.356+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fee6c0ce0c0 con 0x7fee70100d60 2023-07-21T22:20:10.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.356+0000 7fee74e2b700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fee5c0a11e0 0x7fee5c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:10.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.356+0000 7fee74e2b700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fee5c0a11e0 0x7fee5c0a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee60005950 tx=0x7fee600058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:10.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.357+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fee6c0973d0 con 0x7fee70100d60 2023-07-21T22:20:10.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.585+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee701970b0 con 0x7fee70100d60 2023-07-21T22:20:10.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.586+0000 7fee667fc700 1 -- 172.21.15.66:0/523821134 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee6c09a9f0 con 0x7fee70100d60 2023-07-21T22:20:10.630 INFO:teuthology.orchestra.run.smithi066.stdout:111669149756 2023-07-21T22:20:10.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fee5c0a11e0 msgr2=0x7fee5c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fee5c0a11e0 0x7fee5c0a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fee60005950 tx=0x7fee600058e0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 msgr2=0x7fee70195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:10.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 0x7fee70195d50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee6c005910 tx=0x7fee6c004710 comp rx=0 tx=0).stop 2023-07-21T22:20:10.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 shutdown_connections 2023-07-21T22:20:10.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fee5c0a11e0 0x7fee5c0a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee7010a140 0x7fee7019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee70100d60 0x7fee70195d50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 --2- 172.21.15.66:0/523821134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fee70101670 0x7fee70196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:10.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 >> 172.21.15.66:0/523821134 conn(0x7fee700fa5a0 msgr2=0x7fee7010c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:10.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 shutdown_connections 2023-07-21T22:20:10.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:10.588+0000 7fee77890700 1 -- 172.21.15.66:0/523821134 wait complete. 2023-07-21T22:20:10.787 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:10.827 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574915 got 55834574917 for osd.1 2023-07-21T22:20:10.827 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:10.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462957 got 201863462959 for osd.6 2023-07-21T22:20:10.835 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:11.118 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:10 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/163694273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:20:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:10 smithi104 ceph-mon[129021]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:10 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1248252351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:20:11.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:10 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/523821134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:20:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/163694273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:20:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:10 smithi066 ceph-mon[126379]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:11.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1248252351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:20:11.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/523821134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:20:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:10 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/163694273' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T22:20:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:10 smithi149 ceph-mon[129268]: pgmap v308: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:10 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1248252351' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T22:20:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:10 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/523821134' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T22:20:12.078 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888118 got 146028888120 for osd.4 2023-07-21T22:20:12.079 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:12.186 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:12.192 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149754 got 111669149756 for osd.3 2023-07-21T22:20:12.193 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:12.353 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:12.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 -- 172.21.15.66:0/3074690664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40041007a0 msgr2=0x7f4004104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 --2- 172.21.15.66:0/3074690664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40041007a0 0x7f4004104770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40099c0 tx=0x7f3ff4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 -- 172.21.15.66:0/3074690664 shutdown_connections 2023-07-21T22:20:12.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 --2- 172.21.15.66:0/3074690664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40040ffe00 0x7f40041001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 --2- 172.21.15.66:0/3074690664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4004104ea0 0x7f400410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 --2- 172.21.15.66:0/3074690664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40041007a0 0x7f4004104770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 -- 172.21.15.66:0/3074690664 >> 172.21.15.66:0/3074690664 conn(0x7f40040fb660 msgr2=0x7f40040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.598+0000 7f400b802700 1 -- 172.21.15.66:0/3074690664 shutdown_connections 2023-07-21T22:20:12.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.599+0000 7f400b802700 1 -- 172.21.15.66:0/3074690664 wait complete. 2023-07-21T22:20:12.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 Processor -- start 2023-07-21T22:20:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 -- start start 2023-07-21T22:20:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40040ffe00 0x7f4004197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 0x7f4004198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 0x7f400419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400419cec0 con 0x7f40040ffe00 2023-07-21T22:20:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400419d030 con 0x7f4004104ea0 2023-07-21T22:20:12.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.600+0000 7f400b802700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400419d1f0 con 0x7f40041007a0 2023-07-21T22:20:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 0x7f400419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4008d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 0x7f4004198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4008d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 0x7f4004198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:40682/0 (socket says 172.21.15.66:40682) 2023-07-21T22:20:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 0x7f400419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48126/0 (socket says 172.21.15.66:48126) 2023-07-21T22:20:12.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f400959e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40040ffe00 0x7f4004197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4008d9d700 1 -- 172.21.15.66:0/1047329546 learned_addr learned my addr 172.21.15.66:0/1047329546 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 -- 172.21.15.66:0/1047329546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 msgr2=0x7f4004198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 0x7f4004198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 -- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40040ffe00 msgr2=0x7f4004197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40040ffe00 0x7f4004197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 -- 172.21.15.66:0/1047329546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009520 con 0x7f4004104ea0 2023-07-21T22:20:12.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4008d9d700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 0x7f4004198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f4009d9f700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 0x7f400419c8b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00efc0 tx=0x7f3ffc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.601+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc018070 con 0x7f4004104ea0 2023-07-21T22:20:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.602+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff40095f0 con 0x7f4004104ea0 2023-07-21T22:20:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.602+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ffc00f040 con 0x7f4004104ea0 2023-07-21T22:20:12.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.602+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc00cd40 con 0x7f4004104ea0 2023-07-21T22:20:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.602+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40041a4b60 con 0x7f4004104ea0 2023-07-21T22:20:12.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.603+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400404e210 con 0x7f4004104ea0 2023-07-21T22:20:12.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.605+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3ffc003a00 con 0x7f4004104ea0 2023-07-21T22:20:12.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.606+0000 7f3ffa7fc700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ff00a1260 0x7f3ff00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:12.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.606+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3ffc0cca20 con 0x7f4004104ea0 2023-07-21T22:20:12.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.607+0000 7f400959e700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ff00a1260 0x7f3ff00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:12.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.607+0000 7f400959e700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ff00a1260 0x7f3ff00a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4000009df0 tx=0x7f4000009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:12.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.608+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3ffc095d30 con 0x7f4004104ea0 2023-07-21T22:20:12.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.864+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f40041a4cf0 con 0x7f4004104ea0 2023-07-21T22:20:12.909 INFO:teuthology.orchestra.run.smithi066.stdout:223338299435 2023-07-21T22:20:12.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.865+0000 7f3ffa7fc700 1 -- 172.21.15.66:0/1047329546 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ffc099350 con 0x7f4004104ea0 2023-07-21T22:20:12.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ff00a1260 msgr2=0x7f3ff00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ff00a1260 0x7f3ff00a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4000009df0 tx=0x7f4000009260 comp rx=0 tx=0).stop 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 msgr2=0x7f400419c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 0x7f400419c8b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00efc0 tx=0x7f3ffc00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 shutdown_connections 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ff00a1260 0x7f3ff00a3710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4004104ea0 0x7f400419c8b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40040ffe00 0x7f4004197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 --2- 172.21.15.66:0/1047329546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40041007a0 0x7f4004198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.867+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 >> 172.21.15.66:0/1047329546 conn(0x7f40040fb660 msgr2=0x7f40040fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.868+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 shutdown_connections 2023-07-21T22:20:12.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:12.868+0000 7f400b802700 1 -- 172.21.15.66:0/1047329546 wait complete. 2023-07-21T22:20:13.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 -- 172.21.15.66:0/1446300507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180ffe00 msgr2=0x7f84181001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:13.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 --2- 172.21.15.66:0/1446300507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180ffe00 0x7f84181001d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8414009990 tx=0x7f8414009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 -- 172.21.15.66:0/1446300507 shutdown_connections 2023-07-21T22:20:13.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 --2- 172.21.15.66:0/1446300507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84180ffe00 0x7f84181001d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 --2- 172.21.15.66:0/1446300507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8418104ea0 0x7f841810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 --2- 172.21.15.66:0/1446300507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84181007a0 0x7f8418104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 -- 172.21.15.66:0/1446300507 >> 172.21.15.66:0/1446300507 conn(0x7f84180fb660 msgr2=0x7f84180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 -- 172.21.15.66:0/1446300507 shutdown_connections 2023-07-21T22:20:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.044+0000 7f841f95d700 1 -- 172.21.15.66:0/1446300507 wait complete. 2023-07-21T22:20:13.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.045+0000 7f841f95d700 1 Processor -- start 2023-07-21T22:20:13.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.045+0000 7f841f95d700 1 -- start start 2023-07-21T22:20:13.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841f95d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84180ffe00 0x7f8418072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:13.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841f95d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84181007a0 0x7f8418072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:13.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841f95d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418104ea0 0x7f84180732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:13.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841f95d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181a40f0 con 0x7f8418104ea0 2023-07-21T22:20:13.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841f95d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181a4260 con 0x7f84181007a0 2023-07-21T22:20:13.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841f95d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181a4420 con 0x7f84180ffe00 2023-07-21T22:20:13.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841d6f9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84180ffe00 0x7f8418072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:13.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841d6f9700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84180ffe00 0x7f8418072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:40702/0 (socket says 172.21.15.66:40702) 2023-07-21T22:20:13.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841d6f9700 1 -- 172.21.15.66:0/4119675788 learned_addr learned my addr 172.21.15.66:0/4119675788 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:13.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841cef8700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84181007a0 0x7f8418072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841defa700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418104ea0 0x7f84180732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841cef8700 1 -- 172.21.15.66:0/4119675788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84180ffe00 msgr2=0x7f8418072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841cef8700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84180ffe00 0x7f8418072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841cef8700 1 -- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418104ea0 msgr2=0x7f84180732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841cef8700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418104ea0 0x7f84180732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841cef8700 1 -- 172.21.15.66:0/4119675788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8408009520 con 0x7f84181007a0 2023-07-21T22:20:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.046+0000 7f841defa700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418104ea0 0x7f84180732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.047+0000 7f841cef8700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84181007a0 0x7f8418072d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f840800e9c0 tx=0x7f840800ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.047+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800cb70 con 0x7f84181007a0 2023-07-21T22:20:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.047+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84080044e0 con 0x7f84181007a0 2023-07-21T22:20:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.047+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408003dc0 con 0x7f84181007a0 2023-07-21T22:20:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.047+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140095f0 con 0x7f84181007a0 2023-07-21T22:20:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.047+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84181a4a20 con 0x7f84181007a0 2023-07-21T22:20:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.048+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841804e210 con 0x7f84181007a0 2023-07-21T22:20:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.052+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8408003f80 con 0x7f84181007a0 2023-07-21T22:20:13.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.053+0000 7f840e7fc700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84040a1260 0x7f84040a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:13.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.054+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8408014070 con 0x7f84181007a0 2023-07-21T22:20:13.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.054+0000 7f841d6f9700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84040a1260 0x7f84040a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:13.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.054+0000 7f841d6f9700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84040a1260 0x7f84040a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8414006010 tx=0x7f841400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:13.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.057+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8408096480 con 0x7f84181007a0 2023-07-21T22:20:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.291+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f84181a4e30 con 0x7f84181007a0 2023-07-21T22:20:13.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.291+0000 7f840e7fc700 1 -- 172.21.15.66:0/4119675788 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8408099aa0 con 0x7f84181007a0 2023-07-21T22:20:13.336 INFO:teuthology.orchestra.run.smithi066.stdout:292057776161 2023-07-21T22:20:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84040a1260 msgr2=0x7f84040a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84040a1260 0x7f84040a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8414006010 tx=0x7f841400b260 comp rx=0 tx=0).stop 2023-07-21T22:20:13.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84181007a0 msgr2=0x7f8418072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84181007a0 0x7f8418072d50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f840800e9c0 tx=0x7f840800ed80 comp rx=0 tx=0).stop 2023-07-21T22:20:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 shutdown_connections 2023-07-21T22:20:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84040a1260 0x7f84040a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8418104ea0 0x7f84180732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84180ffe00 0x7f8418072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 --2- 172.21.15.66:0/4119675788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84181007a0 0x7f8418072d50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 >> 172.21.15.66:0/4119675788 conn(0x7f84180fb660 msgr2=0x7f8418103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.294+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 shutdown_connections 2023-07-21T22:20:13.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.295+0000 7f841f95d700 1 -- 172.21.15.66:0/4119675788 wait complete. 2023-07-21T22:20:13.361 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.361 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.362 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.362 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:12 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.362 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:12 smithi066 ceph-mon[126379]: pgmap v309: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:20:13.362 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:12 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1047329546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:20:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:12 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:12 smithi104 ceph-mon[129021]: pgmap v309: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:20:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:12 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1047329546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:20:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:12 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:13.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:12 smithi149 ceph-mon[129268]: pgmap v309: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:20:13.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:12 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1047329546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T22:20:14.020 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299433 got 223338299435 for osd.7 2023-07-21T22:20:14.021 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:14.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 -- 172.21.15.66:0/1237007661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 msgr2=0x7fdf0c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:14.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 --2- 172.21.15.66:0/1237007661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 0x7fdf0c10ca20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400b0d0 tx=0x7fdf0400b3e0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 -- 172.21.15.66:0/1237007661 shutdown_connections 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 --2- 172.21.15.66:0/1237007661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 --2- 172.21.15.66:0/1237007661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 0x7fdf0c10ca20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 --2- 172.21.15.66:0/1237007661 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1007a0 0x7fdf0c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 -- 172.21.15.66:0/1237007661 >> 172.21.15.66:0/1237007661 conn(0x7fdf0c0fb660 msgr2=0x7fdf0c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.882+0000 7fdf13af4700 1 -- 172.21.15.66:0/1237007661 shutdown_connections 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.883+0000 7fdf13af4700 1 -- 172.21.15.66:0/1237007661 wait complete. 2023-07-21T22:20:14.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.884+0000 7fdf13af4700 1 Processor -- start 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.884+0000 7fdf13af4700 1 -- start start 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf13af4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf13af4700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1007a0 0x7fdf0c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf13af4700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 0x7fdf0c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf13af4700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c19cf10 con 0x7fdf0c1007a0 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf13af4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c19d080 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf13af4700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c19d240 con 0x7fdf0c104ea0 2023-07-21T22:20:14.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf1108f700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1007a0 0x7fdf0c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf11890700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf11890700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48186/0 (socket says 172.21.15.66:48186) 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.885+0000 7fdf11890700 1 -- 172.21.15.66:0/2009982191 learned_addr learned my addr 172.21.15.66:0/2009982191 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf12091700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 0x7fdf0c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf11890700 1 -- 172.21.15.66:0/2009982191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 msgr2=0x7fdf0c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf11890700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 0x7fdf0c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf11890700 1 -- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1007a0 msgr2=0x7fdf0c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf11890700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1007a0 0x7fdf0c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf11890700 1 -- 172.21.15.66:0/2009982191 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04009d00 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf11890700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c197f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9a0 tx=0x7fdf0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08009790 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c19d520 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.886+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf08010430 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.887+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08018680 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.887+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1a4d80 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.888+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c04e210 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.890+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fdf080105a0 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.892+0000 7fdf02ffd700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdef80a11e0 0x7fdef80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.892+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdf08014070 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.892+0000 7fdf1108f700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdef80a11e0 0x7fdef80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.893+0000 7fdf1108f700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdef80a11e0 0x7fdef80a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c199570 tx=0x7fdefc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:14.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:13.893+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdf08096220 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.122+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdf0c1992b0 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.123+0000 7fdf02ffd700 1 -- 172.21.15.66:0/2009982191 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf08099840 con 0x7fdf0c0ffe00 2023-07-21T22:20:14.167 INFO:teuthology.orchestra.run.smithi066.stdout:167503724597 2023-07-21T22:20:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdef80a11e0 msgr2=0x7fdef80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdef80a11e0 0x7fdef80a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c199570 tx=0x7fdefc009260 comp rx=0 tx=0).stop 2023-07-21T22:20:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 msgr2=0x7fdf0c197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c197f50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800e9a0 tx=0x7fdf0800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 shutdown_connections 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fdef80a11e0 0x7fdef80a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf0c0ffe00 0x7fdf0c197f50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf0c104ea0 0x7fdf0c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 --2- 172.21.15.66:0/2009982191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf0c1007a0 0x7fdf0c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.125+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 >> 172.21.15.66:0/2009982191 conn(0x7fdf0c0fb660 msgr2=0x7fdf0c0fc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.126+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 shutdown_connections 2023-07-21T22:20:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:14.126+0000 7fdf13af4700 1 -- 172.21.15.66:0/2009982191 wait complete. 2023-07-21T22:20:14.368 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4119675788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:20:14.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4119675788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:20:14.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4119675788' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T22:20:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 -- 172.21.15.66:0/637247603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c100df0 msgr2=0x7fb04c1011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 --2- 172.21.15.66:0/637247603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c100df0 0x7fb04c1011c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009990 tx=0x7fb03c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 -- 172.21.15.66:0/637247603 shutdown_connections 2023-07-21T22:20:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 --2- 172.21.15.66:0/637247603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c10a140 0x7fb04c10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 --2- 172.21.15.66:0/637247603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c100df0 0x7fb04c1011c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 --2- 172.21.15.66:0/637247603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 0x7fb04c109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 -- 172.21.15.66:0/637247603 >> 172.21.15.66:0/637247603 conn(0x7fb04c075b70 msgr2=0x7fb04c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.058+0000 7fb05287c700 1 -- 172.21.15.66:0/637247603 shutdown_connections 2023-07-21T22:20:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.059+0000 7fb05287c700 1 -- 172.21.15.66:0/637247603 wait complete. 2023-07-21T22:20:15.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.060+0000 7fb05287c700 1 Processor -- start 2023-07-21T22:20:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 -- start start 2023-07-21T22:20:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c100df0 0x7fb04c1095e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:15.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 0x7fb04c109b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c10a140 0x7fb04c1066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c106eb0 con 0x7fb04c10a140 2023-07-21T22:20:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c107020 con 0x7fb04c101700 2023-07-21T22:20:15.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.061+0000 7fb05287c700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c1071e0 con 0x7fb04c100df0 2023-07-21T22:20:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 0x7fb04c109b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c100df0 0x7fb04c1095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c100df0 0x7fb04c1095e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:40732/0 (socket says 172.21.15.66:40732) 2023-07-21T22:20:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04bfff700 1 -- 172.21.15.66:0/4149587564 learned_addr learned my addr 172.21.15.66:0/4149587564 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb050e19700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c10a140 0x7fb04c1066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:15.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04b7fe700 1 -- 172.21.15.66:0/4149587564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c100df0 msgr2=0x7fb04c1095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04b7fe700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c100df0 0x7fb04c1095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04b7fe700 1 -- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c10a140 msgr2=0x7fb04c1066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04b7fe700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c10a140 0x7fb04c1066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.062+0000 7fb04b7fe700 1 -- 172.21.15.66:0/4149587564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0095f0 con 0x7fb04c101700 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.063+0000 7fb04b7fe700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 0x7fb04c109b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb04000e9f0 tx=0x7fb04000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.063+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0400096d0 con 0x7fb04c101700 2023-07-21T22:20:15.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.063+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04000cbd0 con 0x7fb04c101700 2023-07-21T22:20:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.063+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040018680 con 0x7fb04c101700 2023-07-21T22:20:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.063+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c1a8c50 con 0x7fb04c101700 2023-07-21T22:20:15.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.063+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c1a8fb0 con 0x7fb04c101700 2023-07-21T22:20:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.065+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c04e210 con 0x7fb04c101700 2023-07-21T22:20:15.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.066+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb04000cd40 con 0x7fb04c101700 2023-07-21T22:20:15.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.068+0000 7fb0497fa700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0340a11e0 0x7fb0340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:15.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.068+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb040014070 con 0x7fb04c101700 2023-07-21T22:20:15.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.068+0000 7fb04bfff700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0340a11e0 0x7fb0340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:15.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.069+0000 7fb04bfff700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0340a11e0 0x7fb0340a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006010 tx=0x7fb03c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:15.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.072+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb040051770 con 0x7fb04c101700 2023-07-21T22:20:15.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.290+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fb04c107b30 con 0x7fb04c101700 2023-07-21T22:20:15.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.291+0000 7fb0497fa700 1 -- 172.21.15.66:0/4149587564 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0400d0030 con 0x7fb04c101700 2023-07-21T22:20:15.335 INFO:teuthology.orchestra.run.smithi066.stdout:257698037799 2023-07-21T22:20:15.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0340a11e0 msgr2=0x7fb0340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:15.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0340a11e0 0x7fb0340a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb03c006010 tx=0x7fb03c00b260 comp rx=0 tx=0).stop 2023-07-21T22:20:15.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 msgr2=0x7fb04c109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 0x7fb04c109b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb04000e9f0 tx=0x7fb04000ed00 comp rx=0 tx=0).stop 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 shutdown_connections 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0340a11e0 0x7fb0340a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb04c10a140 0x7fb04c1066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb04c100df0 0x7fb04c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 --2- 172.21.15.66:0/4149587564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb04c101700 0x7fb04c109b20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:15.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 >> 172.21.15.66:0/4149587564 conn(0x7fb04c075b70 msgr2=0x7fb04c10ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:15.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 shutdown_connections 2023-07-21T22:20:15.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:15.293+0000 7fb05287c700 1 -- 172.21.15.66:0/4149587564 wait complete. 2023-07-21T22:20:15.363 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2009982191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:20:15.363 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:15 smithi066 ceph-mon[126379]: pgmap v310: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:15.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2009982191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:20:15.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:15 smithi104 ceph-mon[129021]: pgmap v310: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2009982191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T22:20:15.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:15 smithi149 ceph-mon[129268]: pgmap v310: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:16.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724595 got 167503724597 for osd.5 2023-07-21T22:20:16.058 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:16.106 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776159 got 292057776161 for osd.9 2023-07-21T22:20:16.106 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:16.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:16 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4149587564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:20:16.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:16 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4149587564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:20:16.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:16 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4149587564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T22:20:16.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 -- 172.21.15.66:0/218403933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa980731c0 msgr2=0x7ffa98110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/218403933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa980731c0 0x7ffa98110670 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009990 tx=0x7ffa8c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 -- 172.21.15.66:0/218403933 shutdown_connections 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/218403933 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa98110bb0 0x7ffa98112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/218403933 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980728b0 0x7ffa98072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/218403933 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa980731c0 0x7ffa98110670 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 -- 172.21.15.66:0/218403933 >> 172.21.15.66:0/218403933 conn(0x7ffa980fb660 msgr2=0x7ffa980fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:16.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 -- 172.21.15.66:0/218403933 shutdown_connections 2023-07-21T22:20:16.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.637+0000 7ffa9c90b700 1 -- 172.21.15.66:0/218403933 wait complete. 2023-07-21T22:20:16.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.638+0000 7ffa9c90b700 1 Processor -- start 2023-07-21T22:20:16.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 -- start start 2023-07-21T22:20:16.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 0x7ffa9819c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980731c0 0x7ffa9819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 0x7ffa981a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa981a1250 con 0x7ffa98110bb0 2023-07-21T22:20:16.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa981a13c0 con 0x7ffa980728b0 2023-07-21T22:20:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.639+0000 7ffa9c90b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa981a1580 con 0x7ffa980731c0 2023-07-21T22:20:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa96d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 0x7ffa981a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 0x7ffa9819c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa96d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 0x7ffa981a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39090/0 (socket says 172.21.15.66:39090) 2023-07-21T22:20:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa96d9d700 1 -- 172.21.15.66:0/1399980690 learned_addr learned my addr 172.21.15.66:0/1399980690 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 0x7ffa9819c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48218/0 (socket says 172.21.15.66:48218) 2023-07-21T22:20:16.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa95d9b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980731c0 0x7ffa9819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 -- 172.21.15.66:0/1399980690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980731c0 msgr2=0x7ffa9819c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980731c0 0x7ffa9819c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 -- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 msgr2=0x7ffa981a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 0x7ffa981a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 -- 172.21.15.66:0/1399980690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0095f0 con 0x7ffa980728b0 2023-07-21T22:20:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa95d9b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980731c0 0x7ffa9819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:16.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa96d9d700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 0x7ffa981a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.640+0000 7ffa9659c700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 0x7ffa9819c290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800e9f0 tx=0x7ffa8800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.641+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8800eee0 con 0x7ffa980728b0 2023-07-21T22:20:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.641+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa981a1860 con 0x7ffa980728b0 2023-07-21T22:20:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.641+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa8800cbd0 con 0x7ffa980728b0 2023-07-21T22:20:16.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.641+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88018680 con 0x7ffa980728b0 2023-07-21T22:20:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.641+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa981a90c0 con 0x7ffa980728b0 2023-07-21T22:20:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.642+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa98103090 con 0x7ffa980728b0 2023-07-21T22:20:16.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.644+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ffa8800cd40 con 0x7ffa980728b0 2023-07-21T22:20:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.645+0000 7ffa877fe700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffa800a11e0 0x7ffa800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.646+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa88014070 con 0x7ffa980728b0 2023-07-21T22:20:16.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.646+0000 7ffa95d9b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffa800a11e0 0x7ffa800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.647+0000 7ffa95d9b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffa800a11e0 0x7ffa800a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:16.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.647+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffa880962b0 con 0x7ffa980728b0 2023-07-21T22:20:16.791 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037799 for osd.8 2023-07-21T22:20:16.792 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:16.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 -- 172.21.15.66:0/3308021438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f40ffe00 msgr2=0x7f20f41001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 --2- 172.21.15.66:0/3308021438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f40ffe00 0x7f20f41001d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f20e40099c0 tx=0x7f20e4009cd0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 -- 172.21.15.66:0/3308021438 shutdown_connections 2023-07-21T22:20:16.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 --2- 172.21.15.66:0/3308021438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f4104ea0 0x7f20f410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 --2- 172.21.15.66:0/3308021438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f40ffe00 0x7f20f41001d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 --2- 172.21.15.66:0/3308021438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f41007a0 0x7f20f4104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 -- 172.21.15.66:0/3308021438 >> 172.21.15.66:0/3308021438 conn(0x7f20f40fb660 msgr2=0x7f20f40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:16.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 -- 172.21.15.66:0/3308021438 shutdown_connections 2023-07-21T22:20:16.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.757+0000 7f20f92fd700 1 -- 172.21.15.66:0/3308021438 wait complete. 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.758+0000 7f20f92fd700 1 Processor -- start 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 -- start start 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f40ffe00 0x7f20f419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 0x7f20f419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4104ea0 0x7f20f4195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f4196140 con 0x7f20f4104ea0 2023-07-21T22:20:16.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f41962b0 con 0x7f20f40ffe00 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.759+0000 7f20f92fd700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f4196470 con 0x7f20f41007a0 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 0x7f20f419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f40ffe00 0x7f20f419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 0x7f20f419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:40758/0 (socket says 172.21.15.66:40758) 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f27fc700 1 -- 172.21.15.66:0/1953335291 learned_addr learned my addr 172.21.15.66:0/1953335291 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f37fe700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4104ea0 0x7f20f4195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 -- 172.21.15.66:0/1953335291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 msgr2=0x7f20f419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 0x7f20f419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 -- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4104ea0 msgr2=0x7f20f4195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4104ea0 0x7f20f4195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 -- 172.21.15.66:0/1953335291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e40095f0 con 0x7f20f40ffe00 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f27fc700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 0x7f20f419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.760+0000 7f20f2ffd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f40ffe00 0x7f20f419ba50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20e40098e0 tx=0x7f20e400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:16.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.761+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e401d070 con 0x7f20f40ffe00 2023-07-21T22:20:16.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.761+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f41966f0 con 0x7f20f40ffe00 2023-07-21T22:20:16.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.761+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e4005050 con 0x7f20f40ffe00 2023-07-21T22:20:16.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.761+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4004970 con 0x7f20f40ffe00 2023-07-21T22:20:16.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.761+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f4072910 con 0x7f20f40ffe00 2023-07-21T22:20:16.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.762+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f404e210 con 0x7f20f40ffe00 2023-07-21T22:20:16.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.764+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f20e4005470 con 0x7f20f40ffe00 2023-07-21T22:20:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.766+0000 7f20dbfff700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f20dc0a11e0 0x7f20dc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.766+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f20e40cda40 con 0x7f20f40ffe00 2023-07-21T22:20:16.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.766+0000 7f20f27fc700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f20dc0a11e0 0x7f20dc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:16.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.767+0000 7f20f27fc700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f20dc0a11e0 0x7f20dc0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f20f40febe0 tx=0x7f20e8014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:16.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.770+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f20e4096d50 con 0x7f20f40ffe00 2023-07-21T22:20:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.874+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7ffa9804e210 con 0x7ffa980728b0 2023-07-21T22:20:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.875+0000 7ffa877fe700 1 -- 172.21.15.66:0/1399980690 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7ffa880998d0 con 0x7ffa980728b0 2023-07-21T22:20:16.920 INFO:teuthology.orchestra.run.smithi066.stdout:343597383706 2023-07-21T22:20:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.878+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffa800a11e0 msgr2=0x7ffa800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.878+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffa800a11e0 0x7ffa800a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c006010 tx=0x7ffa8c00b260 comp rx=0 tx=0).stop 2023-07-21T22:20:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.878+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 msgr2=0x7ffa9819c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.878+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 0x7ffa9819c290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800e9f0 tx=0x7ffa8800ed00 comp rx=0 tx=0).stop 2023-07-21T22:20:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 shutdown_connections 2023-07-21T22:20:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffa800a11e0 0x7ffa800a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa980728b0 0x7ffa9819c290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffa98110bb0 0x7ffa981a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 --2- 172.21.15.66:0/1399980690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa980731c0 0x7ffa9819c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 >> 172.21.15.66:0/1399980690 conn(0x7ffa980fb660 msgr2=0x7ffa980fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:16.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 shutdown_connections 2023-07-21T22:20:16.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:16.879+0000 7ffa9c90b700 1 -- 172.21.15.66:0/1399980690 wait complete. 2023-07-21T22:20:17.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.039+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f20f4072b30 con 0x7f20f40ffe00 2023-07-21T22:20:17.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.039+0000 7f20dbfff700 1 -- 172.21.15.66:0/1953335291 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f20e409a370 con 0x7f20f40ffe00 2023-07-21T22:20:17.084 INFO:teuthology.orchestra.run.smithi066.stdout:313532612637 2023-07-21T22:20:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.043+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f20dc0a11e0 msgr2=0x7f20dc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.043+0000 7f20f92fd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f20dc0a11e0 0x7f20dc0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f20f40febe0 tx=0x7f20e8014040 comp rx=0 tx=0).stop 2023-07-21T22:20:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.043+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f40ffe00 msgr2=0x7f20f419ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.043+0000 7f20f92fd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f40ffe00 0x7f20f419ba50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f20e40098e0 tx=0x7f20e400f690 comp rx=0 tx=0).stop 2023-07-21T22:20:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 shutdown_connections 2023-07-21T22:20:17.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f20dc0a11e0 0x7f20dc0a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20f40ffe00 0x7f20f419ba50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f20f4104ea0 0x7f20f4195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 --2- 172.21.15.66:0/1953335291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20f41007a0 0x7f20f419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 >> 172.21.15.66:0/1953335291 conn(0x7f20f40fb660 msgr2=0x7f20f40fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:17.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 shutdown_connections 2023-07-21T22:20:17.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.044+0000 7f20f92fd700 1 -- 172.21.15.66:0/1953335291 wait complete. 2023-07-21T22:20:17.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 -- 172.21.15.66:0/1564239102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc0ff500 msgr2=0x7effdc10b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 --2- 172.21.15.66:0/1564239102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc0ff500 0x7effdc10b940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7effd400b0a0 tx=0x7effd400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 -- 172.21.15.66:0/1564239102 shutdown_connections 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 --2- 172.21.15.66:0/1564239102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc0ff500 0x7effdc10b940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 --2- 172.21.15.66:0/1564239102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc104b70 0x7effdc104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 --2- 172.21.15.66:0/1564239102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effdc0feb50 0x7effdc0fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 -- 172.21.15.66:0/1564239102 >> 172.21.15.66:0/1564239102 conn(0x7effdc0fa5f0 msgr2=0x7effdc0fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:17.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 -- 172.21.15.66:0/1564239102 shutdown_connections 2023-07-21T22:20:17.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.150+0000 7effe34db700 1 -- 172.21.15.66:0/1564239102 wait complete. 2023-07-21T22:20:17.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.152+0000 7effe34db700 1 Processor -- start 2023-07-21T22:20:17.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.152+0000 7effe34db700 1 -- start start 2023-07-21T22:20:17.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe34db700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 0x7effdc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:17.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe34db700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effdc0ff500 0x7effdc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:17.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe34db700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc104b70 0x7effdc19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:17.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe34db700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc19cf90 con 0x7effdc0ff500 2023-07-21T22:20:17.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe34db700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc19d100 con 0x7effdc104b70 2023-07-21T22:20:17.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe34db700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc19d2c0 con 0x7effdc0feb50 2023-07-21T22:20:17.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe1277700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 0x7effdc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:17.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe1277700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 0x7effdc197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:40768/0 (socket says 172.21.15.66:40768) 2023-07-21T22:20:17.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe1277700 1 -- 172.21.15.66:0/2460090271 learned_addr learned my addr 172.21.15.66:0/2460090271 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe1a78700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc104b70 0x7effdc19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.153+0000 7effe0a76700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effdc0ff500 0x7effdc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1a78700 1 -- 172.21.15.66:0/2460090271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 msgr2=0x7effdc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1a78700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 0x7effdc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1a78700 1 -- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effdc0ff500 msgr2=0x7effdc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1a78700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effdc0ff500 0x7effdc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1a78700 1 -- 172.21.15.66:0/2460090271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8009520 con 0x7effdc104b70 2023-07-21T22:20:17.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1277700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 0x7effdc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:17.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe1a78700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc104b70 0x7effdc19c980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effd4009fd0 tx=0x7effd400bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:17.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd400e040 con 0x7effdc104b70 2023-07-21T22:20:17.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd4009d00 con 0x7effdc104b70 2023-07-21T22:20:17.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effd40094c0 con 0x7effdc104b70 2023-07-21T22:20:17.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.154+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4005040 con 0x7effdc104b70 2023-07-21T22:20:17.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.155+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc1a4c30 con 0x7effdc104b70 2023-07-21T22:20:17.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.157+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc0681a0 con 0x7effdc104b70 2023-07-21T22:20:17.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.158+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7effd401d470 con 0x7effdc104b70 2023-07-21T22:20:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.159+0000 7effd27fc700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7effc80a11e0 0x7effc80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:17.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.159+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7effd40ce450 con 0x7effdc104b70 2023-07-21T22:20:17.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.160+0000 7effe1277700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7effc80a11e0 0x7effc80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:17.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.160+0000 7effe1277700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7effc80a11e0 0x7effc80a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7effd8009fd0 tx=0x7effd8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:17.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.163+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7effd4097760 con 0x7effdc104b70 2023-07-21T22:20:17.322 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:17 smithi066 ceph-mon[126379]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:17.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:17 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1399980690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:20:17.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:17 smithi104 ceph-mon[129021]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:17.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:17 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1399980690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:20:17.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.382+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7effdc1a4dc0 con 0x7effdc104b70 2023-07-21T22:20:17.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.383+0000 7effd27fc700 1 -- 172.21.15.66:0/2460090271 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7effd409ad80 con 0x7effdc104b70 2023-07-21T22:20:17.427 INFO:teuthology.orchestra.run.smithi066.stdout:77309411395 2023-07-21T22:20:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7effc80a11e0 msgr2=0x7effc80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7effc80a11e0 0x7effc80a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7effd8009fd0 tx=0x7effd8009260 comp rx=0 tx=0).stop 2023-07-21T22:20:17.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc104b70 msgr2=0x7effdc19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc104b70 0x7effdc19c980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7effd4009fd0 tx=0x7effd400bce0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 shutdown_connections 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7effc80a11e0 0x7effc80a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7effdc104b70 0x7effdc19c980 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7effdc0feb50 0x7effdc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 --2- 172.21.15.66:0/2460090271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7effdc0ff500 0x7effdc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 >> 172.21.15.66:0/2460090271 conn(0x7effdc0fa5f0 msgr2=0x7effdc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:17.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 shutdown_connections 2023-07-21T22:20:17.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:17.385+0000 7effe34db700 1 -- 172.21.15.66:0/2460090271 wait complete. 2023-07-21T22:20:17.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:17 smithi149 ceph-mon[129268]: pgmap v311: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:17.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:17 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1399980690' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T22:20:17.623 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612634 got 313532612637 for osd.10 2023-07-21T22:20:17.623 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:17.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383705 got 343597383706 for osd.11 2023-07-21T22:20:17.974 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:18.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:18 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1953335291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:20:18.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:18 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2460090271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:20:18.306 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411395 for osd.2 2023-07-21T22:20:18.307 DEBUG:teuthology.parallel:result is None 2023-07-21T22:20:18.307 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-21T22:20:18.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T22:20:18.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:18 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1953335291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:20:18.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:18 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2460090271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:20:18.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:18 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1953335291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T22:20:18.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:18 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2460090271' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T22:20:19.066 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:19.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:19 smithi104 ceph-mon[129021]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:19.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:19.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:19 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:19.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:19 smithi066 ceph-mon[126379]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:19.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:19.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:19 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:19.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:19 smithi149 ceph-mon[129268]: pgmap v312: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:19.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:19.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:19 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:20.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:20 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:20:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:20 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:20:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:20 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:20.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:20 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:20:20.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:20 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:20:20.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:20 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:20.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:20 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:20:20.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:20 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:20:20.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:20 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' 2023-07-21T22:20:20.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.661+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/1649459691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54104ea0 msgr2=0x7f9b5410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:20.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.661+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/1649459691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54104ea0 0x7f9b5410ca20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c00b0d0 tx=0x7f9b4c00b3e0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/1649459691 shutdown_connections 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/1649459691 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b54104ea0 0x7f9b5410ca20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/1649459691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b541001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/1649459691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b541007a0 0x7f9b54104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/1649459691 >> 172.21.15.66:0/1649459691 conn(0x7f9b540fb660 msgr2=0x7f9b540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/1649459691 shutdown_connections 2023-07-21T22:20:20.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.662+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/1649459691 wait complete. 2023-07-21T22:20:20.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.663+0000 7f9b5b9ca700 1 Processor -- start 2023-07-21T22:20:20.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.663+0000 7f9b5b9ca700 1 -- start start 2023-07-21T22:20:20.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b5b9ca700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b54195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:20.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b5b9ca700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 0x7f9b54196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:20.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b5b9ca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b54104ea0 0x7f9b5419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:20.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b5b9ca700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5419ad10 con 0x7f9b541007a0 2023-07-21T22:20:20.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b5b9ca700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5419ae80 con 0x7f9b54104ea0 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b5b9ca700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5419b040 con 0x7f9b540ffe00 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 0x7f9b54196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b59766700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b54195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b59766700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b54195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:40794/0 (socket says 172.21.15.66:40794) 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 0x7f9b54196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39154/0 (socket says 172.21.15.66:39154) 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b59766700 1 -- 172.21.15.66:0/3154928169 learned_addr learned my addr 172.21.15.66:0/3154928169 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b59f67700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b54104ea0 0x7f9b5419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:20.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 -- 172.21.15.66:0/3154928169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 msgr2=0x7f9b54195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:20.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b54195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 -- 172.21.15.66:0/3154928169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b54104ea0 msgr2=0x7f9b5419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:20.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b54104ea0 0x7f9b5419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b58f65700 1 -- 172.21.15.66:0/3154928169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c009d00 con 0x7f9b541007a0 2023-07-21T22:20:20.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.664+0000 7f9b59766700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b54195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:20.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.665+0000 7f9b58f65700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 0x7f9b54196290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9c0 tx=0x7f9b4400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:20.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.665+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4400c9c0 con 0x7f9b541007a0 2023-07-21T22:20:20.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.665+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5419b320 con 0x7f9b541007a0 2023-07-21T22:20:20.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.665+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b44010430 con 0x7f9b541007a0 2023-07-21T22:20:20.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.665+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b44018680 con 0x7f9b541007a0 2023-07-21T22:20:20.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.665+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b541a2b80 con 0x7f9b541007a0 2023-07-21T22:20:20.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.666+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5404e210 con 0x7f9b541007a0 2023-07-21T22:20:20.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.668+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9b4400cb20 con 0x7f9b541007a0 2023-07-21T22:20:20.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.670+0000 7f9b4a7fc700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b400a11e0 0x7f9b400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:20.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.670+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9b44014070 con 0x7f9b541007a0 2023-07-21T22:20:20.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.670+0000 7f9b59766700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b400a11e0 0x7f9b400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:20.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.671+0000 7f9b59766700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b400a11e0 0x7f9b400a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b50011550 tx=0x7f9b50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:20.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.672+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b440963c0 con 0x7f9b541007a0 2023-07-21T22:20:20.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.874+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b540fc780 con 0x7f9b400a11e0 2023-07-21T22:20:20.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.888+0000 7f9b4a7fc700 1 -- 172.21.15.66:0/3154928169 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426070 (secure 0 0 0) 0x7f9b540fc780 con 0x7f9b400a11e0 2023-07-21T22:20:20.938 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.894+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b400a11e0 msgr2=0x7f9b400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.894+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b400a11e0 0x7f9b400a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9b50011550 tx=0x7f9b50009260 comp rx=0 tx=0).stop 2023-07-21T22:20:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 msgr2=0x7f9b54196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:20.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 0x7f9b54196290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400e9c0 tx=0x7f9b4400ecd0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 shutdown_connections 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9b400a11e0 0x7f9b400a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b54104ea0 0x7f9b5419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9b540ffe00 0x7f9b54195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 --2- 172.21.15.66:0/3154928169 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9b541007a0 0x7f9b54196290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 >> 172.21.15.66:0/3154928169 conn(0x7f9b540fb660 msgr2=0x7f9b540fc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 shutdown_connections 2023-07-21T22:20:20.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:20.895+0000 7f9b5b9ca700 1 -- 172.21.15.66:0/3154928169 wait complete. 2023-07-21T22:20:20.943 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:20:21.353 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:21 smithi066 ceph-mon[126379]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:21.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:21 smithi104 ceph-mon[129021]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:21.374 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2023-07-21T22:20:20.195363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653610,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":424,"num_read_kb":331,"num_write":338,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335528,"kb_used_data":12104,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606296,"statfs":{"total":1151940427776,"available":1151596847104,"internally_reserved":0,"allocated":12394496,"data_stored":7894412,"data_compressed":318015,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003079"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309536+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:19:37.309536+0000","last_peered":"2023-07-21T22:19:37.309536+0000","last_clean":"2023-07-21T22:19:37.309536+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:19:37.309536+0000","last_undegraded":"2023-07-21T22:19:37.309536+0000","last_fullsized":"2023-07-21T22:19:37.309536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:13:18.888667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308674+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:19:37.308674+0000","last_peered":"2023-07-21T22:19:37.308674+0000","last_clean":"2023-07-21T22:19:37.308674+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:19:37.308674+0000","last_undegraded":"2023-07-21T22:19:37.308674+0000","last_fullsized":"2023-07-21T22:19:37.308674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:37:53.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309364+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:19:37.309364+0000","last_peered":"2023-07-21T22:19:37.309364+0000","last_clean":"2023-07-21T22:19:37.309364+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:19:37.309364+0000","last_undegraded":"2023-07-21T22:19:37.309364+0000","last_fullsized":"2023-07-21T22:19:37.309364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:18.347939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.325254+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:19:37.325254+0000","last_peered":"2023-07-21T22:19:37.325254+0000","last_clean":"2023-07-21T22:19:37.325254+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:19:37.325254+0000","last_undegraded":"2023-07-21T22:19:37.325254+0000","last_fullsized":"2023-07-21T22:19:37.325254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:13:54.817283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309706+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:19:37.309706+0000","last_peered":"2023-07-21T22:19:37.309706+0000","last_clean":"2023-07-21T22:19:37.309706+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:19:37.309706+0000","last_undegraded":"2023-07-21T22:19:37.309706+0000","last_fullsized":"2023-07-21T22:19:37.309706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:19.789707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309427+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:19:37.309427+0000","last_peered":"2023-07-21T22:19:37.309427+0000","last_clean":"2023-07-21T22:19:37.309427+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:19:37.309427+0000","last_undegraded":"2023-07-21T22:19:37.309427+0000","last_fullsized":"2023-07-21T22:19:37.309427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:08.453027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403115+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:19:37.403115+0000","last_peered":"2023-07-21T22:19:37.403115+0000","last_clean":"2023-07-21T22:19:37.403115+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:19:37.403115+0000","last_undegraded":"2023-07-21T22:19:37.403115+0000","last_fullsized":"2023-07-21T22:19:37.403115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:39:46.189678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308057+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:19:37.308057+0000","last_peered":"2023-07-21T22:19:37.308057+0000","last_clean":"2023-07-21T22:19:37.308057+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:19:37.308057+0000","last_undegraded":"2023-07-21T22:19:37.308057+0000","last_fullsized":"2023-07-21T22:19:37.308057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:28.929545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310974+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:19:37.310974+0000","last_peered":"2023-07-21T22:19:37.310974+0000","last_clean":"2023-07-21T22:19:37.310974+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:19:37.310974+0000","last_undegraded":"2023-07-21T22:19:37.310974+0000","last_fullsized":"2023-07-21T22:19:37.310974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:05.850547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310192+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:19:37.310192+0000","last_peered":"2023-07-21T22:19:37.310192+0000","last_clean":"2023-07-21T22:19:37.310192+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:19:37.310192+0000","last_undegraded":"2023-07-21T22:19:37.310192+0000","last_fullsized":"2023-07-21T22:19:37.310192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:21:52.428612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403678+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:19:37.403678+0000","last_peered":"2023-07-21T22:19:37.403678+0000","last_clean":"2023-07-21T22:19:37.403678+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:19:37.403678+0000","last_undegraded":"2023-07-21T22:19:37.403678+0000","last_fullsized":"2023-07-21T22:19:37.403678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:26:59.219489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324807+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:19:37.324807+0000","last_peered":"2023-07-21T22:19:37.324807+0000","last_clean":"2023-07-21T22:19:37.324807+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:19:37.324807+0000","last_undegraded":"2023-07-21T22:19:37.324807+0000","last_fullsized":"2023-07-21T22:19:37.324807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:55:15.589116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313656+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:19:37.313656+0000","last_peered":"2023-07-21T22:19:37.313656+0000","last_clean":"2023-07-21T22:19:37.313656+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:19:37.313656+0000","last_undegraded":"2023-07-21T22:19:37.313656+0000","last_fullsized":"2023-07-21T22:19:37.313656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:52:03.698646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308731+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:19:37.308731+0000","last_peered":"2023-07-21T22:19:37.308731+0000","last_clean":"2023-07-21T22:19:37.308731+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:19:37.308731+0000","last_undegraded":"2023-07-21T22:19:37.308731+0000","last_fullsized":"2023-07-21T22:19:37.308731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:12:54.936169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308470+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:19:37.308470+0000","last_peered":"2023-07-21T22:19:37.308470+0000","last_clean":"2023-07-21T22:19:37.308470+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:19:37.308470+0000","last_undegraded":"2023-07-21T22:19:37.308470+0000","last_fullsized":"2023-07-21T22:19:37.308470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:59:54.865549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309872+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:19:37.309872+0000","last_peered":"2023-07-21T22:19:37.309872+0000","last_clean":"2023-07-21T22:19:37.309872+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:19:37.309872+0000","last_undegraded":"2023-07-21T22:19:37.309872+0000","last_fullsized":"2023-07-21T22:19:37.309872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:46:06.325343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309243+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:19:37.309243+0000","last_peered":"2023-07-21T22:19:37.309243+0000","last_clean":"2023-07-21T22:19:37.309243+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:19:37.309243+0000","last_undegraded":"2023-07-21T22:19:37.309243+0000","last_fullsized":"2023-07-21T22:19:37.309243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:07:42.396016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.305114+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:19:37.305114+0000","last_peered":"2023-07-21T22:19:37.305114+0000","last_clean":"2023-07-21T22:19:37.305114+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:19:37.305114+0000","last_undegraded":"2023-07-21T22:19:37.305114+0000","last_fullsized":"2023-07-21T22:19:37.305114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:36.399061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307143+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:19:37.307143+0000","last_peered":"2023-07-21T22:19:37.307143+0000","last_clean":"2023-07-21T22:19:37.307143+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:19:37.307143+0000","last_undegraded":"2023-07-21T22:19:37.307143+0000","last_fullsized":"2023-07-21T22:19:37.307143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:04:06.149628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306699+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:19:37.306699+0000","last_peered":"2023-07-21T22:19:37.306699+0000","last_clean":"2023-07-21T22:19:37.306699+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:19:37.306699+0000","last_undegraded":"2023-07-21T22:19:37.306699+0000","last_fullsized":"2023-07-21T22:19:37.306699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:41.903328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.305356+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:19:37.305356+0000","last_peered":"2023-07-21T22:19:37.305356+0000","last_clean":"2023-07-21T22:19:37.305356+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:19:37.305356+0000","last_undegraded":"2023-07-21T22:19:37.305356+0000","last_fullsized":"2023-07-21T22:19:37.305356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:41.795680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309580+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:19:37.309580+0000","last_peered":"2023-07-21T22:19:37.309580+0000","last_clean":"2023-07-21T22:19:37.309580+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:19:37.309580+0000","last_undegraded":"2023-07-21T22:19:37.309580+0000","last_fullsized":"2023-07-21T22:19:37.309580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:02.598429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324376+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:19:37.324376+0000","last_peered":"2023-07-21T22:19:37.324376+0000","last_clean":"2023-07-21T22:19:37.324376+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:19:37.324376+0000","last_undegraded":"2023-07-21T22:19:37.324376+0000","last_fullsized":"2023-07-21T22:19:37.324376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:43:39.249253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310460+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:19:37.310460+0000","last_peered":"2023-07-21T22:19:37.310460+0000","last_clean":"2023-07-21T22:19:37.310460+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:19:37.310460+0000","last_undegraded":"2023-07-21T22:19:37.310460+0000","last_fullsized":"2023-07-21T22:19:37.310460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:15.028762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309029+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:19:37.309029+0000","last_peered":"2023-07-21T22:19:37.309029+0000","last_clean":"2023-07-21T22:19:37.309029+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:19:37.309029+0000","last_undegraded":"2023-07-21T22:19:37.309029+0000","last_fullsized":"2023-07-21T22:19:37.309029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:53.913101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324451+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:19:37.324451+0000","last_peered":"2023-07-21T22:19:37.324451+0000","last_clean":"2023-07-21T22:19:37.324451+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:19:37.324451+0000","last_undegraded":"2023-07-21T22:19:37.324451+0000","last_fullsized":"2023-07-21T22:19:37.324451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:53:18.343038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311026+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:19:37.311026+0000","last_peered":"2023-07-21T22:19:37.311026+0000","last_clean":"2023-07-21T22:19:37.311026+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:19:37.311026+0000","last_undegraded":"2023-07-21T22:19:37.311026+0000","last_fullsized":"2023-07-21T22:19:37.311026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:20.544833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311511+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:19:37.311511+0000","last_peered":"2023-07-21T22:19:37.311511+0000","last_clean":"2023-07-21T22:19:37.311511+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:19:37.311511+0000","last_undegraded":"2023-07-21T22:19:37.311511+0000","last_fullsized":"2023-07-21T22:19:37.311511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:10:47.470299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324884+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:19:37.324884+0000","last_peered":"2023-07-21T22:19:37.324884+0000","last_clean":"2023-07-21T22:19:37.324884+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:19:37.324884+0000","last_undegraded":"2023-07-21T22:19:37.324884+0000","last_fullsized":"2023-07-21T22:19:37.324884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:04:52.393245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310052+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:19:37.310052+0000","last_peered":"2023-07-21T22:19:37.310052+0000","last_clean":"2023-07-21T22:19:37.310052+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:19:37.310052+0000","last_undegraded":"2023-07-21T22:19:37.310052+0000","last_fullsized":"2023-07-21T22:19:37.310052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:55.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318598+0000","last_change":"2023-07-21T22:19:07.373299+0000","last_active":"2023-07-21T22:19:37.318598+0000","last_peered":"2023-07-21T22:19:37.318598+0000","last_clean":"2023-07-21T22:19:37.318598+0000","last_became_active":"2023-07-21T22:19:07.372944+0000","last_became_peered":"2023-07-21T22:19:07.372944+0000","last_unstale":"2023-07-21T22:19:37.318598+0000","last_undegraded":"2023-07-21T22:19:37.318598+0000","last_fullsized":"2023-07-21T22:19:37.318598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:47.140681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.404292+0000","last_change":"2023-07-21T22:19:37.404292+0000","last_active":"2023-07-21T22:19:37.404292+0000","last_peered":"2023-07-21T22:19:37.404292+0000","last_clean":"2023-07-21T22:19:37.404292+0000","last_became_active":"2023-07-21T22:19:37.403826+0000","last_became_peered":"2023-07-21T22:19:37.403826+0000","last_unstale":"2023-07-21T22:19:37.404292+0000","last_undegraded":"2023-07-21T22:19:37.404292+0000","last_fullsized":"2023-07-21T22:19:37.404292+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:43.508394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:15.377301+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:20:15.377301+0000","last_peered":"2023-07-21T22:20:15.377301+0000","last_clean":"2023-07-21T22:20:15.377301+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:20:15.377301+0000","last_undegraded":"2023-07-21T22:20:15.377301+0000","last_fullsized":"2023-07-21T22:20:15.377301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:26.565797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310268+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:19:37.310268+0000","last_peered":"2023-07-21T22:19:37.310268+0000","last_clean":"2023-07-21T22:19:37.310268+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:19:37.310268+0000","last_undegraded":"2023-07-21T22:19:37.310268+0000","last_fullsized":"2023-07-21T22:19:37.310268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:48.148796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306802+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:19:37.306802+0000","last_peered":"2023-07-21T22:19:37.306802+0000","last_clean":"2023-07-21T22:19:37.306802+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:19:37.306802+0000","last_undegraded":"2023-07-21T22:19:37.306802+0000","last_fullsized":"2023-07-21T22:19:37.306802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:36:53.824814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.328942+0000","last_change":"2023-07-21T22:19:37.328942+0000","last_active":"2023-07-21T22:19:37.328942+0000","last_peered":"2023-07-21T22:19:37.328942+0000","last_clean":"2023-07-21T22:19:37.328942+0000","last_became_active":"2023-07-21T22:19:37.325738+0000","last_became_peered":"2023-07-21T22:19:37.325738+0000","last_unstale":"2023-07-21T22:19:37.328942+0000","last_undegraded":"2023-07-21T22:19:37.328942+0000","last_fullsized":"2023-07-21T22:19:37.328942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:49:28.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318449+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:19:37.318449+0000","last_peered":"2023-07-21T22:19:37.318449+0000","last_clean":"2023-07-21T22:19:37.318449+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:19:37.318449+0000","last_undegraded":"2023-07-21T22:19:37.318449+0000","last_fullsized":"2023-07-21T22:19:37.318449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:51.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326119+0000","last_change":"2023-07-21T22:19:37.326119+0000","last_active":"2023-07-21T22:19:37.326119+0000","last_peered":"2023-07-21T22:19:37.326119+0000","last_clean":"2023-07-21T22:19:37.326119+0000","last_became_active":"2023-07-21T22:19:37.325675+0000","last_became_peered":"2023-07-21T22:19:37.325675+0000","last_unstale":"2023-07-21T22:19:37.326119+0000","last_undegraded":"2023-07-21T22:19:37.326119+0000","last_fullsized":"2023-07-21T22:19:37.326119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:41:45.097614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.312874+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:19:37.312874+0000","last_peered":"2023-07-21T22:19:37.312874+0000","last_clean":"2023-07-21T22:19:37.312874+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:19:37.312874+0000","last_undegraded":"2023-07-21T22:19:37.312874+0000","last_fullsized":"2023-07-21T22:19:37.312874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:23:16.323346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307917+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:19:37.307917+0000","last_peered":"2023-07-21T22:19:37.307917+0000","last_clean":"2023-07-21T22:19:37.307917+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:19:37.307917+0000","last_undegraded":"2023-07-21T22:19:37.307917+0000","last_fullsized":"2023-07-21T22:19:37.307917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:50:00.787704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308525+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:19:37.308525+0000","last_peered":"2023-07-21T22:19:37.308525+0000","last_clean":"2023-07-21T22:19:37.308525+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:19:37.308525+0000","last_undegraded":"2023-07-21T22:19:37.308525+0000","last_fullsized":"2023-07-21T22:19:37.308525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:19.595185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317845+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:19:37.317845+0000","last_peered":"2023-07-21T22:19:37.317845+0000","last_clean":"2023-07-21T22:19:37.317845+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:19:37.317845+0000","last_undegraded":"2023-07-21T22:19:37.317845+0000","last_fullsized":"2023-07-21T22:19:37.317845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:51:14.936344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.304910+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:19:37.304910+0000","last_peered":"2023-07-21T22:19:37.304910+0000","last_clean":"2023-07-21T22:19:37.304910+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:19:37.304910+0000","last_undegraded":"2023-07-21T22:19:37.304910+0000","last_fullsized":"2023-07-21T22:19:37.304910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:34.474827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308197+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:19:37.308197+0000","last_peered":"2023-07-21T22:19:37.308197+0000","last_clean":"2023-07-21T22:19:37.308197+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:19:37.308197+0000","last_undegraded":"2023-07-21T22:19:37.308197+0000","last_fullsized":"2023-07-21T22:19:37.308197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:28:20.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:16.378209+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:20:16.378209+0000","last_peered":"2023-07-21T22:20:16.378209+0000","last_clean":"2023-07-21T22:20:16.378209+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:20:16.378209+0000","last_undegraded":"2023-07-21T22:20:16.378209+0000","last_fullsized":"2023-07-21T22:20:16.378209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:26:25.437396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311461+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:19:37.311461+0000","last_peered":"2023-07-21T22:19:37.311461+0000","last_clean":"2023-07-21T22:19:37.311461+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:19:37.311461+0000","last_undegraded":"2023-07-21T22:19:37.311461+0000","last_fullsized":"2023-07-21T22:19:37.311461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:04:20.100122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326929+0000","last_change":"2023-07-21T22:19:37.326317+0000","last_active":"2023-07-21T22:19:37.326929+0000","last_peered":"2023-07-21T22:19:37.326929+0000","last_clean":"2023-07-21T22:19:37.326929+0000","last_became_active":"2023-07-21T22:19:37.325791+0000","last_became_peered":"2023-07-21T22:19:37.325791+0000","last_unstale":"2023-07-21T22:19:37.326929+0000","last_undegraded":"2023-07-21T22:19:37.326929+0000","last_fullsized":"2023-07-21T22:19:37.326929+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:11:24.661772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403643+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:19:37.403643+0000","last_peered":"2023-07-21T22:19:37.403643+0000","last_clean":"2023-07-21T22:19:37.403643+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:19:37.403643+0000","last_undegraded":"2023-07-21T22:19:37.403643+0000","last_fullsized":"2023-07-21T22:19:37.403643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:39:53.409122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309161+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:19:37.309161+0000","last_peered":"2023-07-21T22:19:37.309161+0000","last_clean":"2023-07-21T22:19:37.309161+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:19:37.309161+0000","last_undegraded":"2023-07-21T22:19:37.309161+0000","last_fullsized":"2023-07-21T22:19:37.309161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:19.128046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324638+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:19:37.324638+0000","last_peered":"2023-07-21T22:19:37.324638+0000","last_clean":"2023-07-21T22:19:37.324638+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:19:37.324638+0000","last_undegraded":"2023-07-21T22:19:37.324638+0000","last_fullsized":"2023-07-21T22:19:37.324638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:41.858871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318440+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:19:37.318440+0000","last_peered":"2023-07-21T22:19:37.318440+0000","last_clean":"2023-07-21T22:19:37.318440+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:19:37.318440+0000","last_undegraded":"2023-07-21T22:19:37.318440+0000","last_fullsized":"2023-07-21T22:19:37.318440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:18:31.785229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311593+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:19:37.311593+0000","last_peered":"2023-07-21T22:19:37.311593+0000","last_clean":"2023-07-21T22:19:37.311593+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:19:37.311593+0000","last_undegraded":"2023-07-21T22:19:37.311593+0000","last_fullsized":"2023-07-21T22:19:37.311593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:07:46.707479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.304736+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:19:37.304736+0000","last_peered":"2023-07-21T22:19:37.304736+0000","last_clean":"2023-07-21T22:19:37.304736+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:19:37.304736+0000","last_undegraded":"2023-07-21T22:19:37.304736+0000","last_fullsized":"2023-07-21T22:19:37.304736+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:20:49.844080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'30","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:17.380124+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:20:17.380124+0000","last_peered":"2023-07-21T22:20:17.380124+0000","last_clean":"2023-07-21T22:20:17.380124+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:20:17.380124+0000","last_undegraded":"2023-07-21T22:20:17.380124+0000","last_fullsized":"2023-07-21T22:20:17.380124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:08.163210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58702,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":95,"num_read_kb":104,"num_write":31,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311432+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:19:37.311432+0000","last_peered":"2023-07-21T22:19:37.311432+0000","last_clean":"2023-07-21T22:19:37.311432+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:19:37.311432+0000","last_undegraded":"2023-07-21T22:19:37.311432+0000","last_fullsized":"2023-07-21T22:19:37.311432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:01:58.537492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318926+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:19:37.318926+0000","last_peered":"2023-07-21T22:19:37.318926+0000","last_clean":"2023-07-21T22:19:37.318926+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:19:37.318926+0000","last_undegraded":"2023-07-21T22:19:37.318926+0000","last_fullsized":"2023-07-21T22:19:37.318926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:35:25.618831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306879+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:19:37.306879+0000","last_peered":"2023-07-21T22:19:37.306879+0000","last_clean":"2023-07-21T22:19:37.306879+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:19:37.306879+0000","last_undegraded":"2023-07-21T22:19:37.306879+0000","last_fullsized":"2023-07-21T22:19:37.306879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:20:30.957940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313415+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:19:37.313415+0000","last_peered":"2023-07-21T22:19:37.313415+0000","last_clean":"2023-07-21T22:19:37.313415+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:19:37.313415+0000","last_undegraded":"2023-07-21T22:19:37.313415+0000","last_fullsized":"2023-07-21T22:19:37.313415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:58:25.029301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308492+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:19:37.308492+0000","last_peered":"2023-07-21T22:19:37.308492+0000","last_clean":"2023-07-21T22:19:37.308492+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:19:37.308492+0000","last_undegraded":"2023-07-21T22:19:37.308492+0000","last_fullsized":"2023-07-21T22:19:37.308492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:59.600990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318303+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:19:37.318303+0000","last_peered":"2023-07-21T22:19:37.318303+0000","last_clean":"2023-07-21T22:19:37.318303+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:19:37.318303+0000","last_undegraded":"2023-07-21T22:19:37.318303+0000","last_fullsized":"2023-07-21T22:19:37.318303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:54:01.950928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403282+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:19:37.403282+0000","last_peered":"2023-07-21T22:19:37.403282+0000","last_clean":"2023-07-21T22:19:37.403282+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:19:37.403282+0000","last_undegraded":"2023-07-21T22:19:37.403282+0000","last_fullsized":"2023-07-21T22:19:37.403282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:07.126739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.404187+0000","last_change":"2023-07-21T22:19:37.403648+0000","last_active":"2023-07-21T22:19:37.404187+0000","last_peered":"2023-07-21T22:19:37.404187+0000","last_clean":"2023-07-21T22:19:37.404187+0000","last_became_active":"2023-07-21T22:19:37.403252+0000","last_became_peered":"2023-07-21T22:19:37.403252+0000","last_unstale":"2023-07-21T22:19:37.404187+0000","last_undegraded":"2023-07-21T22:19:37.404187+0000","last_fullsized":"2023-07-21T22:19:37.404187+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:09:39.202554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":75,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:14.378779+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:20:14.378779+0000","last_peered":"2023-07-21T22:20:14.378779+0000","last_clean":"2023-07-21T22:20:14.378779+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:20:14.378779+0000","last_undegraded":"2023-07-21T22:20:14.378779+0000","last_fullsized":"2023-07-21T22:20:14.378779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:00:10.807428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403075+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:19:37.403075+0000","last_peered":"2023-07-21T22:19:37.403075+0000","last_clean":"2023-07-21T22:19:37.403075+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:19:37.403075+0000","last_undegraded":"2023-07-21T22:19:37.403075+0000","last_fullsized":"2023-07-21T22:19:37.403075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:54:05.516094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313309+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:19:37.313309+0000","last_peered":"2023-07-21T22:19:37.313309+0000","last_clean":"2023-07-21T22:19:37.313309+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:19:37.313309+0000","last_undegraded":"2023-07-21T22:19:37.313309+0000","last_fullsized":"2023-07-21T22:19:37.313309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:47.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309765+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:19:37.309765+0000","last_peered":"2023-07-21T22:19:37.309765+0000","last_clean":"2023-07-21T22:19:37.309765+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:19:37.309765+0000","last_undegraded":"2023-07-21T22:19:37.309765+0000","last_fullsized":"2023-07-21T22:19:37.309765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:56.223931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318295+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:19:37.318295+0000","last_peered":"2023-07-21T22:19:37.318295+0000","last_clean":"2023-07-21T22:19:37.318295+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:19:37.318295+0000","last_undegraded":"2023-07-21T22:19:37.318295+0000","last_fullsized":"2023-07-21T22:19:37.318295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:27:04.660917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318367+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:19:37.318367+0000","last_peered":"2023-07-21T22:19:37.318367+0000","last_clean":"2023-07-21T22:19:37.318367+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:19:37.318367+0000","last_undegraded":"2023-07-21T22:19:37.318367+0000","last_fullsized":"2023-07-21T22:19:37.318367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:07:25.219481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310611+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:19:37.310611+0000","last_peered":"2023-07-21T22:19:37.310611+0000","last_clean":"2023-07-21T22:19:37.310611+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:19:37.310611+0000","last_undegraded":"2023-07-21T22:19:37.310611+0000","last_fullsized":"2023-07-21T22:19:37.310611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:41:24.004162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309096+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:19:37.309096+0000","last_peered":"2023-07-21T22:19:37.309096+0000","last_clean":"2023-07-21T22:19:37.309096+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:19:37.309096+0000","last_undegraded":"2023-07-21T22:19:37.309096+0000","last_fullsized":"2023-07-21T22:19:37.309096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:32:04.428231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309958+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:19:37.309958+0000","last_peered":"2023-07-21T22:19:37.309958+0000","last_clean":"2023-07-21T22:19:37.309958+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:19:37.309958+0000","last_undegraded":"2023-07-21T22:19:37.309958+0000","last_fullsized":"2023-07-21T22:19:37.309958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:27.009320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.312431+0000","last_change":"2023-07-21T22:19:37.312431+0000","last_active":"2023-07-21T22:19:37.312431+0000","last_peered":"2023-07-21T22:19:37.312431+0000","last_clean":"2023-07-21T22:19:37.312431+0000","last_became_active":"2023-07-21T22:19:37.312038+0000","last_became_peered":"2023-07-21T22:19:37.312038+0000","last_unstale":"2023-07-21T22:19:37.312431+0000","last_undegraded":"2023-07-21T22:19:37.312431+0000","last_fullsized":"2023-07-21T22:19:37.312431+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:40:24.338102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310127+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:19:37.310127+0000","last_peered":"2023-07-21T22:19:37.310127+0000","last_clean":"2023-07-21T22:19:37.310127+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:19:37.310127+0000","last_undegraded":"2023-07-21T22:19:37.310127+0000","last_fullsized":"2023-07-21T22:19:37.310127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:22:26.812058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313198+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:19:37.313198+0000","last_peered":"2023-07-21T22:19:37.313198+0000","last_clean":"2023-07-21T22:19:37.313198+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:19:37.313198+0000","last_undegraded":"2023-07-21T22:19:37.313198+0000","last_fullsized":"2023-07-21T22:19:37.313198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:26:50.417626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403339+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:19:37.403339+0000","last_peered":"2023-07-21T22:19:37.403339+0000","last_clean":"2023-07-21T22:19:37.403339+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:19:37.403339+0000","last_undegraded":"2023-07-21T22:19:37.403339+0000","last_fullsized":"2023-07-21T22:19:37.403339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:04:04.011358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309432+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:19:37.309432+0000","last_peered":"2023-07-21T22:19:37.309432+0000","last_clean":"2023-07-21T22:19:37.309432+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:19:37.309432+0000","last_undegraded":"2023-07-21T22:19:37.309432+0000","last_fullsized":"2023-07-21T22:19:37.309432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:25:52.984944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313268+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:19:37.313268+0000","last_peered":"2023-07-21T22:19:37.313268+0000","last_clean":"2023-07-21T22:19:37.313268+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:19:37.313268+0000","last_undegraded":"2023-07-21T22:19:37.313268+0000","last_fullsized":"2023-07-21T22:19:37.313268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:45.429732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317999+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:19:37.317999+0000","last_peered":"2023-07-21T22:19:37.317999+0000","last_clean":"2023-07-21T22:19:37.317999+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:19:37.317999+0000","last_undegraded":"2023-07-21T22:19:37.317999+0000","last_fullsized":"2023-07-21T22:19:37.317999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:00:22.072167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:16.381043+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:20:16.381043+0000","last_peered":"2023-07-21T22:20:16.381043+0000","last_clean":"2023-07-21T22:20:16.381043+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:20:16.381043+0000","last_undegraded":"2023-07-21T22:20:16.381043+0000","last_fullsized":"2023-07-21T22:20:16.381043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:40.068303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403576+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:19:37.403576+0000","last_peered":"2023-07-21T22:19:37.403576+0000","last_clean":"2023-07-21T22:19:37.403576+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:19:37.403576+0000","last_undegraded":"2023-07-21T22:19:37.403576+0000","last_fullsized":"2023-07-21T22:19:37.403576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:42:34.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308304+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:19:37.308304+0000","last_peered":"2023-07-21T22:19:37.308304+0000","last_clean":"2023-07-21T22:19:37.308304+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:19:37.308304+0000","last_undegraded":"2023-07-21T22:19:37.308304+0000","last_fullsized":"2023-07-21T22:19:37.308304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:52.893699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324555+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:19:37.324555+0000","last_peered":"2023-07-21T22:19:37.324555+0000","last_clean":"2023-07-21T22:19:37.324555+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:19:37.324555+0000","last_undegraded":"2023-07-21T22:19:37.324555+0000","last_fullsized":"2023-07-21T22:19:37.324555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:27:17.708385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.402915+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:19:37.402915+0000","last_peered":"2023-07-21T22:19:37.402915+0000","last_clean":"2023-07-21T22:19:37.402915+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:19:37.402915+0000","last_undegraded":"2023-07-21T22:19:37.402915+0000","last_fullsized":"2023-07-21T22:19:37.402915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:03:56.744877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318021+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:19:37.318021+0000","last_peered":"2023-07-21T22:19:37.318021+0000","last_clean":"2023-07-21T22:19:37.318021+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:19:37.318021+0000","last_undegraded":"2023-07-21T22:19:37.318021+0000","last_fullsized":"2023-07-21T22:19:37.318021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:45.952550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.312718+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:19:37.312718+0000","last_peered":"2023-07-21T22:19:37.312718+0000","last_clean":"2023-07-21T22:19:37.312718+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:19:37.312718+0000","last_undegraded":"2023-07-21T22:19:37.312718+0000","last_fullsized":"2023-07-21T22:19:37.312718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:26:49.849442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.402997+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:19:37.402997+0000","last_peered":"2023-07-21T22:19:37.402997+0000","last_clean":"2023-07-21T22:19:37.402997+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:19:37.402997+0000","last_undegraded":"2023-07-21T22:19:37.402997+0000","last_fullsized":"2023-07-21T22:19:37.402997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:58:20.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308304+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:19:37.308304+0000","last_peered":"2023-07-21T22:19:37.308304+0000","last_clean":"2023-07-21T22:19:37.308304+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:19:37.308304+0000","last_undegraded":"2023-07-21T22:19:37.308304+0000","last_fullsized":"2023-07-21T22:19:37.308304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:40.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311330+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:19:37.311330+0000","last_peered":"2023-07-21T22:19:37.311330+0000","last_clean":"2023-07-21T22:19:37.311330+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:19:37.311330+0000","last_undegraded":"2023-07-21T22:19:37.311330+0000","last_fullsized":"2023-07-21T22:19:37.311330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:29:36.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313447+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:19:37.313447+0000","last_peered":"2023-07-21T22:19:37.313447+0000","last_clean":"2023-07-21T22:19:37.313447+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:19:37.313447+0000","last_undegraded":"2023-07-21T22:19:37.313447+0000","last_fullsized":"2023-07-21T22:19:37.313447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:55.808390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307908+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:19:37.307908+0000","last_peered":"2023-07-21T22:19:37.307908+0000","last_clean":"2023-07-21T22:19:37.307908+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:19:37.307908+0000","last_undegraded":"2023-07-21T22:19:37.307908+0000","last_fullsized":"2023-07-21T22:19:37.307908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:46.255453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:40.780207+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:19:40.780207+0000","last_peered":"2023-07-21T22:19:40.780207+0000","last_clean":"2023-07-21T22:19:40.780207+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:19:40.780207+0000","last_undegraded":"2023-07-21T22:19:40.780207+0000","last_fullsized":"2023-07-21T22:19:40.780207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:21:27.644738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309258+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:19:37.309258+0000","last_peered":"2023-07-21T22:19:37.309258+0000","last_clean":"2023-07-21T22:19:37.309258+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:19:37.309258+0000","last_undegraded":"2023-07-21T22:19:37.309258+0000","last_fullsized":"2023-07-21T22:19:37.309258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:49:50.061521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306844+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:19:37.306844+0000","last_peered":"2023-07-21T22:19:37.306844+0000","last_clean":"2023-07-21T22:19:37.306844+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:19:37.306844+0000","last_undegraded":"2023-07-21T22:19:37.306844+0000","last_fullsized":"2023-07-21T22:19:37.306844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:10:56.610067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.325011+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:19:37.325011+0000","last_peered":"2023-07-21T22:19:37.325011+0000","last_clean":"2023-07-21T22:19:37.325011+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:19:37.325011+0000","last_undegraded":"2023-07-21T22:19:37.325011+0000","last_fullsized":"2023-07-21T22:19:37.325011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:40.440014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.404445+0000","last_change":"2023-07-21T22:19:37.404445+0000","last_active":"2023-07-21T22:19:37.404445+0000","last_peered":"2023-07-21T22:19:37.404445+0000","last_clean":"2023-07-21T22:19:37.404445+0000","last_became_active":"2023-07-21T22:19:37.404011+0000","last_became_peered":"2023-07-21T22:19:37.404011+0000","last_unstale":"2023-07-21T22:19:37.404445+0000","last_undegraded":"2023-07-21T22:19:37.404445+0000","last_fullsized":"2023-07-21T22:19:37.404445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:08.840927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324952+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:19:37.324952+0000","last_peered":"2023-07-21T22:19:37.324952+0000","last_clean":"2023-07-21T22:19:37.324952+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:19:37.324952+0000","last_undegraded":"2023-07-21T22:19:37.324952+0000","last_fullsized":"2023-07-21T22:19:37.324952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:29.925386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310235+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:19:37.310235+0000","last_peered":"2023-07-21T22:19:37.310235+0000","last_clean":"2023-07-21T22:19:37.310235+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:19:37.310235+0000","last_undegraded":"2023-07-21T22:19:37.310235+0000","last_fullsized":"2023-07-21T22:19:37.310235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:17:04.984129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309689+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:19:37.309689+0000","last_peered":"2023-07-21T22:19:37.309689+0000","last_clean":"2023-07-21T22:19:37.309689+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:19:37.309689+0000","last_undegraded":"2023-07-21T22:19:37.309689+0000","last_fullsized":"2023-07-21T22:19:37.309689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:55.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.305251+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:19:37.305251+0000","last_peered":"2023-07-21T22:19:37.305251+0000","last_clean":"2023-07-21T22:19:37.305251+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:19:37.305251+0000","last_undegraded":"2023-07-21T22:19:37.305251+0000","last_fullsized":"2023-07-21T22:19:37.305251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:03:31.356395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308327+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:19:37.308327+0000","last_peered":"2023-07-21T22:19:37.308327+0000","last_clean":"2023-07-21T22:19:37.308327+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:19:37.308327+0000","last_undegraded":"2023-07-21T22:19:37.308327+0000","last_fullsized":"2023-07-21T22:19:37.308327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:55.175149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308117+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:19:37.308117+0000","last_peered":"2023-07-21T22:19:37.308117+0000","last_clean":"2023-07-21T22:19:37.308117+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:19:37.308117+0000","last_undegraded":"2023-07-21T22:19:37.308117+0000","last_fullsized":"2023-07-21T22:19:37.308117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:46:44.426120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324709+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:19:37.324709+0000","last_peered":"2023-07-21T22:19:37.324709+0000","last_clean":"2023-07-21T22:19:37.324709+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:19:37.324709+0000","last_undegraded":"2023-07-21T22:19:37.324709+0000","last_fullsized":"2023-07-21T22:19:37.324709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:33:46.778289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.316192+0000","last_change":"2023-07-21T22:19:37.315809+0000","last_active":"2023-07-21T22:19:37.316192+0000","last_peered":"2023-07-21T22:19:37.316192+0000","last_clean":"2023-07-21T22:19:37.316192+0000","last_became_active":"2023-07-21T22:19:37.312331+0000","last_became_peered":"2023-07-21T22:19:37.312331+0000","last_unstale":"2023-07-21T22:19:37.316192+0000","last_undegraded":"2023-07-21T22:19:37.316192+0000","last_fullsized":"2023-07-21T22:19:37.316192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:09.215699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308886+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:19:37.308886+0000","last_peered":"2023-07-21T22:19:37.308886+0000","last_clean":"2023-07-21T22:19:37.308886+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:19:37.308886+0000","last_undegraded":"2023-07-21T22:19:37.308886+0000","last_fullsized":"2023-07-21T22:19:37.308886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:44:20.850151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324857+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:19:37.324857+0000","last_peered":"2023-07-21T22:19:37.324857+0000","last_clean":"2023-07-21T22:19:37.324857+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:19:37.324857+0000","last_undegraded":"2023-07-21T22:19:37.324857+0000","last_fullsized":"2023-07-21T22:19:37.324857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:50:02.988262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309854+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:19:37.309854+0000","last_peered":"2023-07-21T22:19:37.309854+0000","last_clean":"2023-07-21T22:19:37.309854+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:19:37.309854+0000","last_undegraded":"2023-07-21T22:19:37.309854+0000","last_fullsized":"2023-07-21T22:19:37.309854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:03:53.898872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403658+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:19:37.403658+0000","last_peered":"2023-07-21T22:19:37.403658+0000","last_clean":"2023-07-21T22:19:37.403658+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:19:37.403658+0000","last_undegraded":"2023-07-21T22:19:37.403658+0000","last_fullsized":"2023-07-21T22:19:37.403658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:38:43.593685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310346+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:19:37.310346+0000","last_peered":"2023-07-21T22:19:37.310346+0000","last_clean":"2023-07-21T22:19:37.310346+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:19:37.310346+0000","last_undegraded":"2023-07-21T22:19:37.310346+0000","last_fullsized":"2023-07-21T22:19:37.310346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:46.183044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307135+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:19:37.307135+0000","last_peered":"2023-07-21T22:19:37.307135+0000","last_clean":"2023-07-21T22:19:37.307135+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:19:37.307135+0000","last_undegraded":"2023-07-21T22:19:37.307135+0000","last_fullsized":"2023-07-21T22:19:37.307135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:34:58.835266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403865+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:19:37.403865+0000","last_peered":"2023-07-21T22:19:37.403865+0000","last_clean":"2023-07-21T22:19:37.403865+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:19:37.403865+0000","last_undegraded":"2023-07-21T22:19:37.403865+0000","last_fullsized":"2023-07-21T22:19:37.403865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:48:06.009672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308879+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:19:37.308879+0000","last_peered":"2023-07-21T22:19:37.308879+0000","last_clean":"2023-07-21T22:19:37.308879+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:19:37.308879+0000","last_undegraded":"2023-07-21T22:19:37.308879+0000","last_fullsized":"2023-07-21T22:19:37.308879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:27.207972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318725+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:19:37.318725+0000","last_peered":"2023-07-21T22:19:37.318725+0000","last_clean":"2023-07-21T22:19:37.318725+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:19:37.318725+0000","last_undegraded":"2023-07-21T22:19:37.318725+0000","last_fullsized":"2023-07-21T22:19:37.318725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:01:53.588516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403502+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:19:37.403502+0000","last_peered":"2023-07-21T22:19:37.403502+0000","last_clean":"2023-07-21T22:19:37.403502+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:19:37.403502+0000","last_undegraded":"2023-07-21T22:19:37.403502+0000","last_fullsized":"2023-07-21T22:19:37.403502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:28:06.664433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317672+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:19:37.317672+0000","last_peered":"2023-07-21T22:19:37.317672+0000","last_clean":"2023-07-21T22:19:37.317672+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:19:37.317672+0000","last_undegraded":"2023-07-21T22:19:37.317672+0000","last_fullsized":"2023-07-21T22:19:37.317672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:13:04.207733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313421+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:19:37.313421+0000","last_peered":"2023-07-21T22:19:37.313421+0000","last_clean":"2023-07-21T22:19:37.313421+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:19:37.313421+0000","last_undegraded":"2023-07-21T22:19:37.313421+0000","last_fullsized":"2023-07-21T22:19:37.313421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:02:31.808021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403795+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:19:37.403795+0000","last_peered":"2023-07-21T22:19:37.403795+0000","last_clean":"2023-07-21T22:19:37.403795+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:19:37.403795+0000","last_undegraded":"2023-07-21T22:19:37.403795+0000","last_fullsized":"2023-07-21T22:19:37.403795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:30:43.531900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308261+0000","last_change":"2023-07-21T22:19:07.372870+0000","last_active":"2023-07-21T22:19:37.308261+0000","last_peered":"2023-07-21T22:19:37.308261+0000","last_clean":"2023-07-21T22:19:37.308261+0000","last_became_active":"2023-07-21T22:19:07.372558+0000","last_became_peered":"2023-07-21T22:19:07.372558+0000","last_unstale":"2023-07-21T22:19:37.308261+0000","last_undegraded":"2023-07-21T22:19:37.308261+0000","last_fullsized":"2023-07-21T22:19:37.308261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:07:56.834449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326004+0000","last_change":"2023-07-21T22:19:37.326004+0000","last_active":"2023-07-21T22:19:37.326004+0000","last_peered":"2023-07-21T22:19:37.326004+0000","last_clean":"2023-07-21T22:19:37.326004+0000","last_became_active":"2023-07-21T22:19:37.325611+0000","last_became_peered":"2023-07-21T22:19:37.325611+0000","last_unstale":"2023-07-21T22:19:37.326004+0000","last_undegraded":"2023-07-21T22:19:37.326004+0000","last_fullsized":"2023-07-21T22:19:37.326004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:52:15.766198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306222+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:19:37.306222+0000","last_peered":"2023-07-21T22:19:37.306222+0000","last_clean":"2023-07-21T22:19:37.306222+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:19:37.306222+0000","last_undegraded":"2023-07-21T22:19:37.306222+0000","last_fullsized":"2023-07-21T22:19:37.306222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:53.516407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309252+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:19:37.309252+0000","last_peered":"2023-07-21T22:19:37.309252+0000","last_clean":"2023-07-21T22:19:37.309252+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:19:37.309252+0000","last_undegraded":"2023-07-21T22:19:37.309252+0000","last_fullsized":"2023-07-21T22:19:37.309252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:52.701869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313474+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:19:37.313474+0000","last_peered":"2023-07-21T22:19:37.313474+0000","last_clean":"2023-07-21T22:19:37.313474+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:19:37.313474+0000","last_undegraded":"2023-07-21T22:19:37.313474+0000","last_fullsized":"2023-07-21T22:19:37.313474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:10:39.411498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309279+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:19:37.309279+0000","last_peered":"2023-07-21T22:19:37.309279+0000","last_clean":"2023-07-21T22:19:37.309279+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:19:37.309279+0000","last_undegraded":"2023-07-21T22:19:37.309279+0000","last_fullsized":"2023-07-21T22:19:37.309279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:59.307946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306810+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:19:37.306810+0000","last_peered":"2023-07-21T22:19:37.306810+0000","last_clean":"2023-07-21T22:19:37.306810+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:19:37.306810+0000","last_undegraded":"2023-07-21T22:19:37.306810+0000","last_fullsized":"2023-07-21T22:19:37.306810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:47:20.553040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.314152+0000","last_change":"2023-07-21T22:19:37.314152+0000","last_active":"2023-07-21T22:19:37.314152+0000","last_peered":"2023-07-21T22:19:37.314152+0000","last_clean":"2023-07-21T22:19:37.314152+0000","last_became_active":"2023-07-21T22:19:37.313795+0000","last_became_peered":"2023-07-21T22:19:37.313795+0000","last_unstale":"2023-07-21T22:19:37.314152+0000","last_undegraded":"2023-07-21T22:19:37.314152+0000","last_fullsized":"2023-07-21T22:19:37.314152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:57:15.996755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317952+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:19:37.317952+0000","last_peered":"2023-07-21T22:19:37.317952+0000","last_clean":"2023-07-21T22:19:37.317952+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:19:37.317952+0000","last_undegraded":"2023-07-21T22:19:37.317952+0000","last_fullsized":"2023-07-21T22:19:37.317952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:24:02.673526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308478+0000","last_change":"2023-07-21T22:19:07.381609+0000","last_active":"2023-07-21T22:19:37.308478+0000","last_peered":"2023-07-21T22:19:37.308478+0000","last_clean":"2023-07-21T22:19:37.308478+0000","last_became_active":"2023-07-21T22:19:07.381272+0000","last_became_peered":"2023-07-21T22:19:07.381272+0000","last_unstale":"2023-07-21T22:19:37.308478+0000","last_undegraded":"2023-07-21T22:19:37.308478+0000","last_fullsized":"2023-07-21T22:19:37.308478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:00:23.406427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310743+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:19:37.310743+0000","last_peered":"2023-07-21T22:19:37.310743+0000","last_clean":"2023-07-21T22:19:37.310743+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:19:37.310743+0000","last_undegraded":"2023-07-21T22:19:37.310743+0000","last_fullsized":"2023-07-21T22:19:37.310743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:47:16.929327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313825+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:19:37.313825+0000","last_peered":"2023-07-21T22:19:37.313825+0000","last_clean":"2023-07-21T22:19:37.313825+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:19:37.313825+0000","last_undegraded":"2023-07-21T22:19:37.313825+0000","last_fullsized":"2023-07-21T22:19:37.313825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:24.108010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308758+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:19:37.308758+0000","last_peered":"2023-07-21T22:19:37.308758+0000","last_clean":"2023-07-21T22:19:37.308758+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:19:37.308758+0000","last_undegraded":"2023-07-21T22:19:37.308758+0000","last_fullsized":"2023-07-21T22:19:37.308758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:14.212545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62996,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":288,"num_read_kb":212,"num_write":155,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":496474,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.753}]},{"osd":8,"last update":"Fri Jul 21 22:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.85399999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.79400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1134592,"data_stored":1142786,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":17.132000000000001,"5min":17.132000000000001,"15min":17.132000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":17.084,"5min":17.084,"15min":17.084},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.496},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.433,"15min":0.433},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.48099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92200000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67700000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.48,"5min":0.98899999999999999,"15min":0.90700000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":17.163,"5min":17.163,"15min":17.163},"last":0.71899999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.73999999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776162,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":500806,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.59899999999999998,"15min":0.61799999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":1.754},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0920000000000001,"5min":1.244,"15min":1.244},"last":1.7090000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93000000000000005,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.79700000000000004,"15min":0.81000000000000005},"min":{"1min":0.55600000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94399999999999995,"5min":1.171,"15min":1.171},"last":1.0069999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.89400000000000002,"15min":0.92800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0029999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88600000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":1.952}]},{"osd":3,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.78900000000000003,"15min":0.82199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":2.0190000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.85599999999999998,"15min":0.89400000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85799999999999998,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":2.1230000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.751,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.97199999999999998,"5min":1.107,"15min":1.107},"last":2.0459999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.91100000000000003,"15min":0.93700000000000006},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.96999999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":1.2270000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.90000000000000002,"15min":0.93200000000000005},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96299999999999997,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.851},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.006,"5min":1.474,"15min":1.474},"last":1.123}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.008,"15min":1.038},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.073,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.254},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.0529999999999999,"15min":1.093},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.194,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":1.0369999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.1000000000000001,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1419999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.075},{"interface":"front","average":{"1min":0.77200000000000002,"5min":1.0409999999999999,"15min":1.0860000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.097,"5min":1.99,"15min":1.99},"last":1.9019999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92800000000000005,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.347,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":2.0720000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.29}]},{"osd":11,"last update":"Fri Jul 21 22:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":1.9970000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.2}]}]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":496630,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.87}]},{"osd":1,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83999999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.751}]},{"osd":2,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78100000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.753,"15min":0.745},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.59999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.627,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.111,"15min":1.111},"last":0.79600000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76800000000000002,"15min":0.75900000000000001},"min":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.62,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.97099999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.66200000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.76400000000000001,"15min":0.746},"min":{"1min":0.56699999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.90100000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.94799999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1092832,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.437,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.44,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.625}]},{"osd":3,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.628,"15min":0.624},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82799999999999996,"5min":1.399,"15min":1.399},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.502,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.745,"15min":0.745},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92700000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.66100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79700000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.499,"15min":0.499},"max":{"1min":0.86099999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Fri Jul 21 22:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78300000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":491886,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.627}]},{"osd":1,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73599999999999999,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2210000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.5}]},{"osd":2,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92800000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.56599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0389999999999999,"5min":1.145,"15min":1.145},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1910000000000001,"5min":1.284,"15min":1.284},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1140000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.081,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81100000000000005,"15min":0.80000000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89300000000000002,"15min":0.874},"min":{"1min":0.66300000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2849999999999999,"5min":1.369,"15min":1.369},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Jul 21 22:20:11 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.82699999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.3029999999999999,"5min":1.599,"15min":1.599},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90100000000000002,"15min":0.90800000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3560000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.008,"15min":1.012},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.88},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95099999999999996,"15min":0.95499999999999996},"min":{"1min":0.60399999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4330000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0409999999999999,"15min":1.044},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3340000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.85699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":1052672,"data_stored":550498,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.50700000000000001},"min":{"1min":0.45300000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.95099999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80800000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58099999999999996,"15min":0.56599999999999995},"min":{"1min":0.5,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76700000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Jul 21 22:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.149,"15min":1.149},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.37,"15min":0.37},"max":{"1min":0.85699999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.372},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.33800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89100000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.68600000000000005,"15min":0.66600000000000004},"min":{"1min":0.52900000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.53500000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.44},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.42499999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.748}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99199999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.84699999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0600000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89700000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":493274,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60999999999999999,"15min":0.52800000000000002},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68600000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.3160000000000001,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.98099999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.748,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.282},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.032,"5min":1.151,"15min":1.151},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.056}]},{"osd":4,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.054,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.149}]},{"osd":5,"last update":"Fri Jul 21 22:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.75,"15min":0.70799999999999996},"min":{"1min":0.497,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.246}]},{"osd":6,"last update":"Fri Jul 21 22:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.101,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.306},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93700000000000006,"5min":1.081,"15min":1.081},"last":1.103}]},{"osd":7,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":1.135}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.333}]},{"osd":9,"last update":"Fri Jul 21 22:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.56499999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1739999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":6.0640000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.94099999999999995}]},{"osd":11,"last update":"Fri Jul 21 22:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.038}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":552190,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.63900000000000001},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88800000000000001,"5min":3.988,"15min":3.988},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70699999999999996,"5min":1.421,"15min":1.421},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.94199999999999995,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59599999999999997,"15min":0.53400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92800000000000005,"5min":2.044,"15min":2.044},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93100000000000005,"5min":1.984,"15min":1.984},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.75},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48399999999999999,"15min":0.45100000000000001},"min":{"1min":0.32700000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.88600000000000001,"5min":1.542,"15min":1.542},"last":0.501},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.502,"15min":0.46500000000000002},"min":{"1min":0.36899999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.88700000000000001,"5min":1.76,"15min":1.76},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0640000000000001,"5min":1.907,"15min":1.907},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.107,"5min":1.484,"15min":1.484},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98199999999999998,"5min":1.609,"15min":1.609},"last":0.62},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0620000000000001,"5min":1.859,"15min":1.859},"last":0.67100000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.85499999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3580000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.80100000000000005,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0640000000000001,"5min":1.887,"15min":1.887},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81999999999999995,"15min":0.81499999999999995},"min":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.05,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.90500000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.872},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.94999999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":492080,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73399999999999999,"5min":0.877,"15min":0.877},"last":1.712},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.41399999999999998},"min":{"1min":0.39900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0289999999999999,"5min":1.139,"15min":1.139},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":6.0199999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85699999999999998,"5min":1.02,"15min":1.02},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.68100000000000005,"15min":0.624},"min":{"1min":0.59099999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":1.091,"15min":1.091},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68200000000000005,"15min":0.65000000000000002},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.002,"15min":1.002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.69199999999999995},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.375,"15min":0.375},"max":{"1min":0.99399999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71699999999999997,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.139,"15min":1.139},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.056,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.875}]},{"osd":8,"last update":"Fri Jul 21 22:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.208},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.111,"5min":1.302,"15min":1.302},"last":1.391},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.02,"5min":1.175,"15min":1.175},"last":1.5600000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:20:01 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.94299999999999995,"15min":0.92500000000000004},"min":{"1min":0.83899999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.081},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.873,"15min":0.878},"min":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.6160000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:20:09 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.149},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.5109999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888122,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":496878,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66200000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.439,"15min":0.42699999999999999},"min":{"1min":0.29999999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60299999999999998,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.435}]},{"osd":2,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.624,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.077,"5min":1.3,"15min":1.3},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82199999999999995,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99199999999999999,"5min":1.452,"15min":1.452},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90700000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.53000000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88100000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.49,"15min":1.49},"last":0.52200000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.622,"15min":0.59399999999999997},"min":{"1min":0.46500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.77900000000000003,"15min":0.77600000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3740000000000001,"5min":2.202,"15min":2.202},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.73899999999999999,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.526,"15min":1.526},"last":0.749},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.88200000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.86699999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85399999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":1088078,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81899999999999995,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90100000000000002,"5min":3.319,"15min":3.319},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42899999999999999,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4330000000000001,"5min":1.647,"15min":1.647},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.60299999999999998},"min":{"1min":0.503,"5min":0.436,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70499999999999996,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.796,"15min":1.796},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.53200000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97699999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.624}]},{"osd":7,"last update":"Fri Jul 21 22:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.63},"min":{"1min":0.52400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92000000000000004,"5min":1.623,"15min":1.623},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.59599999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56599999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.495}]},{"osd":9,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.73899999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.026,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.014,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98499999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.64900000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:20:21.384 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T22:20:21.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:21 smithi149 ceph-mon[129268]: pgmap v313: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:22.129 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:22.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:22 smithi104 ceph-mon[129021]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:20:22.405 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:22 smithi066 ceph-mon[126379]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:20:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:22 smithi149 ceph-mon[129268]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:20:23.368 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:23 smithi104 ceph-mon[129021]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:23.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:23 smithi066 ceph-mon[126379]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:23.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:23 smithi149 ceph-mon[129268]: pgmap v314: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:23.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 -- 172.21.15.66:0/2565845535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 msgr2=0x7f61a810e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 --2- 172.21.15.66:0/2565845535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a810e7c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f61a000b0a0 tx=0x7f61a000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 -- 172.21.15.66:0/2565845535 shutdown_connections 2023-07-21T22:20:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 --2- 172.21.15.66:0/2565845535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 --2- 172.21.15.66:0/2565845535 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a810e7c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 --2- 172.21.15.66:0/2565845535 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 0x7f61a810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.673+0000 7f61ac814700 1 -- 172.21.15.66:0/2565845535 >> 172.21.15.66:0/2565845535 conn(0x7f61a80fb660 msgr2=0x7f61a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.674+0000 7f61ac814700 1 -- 172.21.15.66:0/2565845535 shutdown_connections 2023-07-21T22:20:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.674+0000 7f61ac814700 1 -- 172.21.15.66:0/2565845535 wait complete. 2023-07-21T22:20:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.675+0000 7f61ac814700 1 Processor -- start 2023-07-21T22:20:23.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.675+0000 7f61ac814700 1 -- start start 2023-07-21T22:20:23.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.675+0000 7f61ac814700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61ac814700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 0x7f61a8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61ac814700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61ac814700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a819cf30 con 0x7f61a810c3e0 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61ac814700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a819d0a0 con 0x7f61a80728b0 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61ac814700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a819d260 con 0x7f61a80731c0 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:23.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a5d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 0x7f61a8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42534/0 (socket says 172.21.15.66:42534) 2023-07-21T22:20:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 -- 172.21.15.66:0/4004421058 learned_addr learned my addr 172.21.15.66:0/4004421058 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 -- 172.21.15.66:0/4004421058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 msgr2=0x7f61a8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:23.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 0x7f61a8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 -- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 msgr2=0x7f61a819c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a819c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a659c700 1 -- 172.21.15.66:0/4004421058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009d00 con 0x7f61a80728b0 2023-07-21T22:20:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a5d9b700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 0x7f61a8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.676+0000 7f61a6d9d700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:23.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.677+0000 7f61a659c700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8197ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f619800e9a0 tx=0x7f619800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.677+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198009790 con 0x7f61a80728b0 2023-07-21T22:20:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.677+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a819d540 con 0x7f61a80728b0 2023-07-21T22:20:23.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.677+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6198010430 con 0x7f61a80728b0 2023-07-21T22:20:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.677+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6198018680 con 0x7f61a80728b0 2023-07-21T22:20:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.677+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a81a4d60 con 0x7f61a80728b0 2023-07-21T22:20:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.678+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a8106d00 con 0x7f61a80728b0 2023-07-21T22:20:23.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.681+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f61980105a0 con 0x7f61a80728b0 2023-07-21T22:20:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.682+0000 7f61977fe700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f61900a11e0 0x7f61900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.682+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6198014070 con 0x7f61a80728b0 2023-07-21T22:20:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.682+0000 7f61a5d9b700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f61900a11e0 0x7f61900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:23.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.683+0000 7f61a5d9b700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f61900a11e0 0x7f61900a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61a8199500 tx=0x7f619c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:23.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.684+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6198096450 con 0x7f61a80728b0 2023-07-21T22:20:23.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.888+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61a8060950 con 0x7f61900a11e0 2023-07-21T22:20:23.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.901+0000 7f61977fe700 1 -- 172.21.15.66:0/4004421058 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426169 (secure 0 0 0) 0x7f61a8060950 con 0x7f61900a11e0 2023-07-21T22:20:23.950 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:23.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f61900a11e0 msgr2=0x7f61900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f61900a11e0 0x7f61900a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61a8199500 tx=0x7f619c009260 comp rx=0 tx=0).stop 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 msgr2=0x7f61a8197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8197ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f619800e9a0 tx=0x7f619800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 shutdown_connections 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f61900a11e0 0x7f61900a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61a80728b0 0x7f61a8197ee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f61a810c3e0 0x7f61a819c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 --2- 172.21.15.66:0/4004421058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61a80731c0 0x7f61a8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:23.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.907+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 >> 172.21.15.66:0/4004421058 conn(0x7f61a80fb660 msgr2=0x7f61a8103e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:23.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.908+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 shutdown_connections 2023-07-21T22:20:23.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:23.908+0000 7f61ac814700 1 -- 172.21.15.66:0/4004421058 wait complete. 2023-07-21T22:20:23.954 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:20:24.432 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2023-07-21T22:20:22.196102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653610,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":429,"num_read_kb":331,"num_write":338,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335528,"kb_used_data":12104,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606296,"statfs":{"total":1151940427776,"available":1151596847104,"internally_reserved":0,"allocated":12394496,"data_stored":7894412,"data_compressed":318015,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003375"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309536+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:19:37.309536+0000","last_peered":"2023-07-21T22:19:37.309536+0000","last_clean":"2023-07-21T22:19:37.309536+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:19:37.309536+0000","last_undegraded":"2023-07-21T22:19:37.309536+0000","last_fullsized":"2023-07-21T22:19:37.309536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:13:18.888667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308674+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:19:37.308674+0000","last_peered":"2023-07-21T22:19:37.308674+0000","last_clean":"2023-07-21T22:19:37.308674+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:19:37.308674+0000","last_undegraded":"2023-07-21T22:19:37.308674+0000","last_fullsized":"2023-07-21T22:19:37.308674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:37:53.166736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309364+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:19:37.309364+0000","last_peered":"2023-07-21T22:19:37.309364+0000","last_clean":"2023-07-21T22:19:37.309364+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:19:37.309364+0000","last_undegraded":"2023-07-21T22:19:37.309364+0000","last_fullsized":"2023-07-21T22:19:37.309364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:18.347939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.325254+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:19:37.325254+0000","last_peered":"2023-07-21T22:19:37.325254+0000","last_clean":"2023-07-21T22:19:37.325254+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:19:37.325254+0000","last_undegraded":"2023-07-21T22:19:37.325254+0000","last_fullsized":"2023-07-21T22:19:37.325254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:13:54.817283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309706+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:19:37.309706+0000","last_peered":"2023-07-21T22:19:37.309706+0000","last_clean":"2023-07-21T22:19:37.309706+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:19:37.309706+0000","last_undegraded":"2023-07-21T22:19:37.309706+0000","last_fullsized":"2023-07-21T22:19:37.309706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:19.789707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309427+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:19:37.309427+0000","last_peered":"2023-07-21T22:19:37.309427+0000","last_clean":"2023-07-21T22:19:37.309427+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:19:37.309427+0000","last_undegraded":"2023-07-21T22:19:37.309427+0000","last_fullsized":"2023-07-21T22:19:37.309427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:08.453027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403115+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:19:37.403115+0000","last_peered":"2023-07-21T22:19:37.403115+0000","last_clean":"2023-07-21T22:19:37.403115+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:19:37.403115+0000","last_undegraded":"2023-07-21T22:19:37.403115+0000","last_fullsized":"2023-07-21T22:19:37.403115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:39:46.189678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308057+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:19:37.308057+0000","last_peered":"2023-07-21T22:19:37.308057+0000","last_clean":"2023-07-21T22:19:37.308057+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:19:37.308057+0000","last_undegraded":"2023-07-21T22:19:37.308057+0000","last_fullsized":"2023-07-21T22:19:37.308057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:28.929545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310974+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:19:37.310974+0000","last_peered":"2023-07-21T22:19:37.310974+0000","last_clean":"2023-07-21T22:19:37.310974+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:19:37.310974+0000","last_undegraded":"2023-07-21T22:19:37.310974+0000","last_fullsized":"2023-07-21T22:19:37.310974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:12:05.850547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310192+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:19:37.310192+0000","last_peered":"2023-07-21T22:19:37.310192+0000","last_clean":"2023-07-21T22:19:37.310192+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:19:37.310192+0000","last_undegraded":"2023-07-21T22:19:37.310192+0000","last_fullsized":"2023-07-21T22:19:37.310192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:21:52.428612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403678+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:19:37.403678+0000","last_peered":"2023-07-21T22:19:37.403678+0000","last_clean":"2023-07-21T22:19:37.403678+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:19:37.403678+0000","last_undegraded":"2023-07-21T22:19:37.403678+0000","last_fullsized":"2023-07-21T22:19:37.403678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:26:59.219489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324807+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:19:37.324807+0000","last_peered":"2023-07-21T22:19:37.324807+0000","last_clean":"2023-07-21T22:19:37.324807+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:19:37.324807+0000","last_undegraded":"2023-07-21T22:19:37.324807+0000","last_fullsized":"2023-07-21T22:19:37.324807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:55:15.589116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313656+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:19:37.313656+0000","last_peered":"2023-07-21T22:19:37.313656+0000","last_clean":"2023-07-21T22:19:37.313656+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:19:37.313656+0000","last_undegraded":"2023-07-21T22:19:37.313656+0000","last_fullsized":"2023-07-21T22:19:37.313656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:52:03.698646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308731+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:19:37.308731+0000","last_peered":"2023-07-21T22:19:37.308731+0000","last_clean":"2023-07-21T22:19:37.308731+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:19:37.308731+0000","last_undegraded":"2023-07-21T22:19:37.308731+0000","last_fullsized":"2023-07-21T22:19:37.308731+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:12:54.936169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308470+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:19:37.308470+0000","last_peered":"2023-07-21T22:19:37.308470+0000","last_clean":"2023-07-21T22:19:37.308470+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:19:37.308470+0000","last_undegraded":"2023-07-21T22:19:37.308470+0000","last_fullsized":"2023-07-21T22:19:37.308470+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:59:54.865549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309872+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:19:37.309872+0000","last_peered":"2023-07-21T22:19:37.309872+0000","last_clean":"2023-07-21T22:19:37.309872+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:19:37.309872+0000","last_undegraded":"2023-07-21T22:19:37.309872+0000","last_fullsized":"2023-07-21T22:19:37.309872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:46:06.325343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309243+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:19:37.309243+0000","last_peered":"2023-07-21T22:19:37.309243+0000","last_clean":"2023-07-21T22:19:37.309243+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:19:37.309243+0000","last_undegraded":"2023-07-21T22:19:37.309243+0000","last_fullsized":"2023-07-21T22:19:37.309243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:07:42.396016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.305114+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:19:37.305114+0000","last_peered":"2023-07-21T22:19:37.305114+0000","last_clean":"2023-07-21T22:19:37.305114+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:19:37.305114+0000","last_undegraded":"2023-07-21T22:19:37.305114+0000","last_fullsized":"2023-07-21T22:19:37.305114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:36.399061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307143+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:19:37.307143+0000","last_peered":"2023-07-21T22:19:37.307143+0000","last_clean":"2023-07-21T22:19:37.307143+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:19:37.307143+0000","last_undegraded":"2023-07-21T22:19:37.307143+0000","last_fullsized":"2023-07-21T22:19:37.307143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:04:06.149628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306699+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:19:37.306699+0000","last_peered":"2023-07-21T22:19:37.306699+0000","last_clean":"2023-07-21T22:19:37.306699+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:19:37.306699+0000","last_undegraded":"2023-07-21T22:19:37.306699+0000","last_fullsized":"2023-07-21T22:19:37.306699+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:41.903328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.305356+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:19:37.305356+0000","last_peered":"2023-07-21T22:19:37.305356+0000","last_clean":"2023-07-21T22:19:37.305356+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:19:37.305356+0000","last_undegraded":"2023-07-21T22:19:37.305356+0000","last_fullsized":"2023-07-21T22:19:37.305356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:41.795680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309580+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:19:37.309580+0000","last_peered":"2023-07-21T22:19:37.309580+0000","last_clean":"2023-07-21T22:19:37.309580+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:19:37.309580+0000","last_undegraded":"2023-07-21T22:19:37.309580+0000","last_fullsized":"2023-07-21T22:19:37.309580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:43:02.598429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324376+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:19:37.324376+0000","last_peered":"2023-07-21T22:19:37.324376+0000","last_clean":"2023-07-21T22:19:37.324376+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:19:37.324376+0000","last_undegraded":"2023-07-21T22:19:37.324376+0000","last_fullsized":"2023-07-21T22:19:37.324376+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:43:39.249253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310460+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:19:37.310460+0000","last_peered":"2023-07-21T22:19:37.310460+0000","last_clean":"2023-07-21T22:19:37.310460+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:19:37.310460+0000","last_undegraded":"2023-07-21T22:19:37.310460+0000","last_fullsized":"2023-07-21T22:19:37.310460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:15.028762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309029+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:19:37.309029+0000","last_peered":"2023-07-21T22:19:37.309029+0000","last_clean":"2023-07-21T22:19:37.309029+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:19:37.309029+0000","last_undegraded":"2023-07-21T22:19:37.309029+0000","last_fullsized":"2023-07-21T22:19:37.309029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:53.913101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324451+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:19:37.324451+0000","last_peered":"2023-07-21T22:19:37.324451+0000","last_clean":"2023-07-21T22:19:37.324451+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:19:37.324451+0000","last_undegraded":"2023-07-21T22:19:37.324451+0000","last_fullsized":"2023-07-21T22:19:37.324451+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:53:18.343038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311026+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:19:37.311026+0000","last_peered":"2023-07-21T22:19:37.311026+0000","last_clean":"2023-07-21T22:19:37.311026+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:19:37.311026+0000","last_undegraded":"2023-07-21T22:19:37.311026+0000","last_fullsized":"2023-07-21T22:19:37.311026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:20.544833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311511+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:19:37.311511+0000","last_peered":"2023-07-21T22:19:37.311511+0000","last_clean":"2023-07-21T22:19:37.311511+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:19:37.311511+0000","last_undegraded":"2023-07-21T22:19:37.311511+0000","last_fullsized":"2023-07-21T22:19:37.311511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:10:47.470299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324884+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:19:37.324884+0000","last_peered":"2023-07-21T22:19:37.324884+0000","last_clean":"2023-07-21T22:19:37.324884+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:19:37.324884+0000","last_undegraded":"2023-07-21T22:19:37.324884+0000","last_fullsized":"2023-07-21T22:19:37.324884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:04:52.393245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310052+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:19:37.310052+0000","last_peered":"2023-07-21T22:19:37.310052+0000","last_clean":"2023-07-21T22:19:37.310052+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:19:37.310052+0000","last_undegraded":"2023-07-21T22:19:37.310052+0000","last_fullsized":"2023-07-21T22:19:37.310052+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:55.512739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318598+0000","last_change":"2023-07-21T22:19:07.373299+0000","last_active":"2023-07-21T22:19:37.318598+0000","last_peered":"2023-07-21T22:19:37.318598+0000","last_clean":"2023-07-21T22:19:37.318598+0000","last_became_active":"2023-07-21T22:19:07.372944+0000","last_became_peered":"2023-07-21T22:19:07.372944+0000","last_unstale":"2023-07-21T22:19:37.318598+0000","last_undegraded":"2023-07-21T22:19:37.318598+0000","last_fullsized":"2023-07-21T22:19:37.318598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:40:47.140681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.404292+0000","last_change":"2023-07-21T22:19:37.404292+0000","last_active":"2023-07-21T22:19:37.404292+0000","last_peered":"2023-07-21T22:19:37.404292+0000","last_clean":"2023-07-21T22:19:37.404292+0000","last_became_active":"2023-07-21T22:19:37.403826+0000","last_became_peered":"2023-07-21T22:19:37.403826+0000","last_unstale":"2023-07-21T22:19:37.404292+0000","last_undegraded":"2023-07-21T22:19:37.404292+0000","last_fullsized":"2023-07-21T22:19:37.404292+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:37:43.508394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":78,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:15.377301+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:20:15.377301+0000","last_peered":"2023-07-21T22:20:15.377301+0000","last_clean":"2023-07-21T22:20:15.377301+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:20:15.377301+0000","last_undegraded":"2023-07-21T22:20:15.377301+0000","last_fullsized":"2023-07-21T22:20:15.377301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:41:26.565797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310268+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:19:37.310268+0000","last_peered":"2023-07-21T22:19:37.310268+0000","last_clean":"2023-07-21T22:19:37.310268+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:19:37.310268+0000","last_undegraded":"2023-07-21T22:19:37.310268+0000","last_fullsized":"2023-07-21T22:19:37.310268+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:48.148796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306802+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:19:37.306802+0000","last_peered":"2023-07-21T22:19:37.306802+0000","last_clean":"2023-07-21T22:19:37.306802+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:19:37.306802+0000","last_undegraded":"2023-07-21T22:19:37.306802+0000","last_fullsized":"2023-07-21T22:19:37.306802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:36:53.824814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.328942+0000","last_change":"2023-07-21T22:19:37.328942+0000","last_active":"2023-07-21T22:19:37.328942+0000","last_peered":"2023-07-21T22:19:37.328942+0000","last_clean":"2023-07-21T22:19:37.328942+0000","last_became_active":"2023-07-21T22:19:37.325738+0000","last_became_peered":"2023-07-21T22:19:37.325738+0000","last_unstale":"2023-07-21T22:19:37.328942+0000","last_undegraded":"2023-07-21T22:19:37.328942+0000","last_fullsized":"2023-07-21T22:19:37.328942+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:49:28.904915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318449+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:19:37.318449+0000","last_peered":"2023-07-21T22:19:37.318449+0000","last_clean":"2023-07-21T22:19:37.318449+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:19:37.318449+0000","last_undegraded":"2023-07-21T22:19:37.318449+0000","last_fullsized":"2023-07-21T22:19:37.318449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:51.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326119+0000","last_change":"2023-07-21T22:19:37.326119+0000","last_active":"2023-07-21T22:19:37.326119+0000","last_peered":"2023-07-21T22:19:37.326119+0000","last_clean":"2023-07-21T22:19:37.326119+0000","last_became_active":"2023-07-21T22:19:37.325675+0000","last_became_peered":"2023-07-21T22:19:37.325675+0000","last_unstale":"2023-07-21T22:19:37.326119+0000","last_undegraded":"2023-07-21T22:19:37.326119+0000","last_fullsized":"2023-07-21T22:19:37.326119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:41:45.097614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.312874+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:19:37.312874+0000","last_peered":"2023-07-21T22:19:37.312874+0000","last_clean":"2023-07-21T22:19:37.312874+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:19:37.312874+0000","last_undegraded":"2023-07-21T22:19:37.312874+0000","last_fullsized":"2023-07-21T22:19:37.312874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:23:16.323346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307917+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:19:37.307917+0000","last_peered":"2023-07-21T22:19:37.307917+0000","last_clean":"2023-07-21T22:19:37.307917+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:19:37.307917+0000","last_undegraded":"2023-07-21T22:19:37.307917+0000","last_fullsized":"2023-07-21T22:19:37.307917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:50:00.787704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308525+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:19:37.308525+0000","last_peered":"2023-07-21T22:19:37.308525+0000","last_clean":"2023-07-21T22:19:37.308525+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:19:37.308525+0000","last_undegraded":"2023-07-21T22:19:37.308525+0000","last_fullsized":"2023-07-21T22:19:37.308525+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:10:19.595185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317845+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:19:37.317845+0000","last_peered":"2023-07-21T22:19:37.317845+0000","last_clean":"2023-07-21T22:19:37.317845+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:19:37.317845+0000","last_undegraded":"2023-07-21T22:19:37.317845+0000","last_fullsized":"2023-07-21T22:19:37.317845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:51:14.936344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.304910+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:19:37.304910+0000","last_peered":"2023-07-21T22:19:37.304910+0000","last_clean":"2023-07-21T22:19:37.304910+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:19:37.304910+0000","last_undegraded":"2023-07-21T22:19:37.304910+0000","last_fullsized":"2023-07-21T22:19:37.304910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:45:34.474827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308197+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:19:37.308197+0000","last_peered":"2023-07-21T22:19:37.308197+0000","last_clean":"2023-07-21T22:19:37.308197+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:19:37.308197+0000","last_undegraded":"2023-07-21T22:19:37.308197+0000","last_fullsized":"2023-07-21T22:19:37.308197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:28:20.851837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:16.378209+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:20:16.378209+0000","last_peered":"2023-07-21T22:20:16.378209+0000","last_clean":"2023-07-21T22:20:16.378209+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:20:16.378209+0000","last_undegraded":"2023-07-21T22:20:16.378209+0000","last_fullsized":"2023-07-21T22:20:16.378209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:26:25.437396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311461+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:19:37.311461+0000","last_peered":"2023-07-21T22:19:37.311461+0000","last_clean":"2023-07-21T22:19:37.311461+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:19:37.311461+0000","last_undegraded":"2023-07-21T22:19:37.311461+0000","last_fullsized":"2023-07-21T22:19:37.311461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:04:20.100122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326929+0000","last_change":"2023-07-21T22:19:37.326317+0000","last_active":"2023-07-21T22:19:37.326929+0000","last_peered":"2023-07-21T22:19:37.326929+0000","last_clean":"2023-07-21T22:19:37.326929+0000","last_became_active":"2023-07-21T22:19:37.325791+0000","last_became_peered":"2023-07-21T22:19:37.325791+0000","last_unstale":"2023-07-21T22:19:37.326929+0000","last_undegraded":"2023-07-21T22:19:37.326929+0000","last_fullsized":"2023-07-21T22:19:37.326929+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:11:24.661772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403643+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:19:37.403643+0000","last_peered":"2023-07-21T22:19:37.403643+0000","last_clean":"2023-07-21T22:19:37.403643+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:19:37.403643+0000","last_undegraded":"2023-07-21T22:19:37.403643+0000","last_fullsized":"2023-07-21T22:19:37.403643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:39:53.409122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309161+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:19:37.309161+0000","last_peered":"2023-07-21T22:19:37.309161+0000","last_clean":"2023-07-21T22:19:37.309161+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:19:37.309161+0000","last_undegraded":"2023-07-21T22:19:37.309161+0000","last_fullsized":"2023-07-21T22:19:37.309161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:19.128046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324638+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:19:37.324638+0000","last_peered":"2023-07-21T22:19:37.324638+0000","last_clean":"2023-07-21T22:19:37.324638+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:19:37.324638+0000","last_undegraded":"2023-07-21T22:19:37.324638+0000","last_fullsized":"2023-07-21T22:19:37.324638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:41.858871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318440+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:19:37.318440+0000","last_peered":"2023-07-21T22:19:37.318440+0000","last_clean":"2023-07-21T22:19:37.318440+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:19:37.318440+0000","last_undegraded":"2023-07-21T22:19:37.318440+0000","last_fullsized":"2023-07-21T22:19:37.318440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:18:31.785229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311593+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:19:37.311593+0000","last_peered":"2023-07-21T22:19:37.311593+0000","last_clean":"2023-07-21T22:19:37.311593+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:19:37.311593+0000","last_undegraded":"2023-07-21T22:19:37.311593+0000","last_fullsized":"2023-07-21T22:19:37.311593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:07:46.707479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.304736+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:19:37.304736+0000","last_peered":"2023-07-21T22:19:37.304736+0000","last_clean":"2023-07-21T22:19:37.304736+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:19:37.304736+0000","last_undegraded":"2023-07-21T22:19:37.304736+0000","last_fullsized":"2023-07-21T22:19:37.304736+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:20:49.844080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'30","reported_seq":151,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:17.380124+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:20:17.380124+0000","last_peered":"2023-07-21T22:20:17.380124+0000","last_clean":"2023-07-21T22:20:17.380124+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:20:17.380124+0000","last_undegraded":"2023-07-21T22:20:17.380124+0000","last_fullsized":"2023-07-21T22:20:17.380124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:37:08.163210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58702,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":95,"num_read_kb":104,"num_write":31,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311432+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:19:37.311432+0000","last_peered":"2023-07-21T22:19:37.311432+0000","last_clean":"2023-07-21T22:19:37.311432+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:19:37.311432+0000","last_undegraded":"2023-07-21T22:19:37.311432+0000","last_fullsized":"2023-07-21T22:19:37.311432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:01:58.537492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318926+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:19:37.318926+0000","last_peered":"2023-07-21T22:19:37.318926+0000","last_clean":"2023-07-21T22:19:37.318926+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:19:37.318926+0000","last_undegraded":"2023-07-21T22:19:37.318926+0000","last_fullsized":"2023-07-21T22:19:37.318926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:35:25.618831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306879+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:19:37.306879+0000","last_peered":"2023-07-21T22:19:37.306879+0000","last_clean":"2023-07-21T22:19:37.306879+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:19:37.306879+0000","last_undegraded":"2023-07-21T22:19:37.306879+0000","last_fullsized":"2023-07-21T22:19:37.306879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:20:30.957940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313415+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:19:37.313415+0000","last_peered":"2023-07-21T22:19:37.313415+0000","last_clean":"2023-07-21T22:19:37.313415+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:19:37.313415+0000","last_undegraded":"2023-07-21T22:19:37.313415+0000","last_fullsized":"2023-07-21T22:19:37.313415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:58:25.029301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308492+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:19:37.308492+0000","last_peered":"2023-07-21T22:19:37.308492+0000","last_clean":"2023-07-21T22:19:37.308492+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:19:37.308492+0000","last_undegraded":"2023-07-21T22:19:37.308492+0000","last_fullsized":"2023-07-21T22:19:37.308492+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:59.600990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318303+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:19:37.318303+0000","last_peered":"2023-07-21T22:19:37.318303+0000","last_clean":"2023-07-21T22:19:37.318303+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:19:37.318303+0000","last_undegraded":"2023-07-21T22:19:37.318303+0000","last_fullsized":"2023-07-21T22:19:37.318303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:54:01.950928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403282+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:19:37.403282+0000","last_peered":"2023-07-21T22:19:37.403282+0000","last_clean":"2023-07-21T22:19:37.403282+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:19:37.403282+0000","last_undegraded":"2023-07-21T22:19:37.403282+0000","last_fullsized":"2023-07-21T22:19:37.403282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:24:07.126739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.404187+0000","last_change":"2023-07-21T22:19:37.403648+0000","last_active":"2023-07-21T22:19:37.404187+0000","last_peered":"2023-07-21T22:19:37.404187+0000","last_clean":"2023-07-21T22:19:37.404187+0000","last_became_active":"2023-07-21T22:19:37.403252+0000","last_became_peered":"2023-07-21T22:19:37.403252+0000","last_unstale":"2023-07-21T22:19:37.404187+0000","last_undegraded":"2023-07-21T22:19:37.404187+0000","last_fullsized":"2023-07-21T22:19:37.404187+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:09:39.202554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":80,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:19.386129+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:20:19.386129+0000","last_peered":"2023-07-21T22:20:19.386129+0000","last_clean":"2023-07-21T22:20:19.386129+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:20:19.386129+0000","last_undegraded":"2023-07-21T22:20:19.386129+0000","last_fullsized":"2023-07-21T22:20:19.386129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:00:10.807428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403075+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:19:37.403075+0000","last_peered":"2023-07-21T22:19:37.403075+0000","last_clean":"2023-07-21T22:19:37.403075+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:19:37.403075+0000","last_undegraded":"2023-07-21T22:19:37.403075+0000","last_fullsized":"2023-07-21T22:19:37.403075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:54:05.516094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313309+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:19:37.313309+0000","last_peered":"2023-07-21T22:19:37.313309+0000","last_clean":"2023-07-21T22:19:37.313309+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:19:37.313309+0000","last_undegraded":"2023-07-21T22:19:37.313309+0000","last_fullsized":"2023-07-21T22:19:37.313309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:47.924077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309765+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:19:37.309765+0000","last_peered":"2023-07-21T22:19:37.309765+0000","last_clean":"2023-07-21T22:19:37.309765+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:19:37.309765+0000","last_undegraded":"2023-07-21T22:19:37.309765+0000","last_fullsized":"2023-07-21T22:19:37.309765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:56.223931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318295+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:19:37.318295+0000","last_peered":"2023-07-21T22:19:37.318295+0000","last_clean":"2023-07-21T22:19:37.318295+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:19:37.318295+0000","last_undegraded":"2023-07-21T22:19:37.318295+0000","last_fullsized":"2023-07-21T22:19:37.318295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:27:04.660917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318367+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:19:37.318367+0000","last_peered":"2023-07-21T22:19:37.318367+0000","last_clean":"2023-07-21T22:19:37.318367+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:19:37.318367+0000","last_undegraded":"2023-07-21T22:19:37.318367+0000","last_fullsized":"2023-07-21T22:19:37.318367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:07:25.219481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310611+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:19:37.310611+0000","last_peered":"2023-07-21T22:19:37.310611+0000","last_clean":"2023-07-21T22:19:37.310611+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:19:37.310611+0000","last_undegraded":"2023-07-21T22:19:37.310611+0000","last_fullsized":"2023-07-21T22:19:37.310611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:41:24.004162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309096+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:19:37.309096+0000","last_peered":"2023-07-21T22:19:37.309096+0000","last_clean":"2023-07-21T22:19:37.309096+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:19:37.309096+0000","last_undegraded":"2023-07-21T22:19:37.309096+0000","last_fullsized":"2023-07-21T22:19:37.309096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:32:04.428231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309958+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:19:37.309958+0000","last_peered":"2023-07-21T22:19:37.309958+0000","last_clean":"2023-07-21T22:19:37.309958+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:19:37.309958+0000","last_undegraded":"2023-07-21T22:19:37.309958+0000","last_fullsized":"2023-07-21T22:19:37.309958+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:27.009320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.312431+0000","last_change":"2023-07-21T22:19:37.312431+0000","last_active":"2023-07-21T22:19:37.312431+0000","last_peered":"2023-07-21T22:19:37.312431+0000","last_clean":"2023-07-21T22:19:37.312431+0000","last_became_active":"2023-07-21T22:19:37.312038+0000","last_became_peered":"2023-07-21T22:19:37.312038+0000","last_unstale":"2023-07-21T22:19:37.312431+0000","last_undegraded":"2023-07-21T22:19:37.312431+0000","last_fullsized":"2023-07-21T22:19:37.312431+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:40:24.338102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310127+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:19:37.310127+0000","last_peered":"2023-07-21T22:19:37.310127+0000","last_clean":"2023-07-21T22:19:37.310127+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:19:37.310127+0000","last_undegraded":"2023-07-21T22:19:37.310127+0000","last_fullsized":"2023-07-21T22:19:37.310127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:22:26.812058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313198+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:19:37.313198+0000","last_peered":"2023-07-21T22:19:37.313198+0000","last_clean":"2023-07-21T22:19:37.313198+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:19:37.313198+0000","last_undegraded":"2023-07-21T22:19:37.313198+0000","last_fullsized":"2023-07-21T22:19:37.313198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:26:50.417626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403339+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:19:37.403339+0000","last_peered":"2023-07-21T22:19:37.403339+0000","last_clean":"2023-07-21T22:19:37.403339+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:19:37.403339+0000","last_undegraded":"2023-07-21T22:19:37.403339+0000","last_fullsized":"2023-07-21T22:19:37.403339+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:04:04.011358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309432+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:19:37.309432+0000","last_peered":"2023-07-21T22:19:37.309432+0000","last_clean":"2023-07-21T22:19:37.309432+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:19:37.309432+0000","last_undegraded":"2023-07-21T22:19:37.309432+0000","last_fullsized":"2023-07-21T22:19:37.309432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:25:52.984944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313268+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:19:37.313268+0000","last_peered":"2023-07-21T22:19:37.313268+0000","last_clean":"2023-07-21T22:19:37.313268+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:19:37.313268+0000","last_undegraded":"2023-07-21T22:19:37.313268+0000","last_fullsized":"2023-07-21T22:19:37.313268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:22:45.429732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317999+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:19:37.317999+0000","last_peered":"2023-07-21T22:19:37.317999+0000","last_clean":"2023-07-21T22:19:37.317999+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:19:37.317999+0000","last_undegraded":"2023-07-21T22:19:37.317999+0000","last_fullsized":"2023-07-21T22:19:37.317999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:00:22.072167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:20:16.381043+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:20:16.381043+0000","last_peered":"2023-07-21T22:20:16.381043+0000","last_clean":"2023-07-21T22:20:16.381043+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:20:16.381043+0000","last_undegraded":"2023-07-21T22:20:16.381043+0000","last_fullsized":"2023-07-21T22:20:16.381043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:40.068303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403576+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:19:37.403576+0000","last_peered":"2023-07-21T22:19:37.403576+0000","last_clean":"2023-07-21T22:19:37.403576+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:19:37.403576+0000","last_undegraded":"2023-07-21T22:19:37.403576+0000","last_fullsized":"2023-07-21T22:19:37.403576+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:42:34.557201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308304+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:19:37.308304+0000","last_peered":"2023-07-21T22:19:37.308304+0000","last_clean":"2023-07-21T22:19:37.308304+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:19:37.308304+0000","last_undegraded":"2023-07-21T22:19:37.308304+0000","last_fullsized":"2023-07-21T22:19:37.308304+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:01:52.893699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324555+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:19:37.324555+0000","last_peered":"2023-07-21T22:19:37.324555+0000","last_clean":"2023-07-21T22:19:37.324555+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:19:37.324555+0000","last_undegraded":"2023-07-21T22:19:37.324555+0000","last_fullsized":"2023-07-21T22:19:37.324555+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:27:17.708385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.402915+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:19:37.402915+0000","last_peered":"2023-07-21T22:19:37.402915+0000","last_clean":"2023-07-21T22:19:37.402915+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:19:37.402915+0000","last_undegraded":"2023-07-21T22:19:37.402915+0000","last_fullsized":"2023-07-21T22:19:37.402915+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:03:56.744877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318021+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:19:37.318021+0000","last_peered":"2023-07-21T22:19:37.318021+0000","last_clean":"2023-07-21T22:19:37.318021+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:19:37.318021+0000","last_undegraded":"2023-07-21T22:19:37.318021+0000","last_fullsized":"2023-07-21T22:19:37.318021+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:45.952550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.312718+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:19:37.312718+0000","last_peered":"2023-07-21T22:19:37.312718+0000","last_clean":"2023-07-21T22:19:37.312718+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:19:37.312718+0000","last_undegraded":"2023-07-21T22:19:37.312718+0000","last_fullsized":"2023-07-21T22:19:37.312718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:26:49.849442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.402997+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:19:37.402997+0000","last_peered":"2023-07-21T22:19:37.402997+0000","last_clean":"2023-07-21T22:19:37.402997+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:19:37.402997+0000","last_undegraded":"2023-07-21T22:19:37.402997+0000","last_fullsized":"2023-07-21T22:19:37.402997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:58:20.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308304+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:19:37.308304+0000","last_peered":"2023-07-21T22:19:37.308304+0000","last_clean":"2023-07-21T22:19:37.308304+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:19:37.308304+0000","last_undegraded":"2023-07-21T22:19:37.308304+0000","last_fullsized":"2023-07-21T22:19:37.308304+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:40.846884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.311330+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:19:37.311330+0000","last_peered":"2023-07-21T22:19:37.311330+0000","last_clean":"2023-07-21T22:19:37.311330+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:19:37.311330+0000","last_undegraded":"2023-07-21T22:19:37.311330+0000","last_fullsized":"2023-07-21T22:19:37.311330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:29:36.526548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313447+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:19:37.313447+0000","last_peered":"2023-07-21T22:19:37.313447+0000","last_clean":"2023-07-21T22:19:37.313447+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:19:37.313447+0000","last_undegraded":"2023-07-21T22:19:37.313447+0000","last_fullsized":"2023-07-21T22:19:37.313447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:55.808390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307908+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:19:37.307908+0000","last_peered":"2023-07-21T22:19:37.307908+0000","last_clean":"2023-07-21T22:19:37.307908+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:19:37.307908+0000","last_undegraded":"2023-07-21T22:19:37.307908+0000","last_fullsized":"2023-07-21T22:19:37.307908+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:39:46.255453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:40.780207+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:19:40.780207+0000","last_peered":"2023-07-21T22:19:40.780207+0000","last_clean":"2023-07-21T22:19:40.780207+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:19:40.780207+0000","last_undegraded":"2023-07-21T22:19:40.780207+0000","last_fullsized":"2023-07-21T22:19:40.780207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:21:27.644738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309258+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:19:37.309258+0000","last_peered":"2023-07-21T22:19:37.309258+0000","last_clean":"2023-07-21T22:19:37.309258+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:19:37.309258+0000","last_undegraded":"2023-07-21T22:19:37.309258+0000","last_fullsized":"2023-07-21T22:19:37.309258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:49:50.061521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306844+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:19:37.306844+0000","last_peered":"2023-07-21T22:19:37.306844+0000","last_clean":"2023-07-21T22:19:37.306844+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:19:37.306844+0000","last_undegraded":"2023-07-21T22:19:37.306844+0000","last_fullsized":"2023-07-21T22:19:37.306844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:10:56.610067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.325011+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:19:37.325011+0000","last_peered":"2023-07-21T22:19:37.325011+0000","last_clean":"2023-07-21T22:19:37.325011+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:19:37.325011+0000","last_undegraded":"2023-07-21T22:19:37.325011+0000","last_fullsized":"2023-07-21T22:19:37.325011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:42:40.440014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.404445+0000","last_change":"2023-07-21T22:19:37.404445+0000","last_active":"2023-07-21T22:19:37.404445+0000","last_peered":"2023-07-21T22:19:37.404445+0000","last_clean":"2023-07-21T22:19:37.404445+0000","last_became_active":"2023-07-21T22:19:37.404011+0000","last_became_peered":"2023-07-21T22:19:37.404011+0000","last_unstale":"2023-07-21T22:19:37.404445+0000","last_undegraded":"2023-07-21T22:19:37.404445+0000","last_fullsized":"2023-07-21T22:19:37.404445+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:53:08.840927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324952+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:19:37.324952+0000","last_peered":"2023-07-21T22:19:37.324952+0000","last_clean":"2023-07-21T22:19:37.324952+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:19:37.324952+0000","last_undegraded":"2023-07-21T22:19:37.324952+0000","last_fullsized":"2023-07-21T22:19:37.324952+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:29.925386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310235+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:19:37.310235+0000","last_peered":"2023-07-21T22:19:37.310235+0000","last_clean":"2023-07-21T22:19:37.310235+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:19:37.310235+0000","last_undegraded":"2023-07-21T22:19:37.310235+0000","last_fullsized":"2023-07-21T22:19:37.310235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:17:04.984129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309689+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:19:37.309689+0000","last_peered":"2023-07-21T22:19:37.309689+0000","last_clean":"2023-07-21T22:19:37.309689+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:19:37.309689+0000","last_undegraded":"2023-07-21T22:19:37.309689+0000","last_fullsized":"2023-07-21T22:19:37.309689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:55.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.305251+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:19:37.305251+0000","last_peered":"2023-07-21T22:19:37.305251+0000","last_clean":"2023-07-21T22:19:37.305251+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:19:37.305251+0000","last_undegraded":"2023-07-21T22:19:37.305251+0000","last_fullsized":"2023-07-21T22:19:37.305251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:03:31.356395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308327+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:19:37.308327+0000","last_peered":"2023-07-21T22:19:37.308327+0000","last_clean":"2023-07-21T22:19:37.308327+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:19:37.308327+0000","last_undegraded":"2023-07-21T22:19:37.308327+0000","last_fullsized":"2023-07-21T22:19:37.308327+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:22:55.175149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308117+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:19:37.308117+0000","last_peered":"2023-07-21T22:19:37.308117+0000","last_clean":"2023-07-21T22:19:37.308117+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:19:37.308117+0000","last_undegraded":"2023-07-21T22:19:37.308117+0000","last_fullsized":"2023-07-21T22:19:37.308117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:46:44.426120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324709+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:19:37.324709+0000","last_peered":"2023-07-21T22:19:37.324709+0000","last_clean":"2023-07-21T22:19:37.324709+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:19:37.324709+0000","last_undegraded":"2023-07-21T22:19:37.324709+0000","last_fullsized":"2023-07-21T22:19:37.324709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:33:46.778289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.316192+0000","last_change":"2023-07-21T22:19:37.315809+0000","last_active":"2023-07-21T22:19:37.316192+0000","last_peered":"2023-07-21T22:19:37.316192+0000","last_clean":"2023-07-21T22:19:37.316192+0000","last_became_active":"2023-07-21T22:19:37.312331+0000","last_became_peered":"2023-07-21T22:19:37.312331+0000","last_unstale":"2023-07-21T22:19:37.316192+0000","last_undegraded":"2023-07-21T22:19:37.316192+0000","last_fullsized":"2023-07-21T22:19:37.316192+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:46:09.215699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308886+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:19:37.308886+0000","last_peered":"2023-07-21T22:19:37.308886+0000","last_clean":"2023-07-21T22:19:37.308886+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:19:37.308886+0000","last_undegraded":"2023-07-21T22:19:37.308886+0000","last_fullsized":"2023-07-21T22:19:37.308886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:44:20.850151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.324857+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:19:37.324857+0000","last_peered":"2023-07-21T22:19:37.324857+0000","last_clean":"2023-07-21T22:19:37.324857+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:19:37.324857+0000","last_undegraded":"2023-07-21T22:19:37.324857+0000","last_fullsized":"2023-07-21T22:19:37.324857+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:50:02.988262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309854+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:19:37.309854+0000","last_peered":"2023-07-21T22:19:37.309854+0000","last_clean":"2023-07-21T22:19:37.309854+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:19:37.309854+0000","last_undegraded":"2023-07-21T22:19:37.309854+0000","last_fullsized":"2023-07-21T22:19:37.309854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:03:53.898872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403658+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:19:37.403658+0000","last_peered":"2023-07-21T22:19:37.403658+0000","last_clean":"2023-07-21T22:19:37.403658+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:19:37.403658+0000","last_undegraded":"2023-07-21T22:19:37.403658+0000","last_fullsized":"2023-07-21T22:19:37.403658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:38:43.593685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310346+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:19:37.310346+0000","last_peered":"2023-07-21T22:19:37.310346+0000","last_clean":"2023-07-21T22:19:37.310346+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:19:37.310346+0000","last_undegraded":"2023-07-21T22:19:37.310346+0000","last_fullsized":"2023-07-21T22:19:37.310346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:46:46.183044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.307135+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:19:37.307135+0000","last_peered":"2023-07-21T22:19:37.307135+0000","last_clean":"2023-07-21T22:19:37.307135+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:19:37.307135+0000","last_undegraded":"2023-07-21T22:19:37.307135+0000","last_fullsized":"2023-07-21T22:19:37.307135+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:34:58.835266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403865+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:19:37.403865+0000","last_peered":"2023-07-21T22:19:37.403865+0000","last_clean":"2023-07-21T22:19:37.403865+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:19:37.403865+0000","last_undegraded":"2023-07-21T22:19:37.403865+0000","last_fullsized":"2023-07-21T22:19:37.403865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:48:06.009672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308879+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:19:37.308879+0000","last_peered":"2023-07-21T22:19:37.308879+0000","last_clean":"2023-07-21T22:19:37.308879+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:19:37.308879+0000","last_undegraded":"2023-07-21T22:19:37.308879+0000","last_fullsized":"2023-07-21T22:19:37.308879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:27.207972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.318725+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:19:37.318725+0000","last_peered":"2023-07-21T22:19:37.318725+0000","last_clean":"2023-07-21T22:19:37.318725+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:19:37.318725+0000","last_undegraded":"2023-07-21T22:19:37.318725+0000","last_fullsized":"2023-07-21T22:19:37.318725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:01:53.588516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403502+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:19:37.403502+0000","last_peered":"2023-07-21T22:19:37.403502+0000","last_clean":"2023-07-21T22:19:37.403502+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:19:37.403502+0000","last_undegraded":"2023-07-21T22:19:37.403502+0000","last_fullsized":"2023-07-21T22:19:37.403502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:28:06.664433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317672+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:19:37.317672+0000","last_peered":"2023-07-21T22:19:37.317672+0000","last_clean":"2023-07-21T22:19:37.317672+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:19:37.317672+0000","last_undegraded":"2023-07-21T22:19:37.317672+0000","last_fullsized":"2023-07-21T22:19:37.317672+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:13:04.207733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313421+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:19:37.313421+0000","last_peered":"2023-07-21T22:19:37.313421+0000","last_clean":"2023-07-21T22:19:37.313421+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:19:37.313421+0000","last_undegraded":"2023-07-21T22:19:37.313421+0000","last_fullsized":"2023-07-21T22:19:37.313421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:02:31.808021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.403795+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:19:37.403795+0000","last_peered":"2023-07-21T22:19:37.403795+0000","last_clean":"2023-07-21T22:19:37.403795+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:19:37.403795+0000","last_undegraded":"2023-07-21T22:19:37.403795+0000","last_fullsized":"2023-07-21T22:19:37.403795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:30:43.531900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308261+0000","last_change":"2023-07-21T22:19:07.372870+0000","last_active":"2023-07-21T22:19:37.308261+0000","last_peered":"2023-07-21T22:19:37.308261+0000","last_clean":"2023-07-21T22:19:37.308261+0000","last_became_active":"2023-07-21T22:19:07.372558+0000","last_became_peered":"2023-07-21T22:19:07.372558+0000","last_unstale":"2023-07-21T22:19:37.308261+0000","last_undegraded":"2023-07-21T22:19:37.308261+0000","last_fullsized":"2023-07-21T22:19:37.308261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:07:56.834449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.326004+0000","last_change":"2023-07-21T22:19:37.326004+0000","last_active":"2023-07-21T22:19:37.326004+0000","last_peered":"2023-07-21T22:19:37.326004+0000","last_clean":"2023-07-21T22:19:37.326004+0000","last_became_active":"2023-07-21T22:19:37.325611+0000","last_became_peered":"2023-07-21T22:19:37.325611+0000","last_unstale":"2023-07-21T22:19:37.326004+0000","last_undegraded":"2023-07-21T22:19:37.326004+0000","last_fullsized":"2023-07-21T22:19:37.326004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:52:15.766198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306222+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:19:37.306222+0000","last_peered":"2023-07-21T22:19:37.306222+0000","last_clean":"2023-07-21T22:19:37.306222+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:19:37.306222+0000","last_undegraded":"2023-07-21T22:19:37.306222+0000","last_fullsized":"2023-07-21T22:19:37.306222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:27:53.516407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309252+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:19:37.309252+0000","last_peered":"2023-07-21T22:19:37.309252+0000","last_clean":"2023-07-21T22:19:37.309252+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:19:37.309252+0000","last_undegraded":"2023-07-21T22:19:37.309252+0000","last_fullsized":"2023-07-21T22:19:37.309252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:54:52.701869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313474+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:19:37.313474+0000","last_peered":"2023-07-21T22:19:37.313474+0000","last_clean":"2023-07-21T22:19:37.313474+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:19:37.313474+0000","last_undegraded":"2023-07-21T22:19:37.313474+0000","last_fullsized":"2023-07-21T22:19:37.313474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:10:39.411498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.309279+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:19:37.309279+0000","last_peered":"2023-07-21T22:19:37.309279+0000","last_clean":"2023-07-21T22:19:37.309279+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:19:37.309279+0000","last_undegraded":"2023-07-21T22:19:37.309279+0000","last_fullsized":"2023-07-21T22:19:37.309279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:59.307946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.306810+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:19:37.306810+0000","last_peered":"2023-07-21T22:19:37.306810+0000","last_clean":"2023-07-21T22:19:37.306810+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:19:37.306810+0000","last_undegraded":"2023-07-21T22:19:37.306810+0000","last_fullsized":"2023-07-21T22:19:37.306810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:47:20.553040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.314152+0000","last_change":"2023-07-21T22:19:37.314152+0000","last_active":"2023-07-21T22:19:37.314152+0000","last_peered":"2023-07-21T22:19:37.314152+0000","last_clean":"2023-07-21T22:19:37.314152+0000","last_became_active":"2023-07-21T22:19:37.313795+0000","last_became_peered":"2023-07-21T22:19:37.313795+0000","last_unstale":"2023-07-21T22:19:37.314152+0000","last_undegraded":"2023-07-21T22:19:37.314152+0000","last_fullsized":"2023-07-21T22:19:37.314152+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:57:15.996755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.317952+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:19:37.317952+0000","last_peered":"2023-07-21T22:19:37.317952+0000","last_clean":"2023-07-21T22:19:37.317952+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:19:37.317952+0000","last_undegraded":"2023-07-21T22:19:37.317952+0000","last_fullsized":"2023-07-21T22:19:37.317952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:24:02.673526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308478+0000","last_change":"2023-07-21T22:19:07.381609+0000","last_active":"2023-07-21T22:19:37.308478+0000","last_peered":"2023-07-21T22:19:37.308478+0000","last_clean":"2023-07-21T22:19:37.308478+0000","last_became_active":"2023-07-21T22:19:07.381272+0000","last_became_peered":"2023-07-21T22:19:07.381272+0000","last_unstale":"2023-07-21T22:19:37.308478+0000","last_undegraded":"2023-07-21T22:19:37.308478+0000","last_fullsized":"2023-07-21T22:19:37.308478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:00:23.406427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.310743+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:19:37.310743+0000","last_peered":"2023-07-21T22:19:37.310743+0000","last_clean":"2023-07-21T22:19:37.310743+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:19:37.310743+0000","last_undegraded":"2023-07-21T22:19:37.310743+0000","last_fullsized":"2023-07-21T22:19:37.310743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:47:16.929327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.313825+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:19:37.313825+0000","last_peered":"2023-07-21T22:19:37.313825+0000","last_clean":"2023-07-21T22:19:37.313825+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:19:37.313825+0000","last_undegraded":"2023-07-21T22:19:37.313825+0000","last_fullsized":"2023-07-21T22:19:37.313825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:24.108010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-07-21T22:19:37.308758+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:19:37.308758+0000","last_peered":"2023-07-21T22:19:37.308758+0000","last_clean":"2023-07-21T22:19:37.308758+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:19:37.308758+0000","last_undegraded":"2023-07-21T22:19:37.308758+0000","last_fullsized":"2023-07-21T22:19:37.308758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:14.212545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62996,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":293,"num_read_kb":212,"num_write":155,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":188988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":991232,"data_stored":496474,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.67800000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.753}]},{"osd":8,"last update":"Fri Jul 21 22:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.80700000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.85399999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.79400000000000004}]}]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1134592,"data_stored":1142786,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":17.132000000000001,"5min":17.132000000000001,"15min":17.132000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":17.084,"5min":17.084,"15min":17.084},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46300000000000002,"15min":0.46800000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.29999999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.63},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.42299999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.80600000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.496},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.441,"5min":0.433,"15min":0.433},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.58999999999999997,"15min":0.60099999999999998},"min":{"1min":0.36199999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.34100000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45700000000000002}]},{"osd":5,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.80600000000000005,"15min":0.81499999999999995},"min":{"1min":0.48099999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.79500000000000004,"15min":0.80900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.92200000000000004,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.67700000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85599999999999998,"15min":0.84999999999999998},"min":{"1min":0.59999999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.48,"5min":0.98899999999999999,"15min":0.90700000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":17.163,"5min":17.163,"15min":17.163},"last":0.71899999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:19:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.73999999999999999}]}]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":500806,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.59899999999999998,"15min":0.61799999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0920000000000001,"5min":1.244,"15min":1.244},"last":0.34499999999999997}]},{"osd":1,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.80600000000000005,"15min":0.82299999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93000000000000005,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.79700000000000004,"15min":0.81000000000000005},"min":{"1min":0.55600000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94399999999999995,"5min":1.171,"15min":1.171},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.89400000000000002,"15min":0.92800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0029999999999999,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88600000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.78900000000000003,"15min":0.82199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.878,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.85599999999999998,"15min":0.89400000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85799999999999998,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.499}]},{"osd":4,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.751,"15min":0.76700000000000002},"min":{"1min":0.48299999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.97199999999999998,"5min":1.107,"15min":1.107},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.91100000000000003,"15min":0.93700000000000006},"min":{"1min":0.54100000000000004,"5min":0.497,"15min":0.497},"max":{"1min":0.96999999999999997,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.90000000000000002,"15min":0.93200000000000005},"min":{"1min":0.46899999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.96299999999999997,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.93000000000000005,"15min":0.95499999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.006,"5min":1.474,"15min":1.474},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":1.008,"15min":1.038},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.073,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":1.0529999999999999,"15min":1.093},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.194,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":1.1000000000000001,"15min":1.149},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1419999999999999,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":1.0409999999999999,"15min":1.0860000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.097,"5min":1.99,"15min":1.99},"last":0.73699999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92800000000000005,"15min":0.94999999999999996},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.347,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.76700000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.871}]}]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":496630,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.87}]},{"osd":1,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83999999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.751}]},{"osd":2,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78100000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.753,"15min":0.745},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.59999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.627,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.111,"15min":1.111},"last":0.79600000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76800000000000002,"15min":0.75900000000000001},"min":{"1min":0.52200000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.96599999999999997,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.79100000000000004,"15min":0.78300000000000003},"min":{"1min":0.62,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.97099999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:19:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.69399999999999995,"15min":0.67400000000000004},"min":{"1min":0.66200000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.873,"5min":0.76400000000000001,"15min":0.746},"min":{"1min":0.56699999999999995,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.90100000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.94799999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1092832,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.437,"15min":0.437},"min":{"1min":0.27400000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.747,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.44,"15min":0.442},"min":{"1min":0.28199999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78200000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.61499999999999999,"15min":0.60899999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.625}]},{"osd":3,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84399999999999997,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.628,"15min":0.624},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.82799999999999996,"5min":1.399,"15min":1.399},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.502,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.504,"15min":0.51400000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72599999999999998,"5min":0.745,"15min":0.745},"last":0.44400000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63300000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.44800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92700000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.68500000000000005,"15min":0.68600000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89700000000000002,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.52800000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.60599999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.66100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79700000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.98099999999999998,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.73399999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.75,"15min":0.752},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91600000000000004,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.75},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71399999999999997,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.499,"15min":0.499},"max":{"1min":0.86099999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Fri Jul 21 22:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.78300000000000003}]}]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":987136,"data_stored":491886,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.52700000000000002,"15min":0.51900000000000002},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.36799999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.627}]},{"osd":1,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.73599999999999999,"15min":0.71799999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2210000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.5}]},{"osd":2,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92800000000000005,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.77900000000000003,"15min":0.76100000000000001},"min":{"1min":0.56599999999999995,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0389999999999999,"5min":1.145,"15min":1.145},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1910000000000001,"5min":1.284,"15min":1.284},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.51300000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1140000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:19:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.081,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98199999999999998,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Fri Jul 21 22:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81100000000000005,"15min":0.80000000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.89300000000000002,"15min":0.874},"min":{"1min":0.66300000000000003,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.2849999999999999,"5min":1.369,"15min":1.369},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Jul 21 22:20:11 2023","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":0.94799999999999995,"15min":0.93999999999999995},"min":{"1min":0.82699999999999996,"5min":0.622,"15min":0.622},"max":{"1min":1.3029999999999999,"5min":1.599,"15min":1.599},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.90100000000000002,"15min":0.90800000000000003},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.3560000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.008,"15min":1.012},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.4119999999999999,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.88},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.95099999999999996,"15min":0.95499999999999996},"min":{"1min":0.60399999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.4330000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0409999999999999,"15min":1.044},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.3340000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Fri Jul 21 22:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.85699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":1028,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":1052672,"data_stored":550498,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53000000000000003,"15min":0.50700000000000001},"min":{"1min":0.45300000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.95099999999999996,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.53800000000000003,"15min":0.51200000000000001},"min":{"1min":0.38100000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80800000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.58099999999999996,"15min":0.56599999999999995},"min":{"1min":0.5,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.46400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.76700000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Jul 21 22:19:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.149,"15min":1.149},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57099999999999995},"min":{"1min":0.44,"5min":0.37,"15min":0.37},"max":{"1min":0.85699999999999998,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Fri Jul 21 22:19:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.44700000000000001,"15min":0.42699999999999999},"min":{"1min":0.315,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72199999999999998,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.372},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45800000000000002},"min":{"1min":0.33800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.48599999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89100000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.67000000000000004,"15min":0.65600000000000003},"min":{"1min":0.50900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91300000000000003,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.68600000000000005,"15min":0.66600000000000004},"min":{"1min":0.52900000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.69499999999999995,"15min":0.68700000000000006},"min":{"1min":0.53500000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91600000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.41099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.44},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61499999999999999,"15min":0.58499999999999996},"min":{"1min":0.42499999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.748}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99199999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:19:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.55000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81000000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.02,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.84699999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73899999999999999,"15min":0.72699999999999998},"min":{"1min":0.55300000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0600000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.89700000000000002}]},{"osd":11,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69599999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738445,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":493274,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60999999999999999,"15min":0.52800000000000002},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.68600000000000005,"15min":0.54800000000000004},"min":{"1min":0.42399999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.3160000000000001,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.98099999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.748,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.137,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.282},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.71499999999999997,"15min":0.64300000000000002},"min":{"1min":0.48199999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.032,"5min":1.151,"15min":1.151},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri Jul 21 22:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":1.056}]},{"osd":4,"last update":"Fri Jul 21 22:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.054,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69799999999999995,"15min":0.66900000000000004},"min":{"1min":0.41999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":1.149}]},{"osd":5,"last update":"Fri Jul 21 22:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.72699999999999998},"min":{"1min":0.48099999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.75,"15min":0.70799999999999996},"min":{"1min":0.497,"5min":0.495,"15min":0.495},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":1.246}]},{"osd":6,"last update":"Fri Jul 21 22:19:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.101,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.306},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.93700000000000006,"5min":1.081,"15min":1.081},"last":1.103}]},{"osd":7,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":1.135}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.333}]},{"osd":9,"last update":"Fri Jul 21 22:19:42 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.97399999999999998,"15min":0.98999999999999999},"min":{"1min":0.56499999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1739999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80300000000000005,"15min":0.80400000000000005},"min":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":6.0640000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.98899999999999999,"15min":0.996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.85199999999999998,"15min":0.84199999999999997},"min":{"1min":0.61499999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.94099999999999995}]},{"osd":11,"last update":"Fri Jul 21 22:19:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.038}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1077248,"data_stored":552190,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60299999999999998,"15min":0.63900000000000001},"min":{"1min":0.39900000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88800000000000001,"5min":3.988,"15min":3.988},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56499999999999995,"15min":0.53200000000000003},"min":{"1min":0.35199999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.70699999999999996,"5min":1.421,"15min":1.421},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59199999999999997,"15min":0.55200000000000005},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.94199999999999995,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.59599999999999997,"15min":0.53400000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.92800000000000005,"5min":2.044,"15min":2.044},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Jul 21 22:19:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60699999999999998,"15min":0.60499999999999998},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.93100000000000005,"5min":1.984,"15min":1.984},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60599999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.96999999999999997,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.75},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Jul 21 22:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48399999999999999,"15min":0.45100000000000001},"min":{"1min":0.32700000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.88600000000000001,"5min":1.542,"15min":1.542},"last":0.501},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.502,"15min":0.46500000000000002},"min":{"1min":0.36899999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.88700000000000001,"5min":1.76,"15min":1.76},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69999999999999996,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0640000000000001,"5min":1.907,"15min":1.907},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.68899999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.107,"5min":1.484,"15min":1.484},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:20:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.80800000000000005}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98199999999999998,"5min":1.609,"15min":1.609},"last":0.62},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0620000000000001,"5min":1.859,"15min":1.859},"last":0.67100000000000004}]},{"osd":9,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.016,"5min":2.016,"15min":2.016},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.85499999999999998},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3580000000000001,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.80100000000000005,"15min":0.79200000000000004},"min":{"1min":0.55100000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0640000000000001,"5min":1.887,"15min":1.887},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.81999999999999995,"15min":0.81499999999999995},"min":{"1min":0.57499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.05,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.90500000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.872},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.94999999999999996}]}]},{"osd":3,"up_from":26,"seq":111669149759,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":492080,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.38700000000000001},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.73399999999999999,"5min":0.877,"15min":0.877},"last":1.712},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.41399999999999998},"min":{"1min":0.39900000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58099999999999996,"15min":0.55000000000000004},"min":{"1min":0.41899999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.90600000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.60099999999999998,"15min":0.55200000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0289999999999999,"5min":1.139,"15min":1.139},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.65400000000000003,"15min":0.62},"min":{"1min":0.49099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":6.0199999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.59099999999999997},"min":{"1min":0.45800000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85699999999999998,"5min":1.02,"15min":1.02},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.68100000000000005,"15min":0.624},"min":{"1min":0.59099999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98299999999999998,"5min":1.091,"15min":1.091},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.68200000000000005,"15min":0.65000000000000002},"min":{"1min":0.59199999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97299999999999998,"5min":1.002,"15min":1.002},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.75,"15min":0.71099999999999997},"min":{"1min":0.54100000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:19:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.72299999999999998,"15min":0.69199999999999995},"min":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.442,"5min":0.375,"15min":0.375},"max":{"1min":0.99399999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:19:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71699999999999997,"15min":0.68899999999999995},"min":{"1min":0.51100000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96699999999999997,"5min":1.139,"15min":1.139},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76900000000000002,"15min":0.74399999999999999},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.056,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.875}]},{"osd":8,"last update":"Fri Jul 21 22:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.208},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:19:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.93899999999999995,"15min":0.95699999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.111,"5min":1.302,"15min":1.302},"last":1.391},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.879,"15min":0.90300000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.02,"5min":1.175,"15min":1.175},"last":1.5600000000000001}]},{"osd":10,"last update":"Fri Jul 21 22:20:01 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.94299999999999995,"15min":0.92500000000000004},"min":{"1min":0.83899999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.575,"5min":1.575,"15min":1.575},"last":1.081},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.873,"15min":0.878},"min":{"1min":0.67000000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0620000000000001,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.6160000000000001}]},{"osd":11,"last update":"Fri Jul 21 22:20:09 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.883,"5min":2.883,"15min":2.883},"last":1.149},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.5109999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":999424,"data_stored":496878,"data_compressed":17556,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.40799999999999997,"15min":0.38},"min":{"1min":0.28399999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66200000000000003,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.439,"15min":0.42699999999999999},"min":{"1min":0.29999999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60299999999999998,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.624,"15min":0.58599999999999997},"min":{"1min":0.39900000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.077,"5min":1.3,"15min":1.3},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.60999999999999999,"15min":0.56599999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.91600000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82199999999999995,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.63},{"interface":"front","average":{"1min":0.629,"5min":0.61499999999999999,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.99199999999999999,"5min":1.452,"15min":1.452},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.64700000000000002,"15min":0.627},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98099999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.747},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64900000000000002,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90700000000000003,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:19:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.53000000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88100000000000001,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.042,"5min":1.49,"15min":1.49},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.42899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.81100000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.622,"15min":0.59399999999999997},"min":{"1min":0.46500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72999999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.95199999999999996,"5min":2.202,"15min":2.202},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.752,"15min":0.72799999999999998},"min":{"1min":0.65400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.88400000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.83699999999999997,"15min":0.84299999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.077,"5min":1.526,"15min":1.526},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Jul 21 22:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.83199999999999996}]}]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":1040,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":1064960,"data_stored":1088078,"data_compressed":53337,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81899999999999995,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.42799999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90100000000000002,"5min":3.319,"15min":3.319},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42899999999999999,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.32600000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4330000000000001,"5min":1.647,"15min":1.647},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.60299999999999998},"min":{"1min":0.503,"5min":0.436,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Jul 21 22:19:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70499999999999996,"15min":0.67500000000000004},"min":{"1min":0.497,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.796,"15min":1.796},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.53200000000000003,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97699999999999998,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.624}]},{"osd":7,"last update":"Fri Jul 21 22:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.63},"min":{"1min":0.52400000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.92000000000000004,"5min":1.623,"15min":1.623},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.59599999999999997,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.56599999999999995}]},{"osd":8,"last update":"Fri Jul 21 22:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.495}]},{"osd":9,"last update":"Fri Jul 21 22:19:39 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.83399999999999996,"15min":0.82999999999999996},"min":{"1min":0.73899999999999999,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.026,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.014,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.69599999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.71499999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98499999999999999,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.64900000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:20:24.442 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-21T22:20:24.443 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-21T22:20:24.443 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-21T22:20:24.444 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-21T22:20:25.212 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:25.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:25 smithi066 ceph-mon[126379]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:20:25.553 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:25 smithi066 ceph-mon[126379]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:25.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:25 smithi104 ceph-mon[129021]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:20:25.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:25 smithi104 ceph-mon[129021]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:25.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:25 smithi149 ceph-mon[129268]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:20:25.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:25 smithi149 ceph-mon[129268]: pgmap v315: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:26.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.602+0000 7eff319e3700 1 -- 172.21.15.66:0/3106115101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c10a180 msgr2=0x7eff2c10c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:26.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.602+0000 7eff319e3700 1 --2- 172.21.15.66:0/3106115101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c10a180 0x7eff2c10c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7eff2400b0a0 tx=0x7eff2400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 -- 172.21.15.66:0/3106115101 shutdown_connections 2023-07-21T22:20:26.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 --2- 172.21.15.66:0/3106115101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c100c60 0x7eff2c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 --2- 172.21.15.66:0/3106115101 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c10a180 0x7eff2c10c5a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 --2- 172.21.15.66:0/3106115101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c101570 0x7eff2c109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 -- 172.21.15.66:0/3106115101 >> 172.21.15.66:0/3106115101 conn(0x7eff2c075b70 msgr2=0x7eff2c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:26.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 -- 172.21.15.66:0/3106115101 shutdown_connections 2023-07-21T22:20:26.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.603+0000 7eff319e3700 1 -- 172.21.15.66:0/3106115101 wait complete. 2023-07-21T22:20:26.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 Processor -- start 2023-07-21T22:20:26.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 -- start start 2023-07-21T22:20:26.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 0x7eff2c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:26.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c101570 0x7eff2c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:26.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c10a180 0x7eff2c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:26.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c19d010 con 0x7eff2c100c60 2023-07-21T22:20:26.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c19d180 con 0x7eff2c10a180 2023-07-21T22:20:26.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.604+0000 7eff319e3700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c19d340 con 0x7eff2c101570 2023-07-21T22:20:26.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 0x7eff2c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:26.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 0x7eff2c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39752/0 (socket says 172.21.15.66:39752) 2023-07-21T22:20:26.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 -- 172.21.15.66:0/1975073309 learned_addr learned my addr 172.21.15.66:0/1975073309 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:26.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2a7fc700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c101570 0x7eff2c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:26.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2b7fe700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c10a180 0x7eff2c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 -- 172.21.15.66:0/1975073309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c101570 msgr2=0x7eff2c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c101570 0x7eff2c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 -- 172.21.15.66:0/1975073309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c10a180 msgr2=0x7eff2c19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:26.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c10a180 0x7eff2c19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 -- 172.21.15.66:0/1975073309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24009d00 con 0x7eff2c100c60 2023-07-21T22:20:26.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.605+0000 7eff2affd700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 0x7eff2c198050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00e9a0 tx=0x7eff1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:26.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.606+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c009790 con 0x7eff2c100c60 2023-07-21T22:20:26.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.606+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2c19d620 con 0x7eff2c100c60 2023-07-21T22:20:26.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.606+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff1c00cbd0 con 0x7eff2c100c60 2023-07-21T22:20:26.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.606+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c018680 con 0x7eff2c100c60 2023-07-21T22:20:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.606+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff2c1a4e80 con 0x7eff2c100c60 2023-07-21T22:20:26.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.607+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2c107790 con 0x7eff2c100c60 2023-07-21T22:20:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.609+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7eff1c00cd40 con 0x7eff2c100c60 2023-07-21T22:20:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.610+0000 7eff309e1700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff140a11e0 0x7eff140a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:26.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.611+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7eff1c014070 con 0x7eff2c100c60 2023-07-21T22:20:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.611+0000 7eff2a7fc700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff140a11e0 0x7eff140a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.611+0000 7eff2a7fc700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff140a11e0 0x7eff140a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff2c199670 tx=0x7eff20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:26.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.614+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7eff1c051770 con 0x7eff2c100c60 2023-07-21T22:20:26.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.866+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7eff2c107170 con 0x7eff2c100c60 2023-07-21T22:20:26.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.866+0000 7eff309e1700 1 -- 172.21.15.66:0/1975073309 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7eff1c096330 con 0x7eff2c100c60 2023-07-21T22:20:26.911 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:26.911 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-21T22:20:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.868+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff140a11e0 msgr2=0x7eff140a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.868+0000 7eff319e3700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff140a11e0 0x7eff140a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7eff2c199670 tx=0x7eff20009260 comp rx=0 tx=0).stop 2023-07-21T22:20:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.868+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 msgr2=0x7eff2c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.868+0000 7eff319e3700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 0x7eff2c198050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00e9a0 tx=0x7eff1c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 shutdown_connections 2023-07-21T22:20:26.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff140a11e0 0x7eff140a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff2c10a180 0x7eff2c19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff2c100c60 0x7eff2c198050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 --2- 172.21.15.66:0/1975073309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff2c101570 0x7eff2c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 >> 172.21.15.66:0/1975073309 conn(0x7eff2c075b70 msgr2=0x7eff2c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 shutdown_connections 2023-07-21T22:20:26.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:26.869+0000 7eff319e3700 1 -- 172.21.15.66:0/1975073309 wait complete. 2023-07-21T22:20:27.321 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-21T22:20:27.321 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:20:27.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 -- 172.21.15.66:0/4130392094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8100cf0 msgr2=0x7f7ca8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/4130392094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8101140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009990 tx=0x7f7c9c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 -- 172.21.15.66:0/4130392094 shutdown_connections 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/4130392094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ca8101680 0x7f7ca8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/4130392094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca80ffb00 0x7f7ca80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/4130392094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8101140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 -- 172.21.15.66:0/4130392094 >> 172.21.15.66:0/4130392094 conn(0x7f7ca80fb0e0 msgr2=0x7f7ca80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:27.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 -- 172.21.15.66:0/4130392094 shutdown_connections 2023-07-21T22:20:27.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.492+0000 7f7cac9ff700 1 -- 172.21.15.66:0/4130392094 wait complete. 2023-07-21T22:20:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.493+0000 7f7cac9ff700 1 Processor -- start 2023-07-21T22:20:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.493+0000 7f7cac9ff700 1 -- start start 2023-07-21T22:20:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7cac9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ca80ffb00 0x7f7ca80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7cac9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7cac9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 0x7f7ca8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7cac9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca81996a0 con 0x7f7ca8101680 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7cac9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8073380 con 0x7f7ca80ffb00 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7cac9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8073540 con 0x7f7ca8100cf0 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 0x7f7ca8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 0x7f7ca8072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39772/0 (socket says 172.21.15.66:39772) 2023-07-21T22:20:27.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca6d9d700 1 -- 172.21.15.66:0/1909119017 learned_addr learned my addr 172.21.15.66:0/1909119017 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca5d9b700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca659c700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ca80ffb00 0x7f7ca80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca5d9b700 1 -- 172.21.15.66:0/1909119017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ca80ffb00 msgr2=0x7f7ca80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca5d9b700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ca80ffb00 0x7f7ca80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca5d9b700 1 -- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 msgr2=0x7f7ca8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca5d9b700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 0x7f7ca8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca5d9b700 1 -- 172.21.15.66:0/1909119017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0009c40 con 0x7f7ca8100cf0 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.494+0000 7f7ca6d9d700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 0x7f7ca8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.495+0000 7f7ca5d9b700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8072900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00b620 tx=0x7f7c9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.495+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c01d070 con 0x7f7ca8100cf0 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.495+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c0095f0 con 0x7f7ca8100cf0 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.495+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca81a4230 con 0x7f7ca8100cf0 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.495+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9c021410 con 0x7f7ca8100cf0 2023-07-21T22:20:27.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.495+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c017650 con 0x7f7ca8100cf0 2023-07-21T22:20:27.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.496+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c88005320 con 0x7f7ca8100cf0 2023-07-21T22:20:27.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.498+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7c9c00fad0 con 0x7f7ca8100cf0 2023-07-21T22:20:27.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.500+0000 7f7c977fe700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7c900a12a0 0x7f7c900a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.500+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c9c0ce1f0 con 0x7f7ca8100cf0 2023-07-21T22:20:27.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.500+0000 7f7ca659c700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7c900a12a0 0x7f7c900a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.501+0000 7f7ca659c700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7c900a12a0 0x7f7c900a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8100b50 tx=0x7f7c98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:27.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.502+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7c9c097500 con 0x7f7ca8100cf0 2023-07-21T22:20:27.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:27 smithi104 ceph-mon[129021]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:27.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:27 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1975073309' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:20:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:27 smithi066 ceph-mon[126379]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:27.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:27 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1975073309' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:20:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:27 smithi149 ceph-mon[129268]: pgmap v316: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:27 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1975073309' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T22:20:27.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.713+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c88005190 con 0x7f7ca8100cf0 2023-07-21T22:20:27.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.715+0000 7f7c977fe700 1 -- 172.21.15.66:0/1909119017 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24442 (secure 0 0 0) 0x7f7c9c09ab20 con 0x7f7ca8100cf0 2023-07-21T22:20:27.759 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:27.760 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:37.301201+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:20:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7c900a12a0 msgr2=0x7f7c900a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7c900a12a0 0x7f7c900a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8100b50 tx=0x7f7c98009260 comp rx=0 tx=0).stop 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca8100cf0 msgr2=0x7f7ca8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8072900 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00b620 tx=0x7f7c9c004730 comp rx=0 tx=0).stop 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 shutdown_connections 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7c900a12a0 0x7f7c900a3750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ca80ffb00 0x7f7ca80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7ca8101680 0x7f7ca8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 --2- 172.21.15.66:0/1909119017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7ca8100cf0 0x7f7ca8072900 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 >> 172.21.15.66:0/1909119017 conn(0x7f7ca80fb0e0 msgr2=0x7f7ca8104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 shutdown_connections 2023-07-21T22:20:27.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.717+0000 7f7cac9ff700 1 -- 172.21.15.66:0/1909119017 wait complete. 2023-07-21T22:20:27.781 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:20:27.781 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:20:27.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 -- 172.21.15.66:0/1980119791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50074550 msgr2=0x7f7e500729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 --2- 172.21.15.66:0/1980119791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50074550 0x7f7e500729a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7e400098f0 tx=0x7f7e40009c00 comp rx=0 tx=0).stop 2023-07-21T22:20:27.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 -- 172.21.15.66:0/1980119791 shutdown_connections 2023-07-21T22:20:27.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 --2- 172.21.15.66:0/1980119791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e501079a0 0x7f7e50109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 --2- 172.21.15.66:0/1980119791 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50074550 0x7f7e500729a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 --2- 172.21.15.66:0/1980119791 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e50072f70 0x7f7e500733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 -- 172.21.15.66:0/1980119791 >> 172.21.15.66:0/1980119791 conn(0x7f7e500fb0e0 msgr2=0x7f7e500fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:27.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 -- 172.21.15.66:0/1980119791 shutdown_connections 2023-07-21T22:20:27.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.948+0000 7f7e555ff700 1 -- 172.21.15.66:0/1980119791 wait complete. 2023-07-21T22:20:27.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.949+0000 7f7e555ff700 1 Processor -- start 2023-07-21T22:20:27.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 -- start start 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50072f70 0x7f7e50197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e50074550 0x7f7e50198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 0x7f7e5019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5019f6b0 con 0x7f7e501079a0 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5019f7f0 con 0x7f7e50074550 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e555ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5019f980 con 0x7f7e50072f70 2023-07-21T22:20:27.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e4f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 0x7f7e5019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e4f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 0x7f7e5019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39784/0 (socket says 172.21.15.66:39784) 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e4f7fe700 1 -- 172.21.15.66:0/4000480622 learned_addr learned my addr 172.21.15.66:0/4000480622 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e4e7fc700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e50074550 0x7f7e50198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.950+0000 7f7e4effd700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50072f70 0x7f7e50197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e4f7fe700 1 -- 172.21.15.66:0/4000480622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50072f70 msgr2=0x7f7e50197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e4f7fe700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50072f70 0x7f7e50197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e4f7fe700 1 -- 172.21.15.66:0/4000480622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e50074550 msgr2=0x7f7e50198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e4f7fe700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e50074550 0x7f7e50198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e4f7fe700 1 -- 172.21.15.66:0/4000480622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e44009520 con 0x7f7e501079a0 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e4f7fe700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 0x7f7e5019f0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800e3c0 tx=0x7f7e4800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e480090d0 con 0x7f7e501079a0 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e48009d00 con 0x7f7e501079a0 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e4800f040 con 0x7f7e501079a0 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48007c20 con 0x7f7e501079a0 2023-07-21T22:20:27.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.951+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e40009520 con 0x7f7e501079a0 2023-07-21T22:20:27.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.952+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e50106050 con 0x7f7e501079a0 2023-07-21T22:20:27.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.954+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7e48003a00 con 0x7f7e501079a0 2023-07-21T22:20:28.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.956+0000 7f7e37fff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e380a11e0 0x7f7e380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.956+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7e480ccd30 con 0x7f7e501079a0 2023-07-21T22:20:28.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.956+0000 7f7e4effd700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e380a11e0 0x7f7e380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.957+0000 7f7e4effd700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e380a11e0 0x7f7e380a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009fd0 tx=0x7f7e40019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:28.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:27.960+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7e48096040 con 0x7f7e501079a0 2023-07-21T22:20:28.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.173+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e5002ca20 con 0x7f7e501079a0 2023-07-21T22:20:28.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.174+0000 7f7e37fff700 1 -- 172.21.15.66:0/4000480622 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7e48099660 con 0x7f7e501079a0 2023-07-21T22:20:28.219 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:20:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e380a11e0 msgr2=0x7f7e380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e380a11e0 0x7f7e380a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e40009fd0 tx=0x7f7e40019040 comp rx=0 tx=0).stop 2023-07-21T22:20:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 msgr2=0x7f7e5019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 0x7f7e5019f0e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800e3c0 tx=0x7f7e4800e6d0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 shutdown_connections 2023-07-21T22:20:28.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f7e380a11e0 0x7f7e380a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7e501079a0 0x7f7e5019f0e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e50072f70 0x7f7e50197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 --2- 172.21.15.66:0/4000480622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7e50074550 0x7f7e50198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 >> 172.21.15.66:0/4000480622 conn(0x7f7e500fb0e0 msgr2=0x7f7e500fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.176+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 shutdown_connections 2023-07-21T22:20:28.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.177+0000 7f7e555ff700 1 -- 172.21.15.66:0/4000480622 wait complete. 2023-07-21T22:20:28.241 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:20:28.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.407+0000 7f060edff700 1 -- 172.21.15.66:0/4087742426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06081005d0 msgr2=0x7f06081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.407+0000 7f060edff700 1 --2- 172.21.15.66:0/4087742426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06081005d0 0x7f06081029b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009990 tx=0x7f05fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 -- 172.21.15.66:0/4087742426 shutdown_connections 2023-07-21T22:20:28.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 --2- 172.21.15.66:0/4087742426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06081005d0 0x7f06081029b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 --2- 172.21.15.66:0/4087742426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 0x7f0608107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 --2- 172.21.15.66:0/4087742426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0608102ef0 0x7f06081052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 -- 172.21.15.66:0/4087742426 >> 172.21.15.66:0/4087742426 conn(0x7f06080fa070 msgr2=0x7f06080fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:28.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 -- 172.21.15.66:0/4087742426 shutdown_connections 2023-07-21T22:20:28.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.408+0000 7f060edff700 1 -- 172.21.15.66:0/4087742426 wait complete. 2023-07-21T22:20:28.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.409+0000 7f060edff700 1 Processor -- start 2023-07-21T22:20:28.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.409+0000 7f060edff700 1 -- start start 2023-07-21T22:20:28.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060edff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06081005d0 0x7f0608197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060edff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0608102ef0 0x7f0608197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060edff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 0x7f060819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060edff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819f5b0 con 0x7f0608105810 2023-07-21T22:20:28.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060edff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819f6f0 con 0x7f0608102ef0 2023-07-21T22:20:28.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060edff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819f880 con 0x7f06081005d0 2023-07-21T22:20:28.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060d39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 0x7f060819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060cb9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06081005d0 0x7f0608197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f0607fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0608102ef0 0x7f0608197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060d39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 0x7f060819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39812/0 (socket says 172.21.15.66:39812) 2023-07-21T22:20:28.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.410+0000 7f060d39c700 1 -- 172.21.15.66:0/1486271208 learned_addr learned my addr 172.21.15.66:0/1486271208 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:28.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060cb9b700 1 -- 172.21.15.66:0/1486271208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0608102ef0 msgr2=0x7f0608197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060cb9b700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0608102ef0 0x7f0608197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060cb9b700 1 -- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 msgr2=0x7f060819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060cb9b700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 0x7f060819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060cb9b700 1 -- 172.21.15.66:0/1486271208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f4009520 con 0x7f06081005d0 2023-07-21T22:20:28.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060cb9b700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06081005d0 0x7f0608197a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098e0 tx=0x7f05fc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:28.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc01d070 con 0x7f06081005d0 2023-07-21T22:20:28.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc021410 con 0x7f06081005d0 2023-07-21T22:20:28.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc0095f0 con 0x7f06081005d0 2023-07-21T22:20:28.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.411+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005720 con 0x7f06081005d0 2023-07-21T22:20:28.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.412+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060819fd90 con 0x7f06081005d0 2023-07-21T22:20:28.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.412+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06081073d0 con 0x7f06081005d0 2023-07-21T22:20:28.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.415+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f05fc021580 con 0x7f06081005d0 2023-07-21T22:20:28.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.416+0000 7f0605ffb700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f05f80a11e0 0x7f05f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.416+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f05fc0ce2a0 con 0x7f06081005d0 2023-07-21T22:20:28.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.416+0000 7f0607fff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f05f80a11e0 0x7f05f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.417+0000 7f0607fff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f05f80a11e0 0x7f05f80a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009df0 tx=0x7f05f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:28.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.418+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f05fc0975b0 con 0x7f06081005d0 2023-07-21T22:20:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:28 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1909119017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:28.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:28 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4000480622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:28.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.628+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f060802ccd0 con 0x7f06081005d0 2023-07-21T22:20:28.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.629+0000 7f0605ffb700 1 -- 172.21.15.66:0/1486271208 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f060802ccd0 con 0x7f06081005d0 2023-07-21T22:20:28.673 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:20:28.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f05f80a11e0 msgr2=0x7f05f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f05f80a11e0 0x7f05f80a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009df0 tx=0x7f05f4009260 comp rx=0 tx=0).stop 2023-07-21T22:20:28.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06081005d0 msgr2=0x7f0608197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06081005d0 0x7f0608197a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098e0 tx=0x7f05fc009580 comp rx=0 tx=0).stop 2023-07-21T22:20:28.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 shutdown_connections 2023-07-21T22:20:28.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f05f80a11e0 0x7f05f80a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0608105810 0x7f060819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06081005d0 0x7f0608197a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 --2- 172.21.15.66:0/1486271208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0608102ef0 0x7f0608197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 >> 172.21.15.66:0/1486271208 conn(0x7f06080fa070 msgr2=0x7f06080fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:28.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.631+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 shutdown_connections 2023-07-21T22:20:28.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.632+0000 7f060edff700 1 -- 172.21.15.66:0/1486271208 wait complete. 2023-07-21T22:20:28.692 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:28 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1909119017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:28.693 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:28 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4000480622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:28.695 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:20:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:28 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1909119017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:28 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4000480622' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 -- 172.21.15.66:0/3796755200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c0ffaf0 msgr2=0x7fc26c0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 --2- 172.21.15.66:0/3796755200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0fff00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc268009990 tx=0x7fc268009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 -- 172.21.15.66:0/3796755200 shutdown_connections 2023-07-21T22:20:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 --2- 172.21.15.66:0/3796755200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0fff00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 --2- 172.21.15.66:0/3796755200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc26c101670 0x7fc26c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 --2- 172.21.15.66:0/3796755200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c100ce0 0x7fc26c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 -- 172.21.15.66:0/3796755200 >> 172.21.15.66:0/3796755200 conn(0x7fc26c0fb130 msgr2=0x7fc26c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.862+0000 7fc2735ff700 1 -- 172.21.15.66:0/3796755200 shutdown_connections 2023-07-21T22:20:28.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.863+0000 7fc2735ff700 1 -- 172.21.15.66:0/3796755200 wait complete. 2023-07-21T22:20:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.864+0000 7fc2735ff700 1 Processor -- start 2023-07-21T22:20:28.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.864+0000 7fc2735ff700 1 -- start start 2023-07-21T22:20:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc2735ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc2735ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c100ce0 0x7fc26c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc2735ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc26c101670 0x7fc26c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc2735ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c0735e0 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc2735ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c073750 con 0x7fc26c100ce0 2023-07-21T22:20:28.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc2735ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c19f8f0 con 0x7fc26c101670 2023-07-21T22:20:28.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc27139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc27139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39832/0 (socket says 172.21.15.66:39832) 2023-07-21T22:20:28.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc27139b700 1 -- 172.21.15.66:0/1889805543 learned_addr learned my addr 172.21.15.66:0/1889805543 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:28.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc270b9a700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c100ce0 0x7fc26c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.865+0000 7fc271b9c700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc26c101670 0x7fc26c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc27139b700 1 -- 172.21.15.66:0/1889805543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc26c101670 msgr2=0x7fc26c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc27139b700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc26c101670 0x7fc26c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc27139b700 1 -- 172.21.15.66:0/1889805543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c100ce0 msgr2=0x7fc26c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc27139b700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c100ce0 0x7fc26c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:28.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc27139b700 1 -- 172.21.15.66:0/1889805543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680095f0 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc27139b700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0742c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc268004730 tx=0x7fc268004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26801d070 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc268021410 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c19fb70 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268017650 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.866+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c19ff90 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.867+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c106a70 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.869+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc268021580 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.871+0000 7fc2627fc700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2580a11e0 0x7fc2580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:28.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.871+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc2680ce1f0 con 0x7fc26c0ffaf0 2023-07-21T22:20:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.871+0000 7fc270b9a700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2580a11e0 0x7fc2580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.872+0000 7fc270b9a700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2580a11e0 0x7fc2580a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009c60 tx=0x7fc25c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:28.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:28.875+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc268097500 con 0x7fc26c0ffaf0 2023-07-21T22:20:29.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.087+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc26c02ca70 con 0x7fc26c0ffaf0 2023-07-21T22:20:29.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.087+0000 7fc2627fc700 1 -- 172.21.15.66:0/1889805543 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc26809ab20 con 0x7fc26c0ffaf0 2023-07-21T22:20:29.132 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:20:29.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2580a11e0 msgr2=0x7fc2580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2580a11e0 0x7fc2580a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009c60 tx=0x7fc25c009260 comp rx=0 tx=0).stop 2023-07-21T22:20:29.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 msgr2=0x7fc26c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0742c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc268004730 tx=0x7fc268004760 comp rx=0 tx=0).stop 2023-07-21T22:20:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 shutdown_connections 2023-07-21T22:20:29.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc2580a11e0 0x7fc2580a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc26c101670 0x7fc26c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc26c0ffaf0 0x7fc26c0742c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 --2- 172.21.15.66:0/1889805543 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc26c100ce0 0x7fc26c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 >> 172.21.15.66:0/1889805543 conn(0x7fc26c0fb130 msgr2=0x7fc26c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 shutdown_connections 2023-07-21T22:20:29.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.090+0000 7fc2735ff700 1 -- 172.21.15.66:0/1889805543 wait complete. 2023-07-21T22:20:29.153 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:20:29.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 -- 172.21.15.66:0/2260617209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc0ffae0 msgr2=0x7f93dc0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 --2- 172.21.15.66:0/2260617209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc0ffae0 0x7f93dc0ffef0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0098f0 tx=0x7f93cc009c00 comp rx=0 tx=0).stop 2023-07-21T22:20:29.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 -- 172.21.15.66:0/2260617209 shutdown_connections 2023-07-21T22:20:29.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 --2- 172.21.15.66:0/2260617209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 --2- 172.21.15.66:0/2260617209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc0ffae0 0x7f93dc0ffef0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 --2- 172.21.15.66:0/2260617209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93dc100cd0 0x7f93dc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 -- 172.21.15.66:0/2260617209 >> 172.21.15.66:0/2260617209 conn(0x7f93dc0fb0e0 msgr2=0x7f93dc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.319+0000 7f93e13ff700 1 -- 172.21.15.66:0/2260617209 shutdown_connections 2023-07-21T22:20:29.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.320+0000 7f93e13ff700 1 -- 172.21.15.66:0/2260617209 wait complete. 2023-07-21T22:20:29.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.320+0000 7f93e13ff700 1 Processor -- start 2023-07-21T22:20:29.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 -- start start 2023-07-21T22:20:29.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93dc0ffae0 0x7f93dc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc100cd0 0x7f93dc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc19f600 con 0x7f93dc101660 2023-07-21T22:20:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc19f740 con 0x7f93dc0ffae0 2023-07-21T22:20:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93e13ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc19f8d0 con 0x7f93dc100cd0 2023-07-21T22:20:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93da7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc100cd0 0x7f93dc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93db7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc19f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39844/0 (socket says 172.21.15.66:39844) 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93db7fe700 1 -- 172.21.15.66:0/117834845 learned_addr learned my addr 172.21.15.66:0/117834845 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.321+0000 7f93daffd700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93dc0ffae0 0x7f93dc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93db7fe700 1 -- 172.21.15.66:0/117834845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc100cd0 msgr2=0x7f93dc197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93db7fe700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc100cd0 0x7f93dc197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93db7fe700 1 -- 172.21.15.66:0/117834845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93dc0ffae0 msgr2=0x7f93dc197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93db7fe700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93dc0ffae0 0x7f93dc197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93db7fe700 1 -- 172.21.15.66:0/117834845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00095f0 con 0x7f93dc101660 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93db7fe700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc19f030 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93d400c370 tx=0x7f93d400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4017070 con 0x7f93dc101660 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d400f040 con 0x7f93dc101660 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d4009c40 con 0x7f93dc101660 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d400e050 con 0x7f93dc101660 2023-07-21T22:20:29.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.322+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc009520 con 0x7f93dc101660 2023-07-21T22:20:29.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.323+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93dc192160 con 0x7f93dc101660 2023-07-21T22:20:29.371 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.327+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f93d400cb60 con 0x7f93dc101660 2023-07-21T22:20:29.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.328+0000 7f93c3fff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93c40a1260 0x7f93c40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.329+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93d40ccd80 con 0x7f93dc101660 2023-07-21T22:20:29.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.329+0000 7f93daffd700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93c40a1260 0x7f93c40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.329+0000 7f93daffd700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93c40a1260 0x7f93c40a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009fd0 tx=0x7f93cc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:29.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.331+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f93d4096000 con 0x7f93dc101660 2023-07-21T22:20:29.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:29 smithi066 ceph-mon[126379]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:29.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:29 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1486271208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:29.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:29 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1889805543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:29.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.540+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f93dc02c9e0 con 0x7f93dc101660 2023-07-21T22:20:29.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.540+0000 7f93c3fff700 1 -- 172.21.15.66:0/117834845 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f93d4099620 con 0x7f93dc101660 2023-07-21T22:20:29.585 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:20:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93c40a1260 msgr2=0x7f93c40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93c40a1260 0x7f93c40a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009fd0 tx=0x7f93cc019040 comp rx=0 tx=0).stop 2023-07-21T22:20:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 msgr2=0x7f93dc19f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc19f030 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93d400c370 tx=0x7f93d400c680 comp rx=0 tx=0).stop 2023-07-21T22:20:29.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 shutdown_connections 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f93c40a1260 0x7f93c40a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f93dc0ffae0 0x7f93dc197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93dc101660 0x7f93dc19f030 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 --2- 172.21.15.66:0/117834845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93dc100cd0 0x7f93dc197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 >> 172.21.15.66:0/117834845 conn(0x7f93dc0fb0e0 msgr2=0x7f93dc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 shutdown_connections 2023-07-21T22:20:29.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.543+0000 7f93e13ff700 1 -- 172.21.15.66:0/117834845 wait complete. 2023-07-21T22:20:29.606 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:20:29.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:29 smithi104 ceph-mon[129021]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:29.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:29 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1486271208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:29.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:29 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1889805543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:29.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:29 smithi149 ceph-mon[129268]: pgmap v317: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:29.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:29 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1486271208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:29.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:29 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1889805543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.768+0000 7efc241ff700 1 -- 172.21.15.66:0/1370690746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c0ffb00 msgr2=0x7efc1c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.768+0000 7efc241ff700 1 --2- 172.21.15.66:0/1370690746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c0ffb00 0x7efc1c0fff10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc18009990 tx=0x7efc18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 -- 172.21.15.66:0/1370690746 shutdown_connections 2023-07-21T22:20:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 --2- 172.21.15.66:0/1370690746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1c101680 0x7efc1c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 --2- 172.21.15.66:0/1370690746 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c0ffb00 0x7efc1c0fff10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 --2- 172.21.15.66:0/1370690746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c100cf0 0x7efc1c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 -- 172.21.15.66:0/1370690746 >> 172.21.15.66:0/1370690746 conn(0x7efc1c0fb0e0 msgr2=0x7efc1c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 -- 172.21.15.66:0/1370690746 shutdown_connections 2023-07-21T22:20:29.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.769+0000 7efc241ff700 1 -- 172.21.15.66:0/1370690746 wait complete. 2023-07-21T22:20:29.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 Processor -- start 2023-07-21T22:20:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 -- start start 2023-07-21T22:20:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1c0ffb00 0x7efc1c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 0x7efc1c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c101680 0x7efc1c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c0735e0 con 0x7efc1c100cf0 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c073750 con 0x7efc1c0ffb00 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc241ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c19f8b0 con 0x7efc1c101680 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc2179a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 0x7efc1c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc2179a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 0x7efc1c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39864/0 (socket says 172.21.15.66:39864) 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc2179a700 1 -- 172.21.15.66:0/2635925629 learned_addr learned my addr 172.21.15.66:0/2635925629 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.770+0000 7efc21f9b700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1c0ffb00 0x7efc1c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2279c700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c101680 0x7efc1c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2179a700 1 -- 172.21.15.66:0/2635925629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c101680 msgr2=0x7efc1c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2179a700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c101680 0x7efc1c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2179a700 1 -- 172.21.15.66:0/2635925629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1c0ffb00 msgr2=0x7efc1c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2179a700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1c0ffb00 0x7efc1c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2179a700 1 -- 172.21.15.66:0/2635925629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc180095f0 con 0x7efc1c100cf0 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2279c700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c101680 0x7efc1c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:29.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc2179a700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 0x7efc1c072900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00e9a0 tx=0x7efc0c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c009790 con 0x7efc1c100cf0 2023-07-21T22:20:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc0c00cbd0 con 0x7efc1c100cf0 2023-07-21T22:20:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0c018680 con 0x7efc1c100cf0 2023-07-21T22:20:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.771+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1c19fb90 con 0x7efc1c100cf0 2023-07-21T22:20:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.772+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1c1a0050 con 0x7efc1c100cf0 2023-07-21T22:20:29.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.773+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1c10b320 con 0x7efc1c100cf0 2023-07-21T22:20:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.775+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efc0c00cd40 con 0x7efc1c100cf0 2023-07-21T22:20:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.776+0000 7efc12ffd700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc080a12a0 0x7efc080a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.776+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc0c014070 con 0x7efc1c100cf0 2023-07-21T22:20:29.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.776+0000 7efc21f9b700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc080a12a0 0x7efc080a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:29.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.777+0000 7efc21f9b700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc080a12a0 0x7efc080a3750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b2e0 tx=0x7efc18005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:29.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:29.780+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efc0c051770 con 0x7efc1c100cf0 2023-07-21T22:20:30.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.035+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc1c1069b0 con 0x7efc1c100cf0 2023-07-21T22:20:30.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.036+0000 7efc12ffd700 1 -- 172.21.15.66:0/2635925629 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7efc0c0962b0 con 0x7efc1c100cf0 2023-07-21T22:20:30.080 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:30.080 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:20:30.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc080a12a0 msgr2=0x7efc080a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc080a12a0 0x7efc080a3750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b2e0 tx=0x7efc18005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 msgr2=0x7efc1c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 0x7efc1c072900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00e9a0 tx=0x7efc0c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 shutdown_connections 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc080a12a0 0x7efc080a3750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1c0ffb00 0x7efc1c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc1c101680 0x7efc1c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 --2- 172.21.15.66:0/2635925629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc1c100cf0 0x7efc1c072900 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.038+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 >> 172.21.15.66:0/2635925629 conn(0x7efc1c0fb0e0 msgr2=0x7efc1c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:30.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.039+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 shutdown_connections 2023-07-21T22:20:30.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.039+0000 7efc241ff700 1 -- 172.21.15.66:0/2635925629 wait complete. 2023-07-21T22:20:30.084 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:20:30.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:20:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.265+0000 7ff2649ff700 1 -- 172.21.15.66:0/2669167453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff260100cf0 msgr2=0x7ff260101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.265+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2669167453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff260100cf0 0x7ff260101140 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff254009990 tx=0x7ff254009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 -- 172.21.15.66:0/2669167453 shutdown_connections 2023-07-21T22:20:30.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2669167453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2600ffb00 0x7ff2600fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2669167453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff260101680 0x7ff260107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2669167453 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff260100cf0 0x7ff260101140 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 -- 172.21.15.66:0/2669167453 >> 172.21.15.66:0/2669167453 conn(0x7ff2600fb0e0 msgr2=0x7ff2600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 -- 172.21.15.66:0/2669167453 shutdown_connections 2023-07-21T22:20:30.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.266+0000 7ff2649ff700 1 -- 172.21.15.66:0/2669167453 wait complete. 2023-07-21T22:20:30.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 Processor -- start 2023-07-21T22:20:30.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 -- start start 2023-07-21T22:20:30.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2600ffb00 0x7ff2601979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 0x7ff260197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff260101680 0x7ff26019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26019f570 con 0x7ff2600ffb00 2023-07-21T22:20:30.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26019f6b0 con 0x7ff260100cf0 2023-07-21T22:20:30.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.267+0000 7ff2649ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26019f840 con 0x7ff260101680 2023-07-21T22:20:30.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2600ffb00 0x7ff2601979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 0x7ff260197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 0x7ff260197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42704/0 (socket says 172.21.15.66:42704) 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 -- 172.21.15.66:0/2109862698 learned_addr learned my addr 172.21.15.66:0/2109862698 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25ed9d700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff260101680 0x7ff26019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 -- 172.21.15.66:0/2109862698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff260101680 msgr2=0x7ff26019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff260101680 0x7ff26019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 -- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2600ffb00 msgr2=0x7ff2601979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2600ffb00 0x7ff2601979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 -- 172.21.15.66:0/2109862698 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250009520 con 0x7ff260100cf0 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.268+0000 7ff25dd9b700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 0x7ff260197f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff254005740 tx=0x7ff254005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.269+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25401d070 con 0x7ff260100cf0 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.269+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff254021410 con 0x7ff260100cf0 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.269+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254017650 con 0x7ff260100cf0 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.269+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540095f0 con 0x7ff260100cf0 2023-07-21T22:20:30.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.269+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26019fd90 con 0x7ff260100cf0 2023-07-21T22:20:30.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.270+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26010b2d0 con 0x7ff260100cf0 2023-07-21T22:20:30.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.272+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7ff254021580 con 0x7ff260100cf0 2023-07-21T22:20:30.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.273+0000 7ff24f7fe700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff2480a1220 0x7ff2480a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.273+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2540ce0e0 con 0x7ff260100cf0 2023-07-21T22:20:30.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.273+0000 7ff25e59c700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff2480a1220 0x7ff2480a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.274+0000 7ff25e59c700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff2480a1220 0x7ff2480a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff260100b50 tx=0x7ff250009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:30.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.277+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff2540973f0 con 0x7ff260100cf0 2023-07-21T22:20:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:30 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/117834845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:30.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:30 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2635925629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:30.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.490+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff26019fa00 con 0x7ff260100cf0 2023-07-21T22:20:30.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.491+0000 7ff24f7fe700 1 -- 172.21.15.66:0/2109862698 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24442 (secure 0 0 0) 0x7ff25409aa10 con 0x7ff260100cf0 2023-07-21T22:20:30.536 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:30.536 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:37.301201+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:20:30.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff2480a1220 msgr2=0x7ff2480a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff2480a1220 0x7ff2480a36d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff260100b50 tx=0x7ff250009260 comp rx=0 tx=0).stop 2023-07-21T22:20:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 msgr2=0x7ff260197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 0x7ff260197f30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff254005740 tx=0x7ff254005770 comp rx=0 tx=0).stop 2023-07-21T22:20:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 shutdown_connections 2023-07-21T22:20:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ff2480a1220 0x7ff2480a36d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff260101680 0x7ff26019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2600ffb00 0x7ff2601979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 --2- 172.21.15.66:0/2109862698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff260100cf0 0x7ff260197f30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 >> 172.21.15.66:0/2109862698 conn(0x7ff2600fb0e0 msgr2=0x7ff2600fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 shutdown_connections 2023-07-21T22:20:30.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.494+0000 7ff2649ff700 1 -- 172.21.15.66:0/2109862698 wait complete. 2023-07-21T22:20:30.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:20:30.558 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:20:30.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:30 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/117834845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:30.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:30 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2635925629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:30.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:30 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/117834845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:30.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:30 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2635925629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:30.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 -- 172.21.15.66:0/3526225216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788100cd0 msgr2=0x7f0788101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 --2- 172.21.15.66:0/3526225216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788100cd0 0x7f0788101120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0770009990 tx=0x7f0770009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 -- 172.21.15.66:0/3526225216 shutdown_connections 2023-07-21T22:20:30.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 --2- 172.21.15.66:0/3526225216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0788101660 0x7f0788107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 --2- 172.21.15.66:0/3526225216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07880ffae0 0x7f07880ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 --2- 172.21.15.66:0/3526225216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788100cd0 0x7f0788101120 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 -- 172.21.15.66:0/3526225216 >> 172.21.15.66:0/3526225216 conn(0x7f07880fb0e0 msgr2=0x7f07880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:30.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 -- 172.21.15.66:0/3526225216 shutdown_connections 2023-07-21T22:20:30.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.725+0000 7f078cdff700 1 -- 172.21.15.66:0/3526225216 wait complete. 2023-07-21T22:20:30.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.726+0000 7f078cdff700 1 Processor -- start 2023-07-21T22:20:30.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.726+0000 7f078cdff700 1 -- start start 2023-07-21T22:20:30.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078cdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07880ffae0 0x7f0788197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078cdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0788100cd0 0x7f0788197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078cdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 0x7f078819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078cdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819f610 con 0x7f0788101660 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078cdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819f750 con 0x7f07880ffae0 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078cdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819f8e0 con 0x7f0788100cd0 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f0786d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 0x7f078819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f0786d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 0x7f078819f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39890/0 (socket says 172.21.15.66:39890) 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f0786d9d700 1 -- 172.21.15.66:0/2434926072 learned_addr learned my addr 172.21.15.66:0/2434926072 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f077ffff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0788100cd0 0x7f0788197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.727+0000 7f078659c700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07880ffae0 0x7f0788197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f0786d9d700 1 -- 172.21.15.66:0/2434926072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0788100cd0 msgr2=0x7f0788197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f0786d9d700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0788100cd0 0x7f0788197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f0786d9d700 1 -- 172.21.15.66:0/2434926072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07880ffae0 msgr2=0x7f0788197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f0786d9d700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07880ffae0 0x7f0788197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f0786d9d700 1 -- 172.21.15.66:0/2434926072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700095f0 con 0x7f0788101660 2023-07-21T22:20:30.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f0786d9d700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 0x7f078819f040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f078000ef30 tx=0x7f078000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:30.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780018070 con 0x7f0788101660 2023-07-21T22:20:30.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f078000f040 con 0x7f0788101660 2023-07-21T22:20:30.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078819fb60 con 0x7f0788101660 2023-07-21T22:20:30.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000cdb0 con 0x7f0788101660 2023-07-21T22:20:30.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.728+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07881a00c0 con 0x7f0788101660 2023-07-21T22:20:30.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.729+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0788192170 con 0x7f0788101660 2023-07-21T22:20:30.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.731+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0780003a00 con 0x7f0788101660 2023-07-21T22:20:30.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.733+0000 7f077f7fe700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f07740a1260 0x7f07740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:30.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.733+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f07800cc8e0 con 0x7f0788101660 2023-07-21T22:20:30.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.733+0000 7f078659c700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f07740a1260 0x7f07740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:30.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.734+0000 7f078659c700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f07740a1260 0x7f07740a3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0788100b30 tx=0x7f0778009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:30.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.736+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0780095bf0 con 0x7f0788101660 2023-07-21T22:20:30.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.947+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f078802ca70 con 0x7f0788101660 2023-07-21T22:20:30.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.947+0000 7f077f7fe700 1 -- 172.21.15.66:0/2434926072 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0780099210 con 0x7f0788101660 2023-07-21T22:20:30.992 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:20:30.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.949+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f07740a1260 msgr2=0x7f07740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.949+0000 7f078cdff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f07740a1260 0x7f07740a3710 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0788100b30 tx=0x7f0778009260 comp rx=0 tx=0).stop 2023-07-21T22:20:30.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.949+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 msgr2=0x7f078819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:30.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.949+0000 7f078cdff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 0x7f078819f040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f078000ef30 tx=0x7f078000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 shutdown_connections 2023-07-21T22:20:30.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f07740a1260 0x7f07740a3710 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07880ffae0 0x7f0788197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0788101660 0x7f078819f040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 --2- 172.21.15.66:0/2434926072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0788100cd0 0x7f0788197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:30.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 >> 172.21.15.66:0/2434926072 conn(0x7f07880fb0e0 msgr2=0x7f07881065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:30.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 shutdown_connections 2023-07-21T22:20:30.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:30.950+0000 7f078cdff700 1 -- 172.21.15.66:0/2434926072 wait complete. 2023-07-21T22:20:31.013 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:20:31.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 -- 172.21.15.66:0/3902047911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98101660 msgr2=0x7fba98107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3902047911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98101660 0x7fba98107f20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fba9000b0a0 tx=0x7fba9000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 -- 172.21.15.66:0/3902047911 shutdown_connections 2023-07-21T22:20:31.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3902047911 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98101660 0x7fba98107f20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3902047911 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba980ffae0 0x7fba980ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3902047911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba98100cd0 0x7fba98101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 -- 172.21.15.66:0/3902047911 >> 172.21.15.66:0/3902047911 conn(0x7fba980fb0e0 msgr2=0x7fba980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:31.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 -- 172.21.15.66:0/3902047911 shutdown_connections 2023-07-21T22:20:31.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.179+0000 7fba9ddff700 1 -- 172.21.15.66:0/3902047911 wait complete. 2023-07-21T22:20:31.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.180+0000 7fba9ddff700 1 Processor -- start 2023-07-21T22:20:31.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.180+0000 7fba9ddff700 1 -- start start 2023-07-21T22:20:31.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.180+0000 7fba9ddff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba980ffae0 0x7fba980742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba9ddff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 0x7fba98072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba9ddff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba98101660 0x7fba98072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba9ddff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980735e0 con 0x7fba98100cd0 2023-07-21T22:20:31.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba9ddff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98073750 con 0x7fba980ffae0 2023-07-21T22:20:31.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba9ddff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819f9c0 con 0x7fba98101660 2023-07-21T22:20:31.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 0x7fba98072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 0x7fba98072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39912/0 (socket says 172.21.15.66:39912) 2023-07-21T22:20:31.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 -- 172.21.15.66:0/3650620842 learned_addr learned my addr 172.21.15.66:0/3650620842 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:31.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba97fff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba98101660 0x7fba98072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba977fe700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba980ffae0 0x7fba980742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 -- 172.21.15.66:0/3650620842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba98101660 msgr2=0x7fba98072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba98101660 0x7fba98072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 -- 172.21.15.66:0/3650620842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba980ffae0 msgr2=0x7fba980742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba980ffae0 0x7fba980742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba96ffd700 1 -- 172.21.15.66:0/3650620842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88009520 con 0x7fba98100cd0 2023-07-21T22:20:31.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.181+0000 7fba97fff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba98101660 0x7fba98072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:31.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.182+0000 7fba96ffd700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 0x7fba98072900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e9a0 tx=0x7fba8c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:31.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.182+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c009790 con 0x7fba98100cd0 2023-07-21T22:20:31.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.182+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba90009d00 con 0x7fba98100cd0 2023-07-21T22:20:31.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.182+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba8c010430 con 0x7fba98100cd0 2023-07-21T22:20:31.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.182+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c018680 con 0x7fba98100cd0 2023-07-21T22:20:31.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.182+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981a0040 con 0x7fba98100cd0 2023-07-21T22:20:31.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.183+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9810b310 con 0x7fba98100cd0 2023-07-21T22:20:31.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.185+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fba8c0105a0 con 0x7fba98100cd0 2023-07-21T22:20:31.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.186+0000 7fba94ff9700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fba800a11e0 0x7fba800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.187+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba8c014070 con 0x7fba98100cd0 2023-07-21T22:20:31.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.187+0000 7fba977fe700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fba800a11e0 0x7fba800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.187+0000 7fba977fe700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fba800a11e0 0x7fba800a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba98100b30 tx=0x7fba88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:31.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.190+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fba8c051770 con 0x7fba98100cd0 2023-07-21T22:20:31.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.405+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba98106940 con 0x7fba98100cd0 2023-07-21T22:20:31.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.405+0000 7fba94ff9700 1 -- 172.21.15.66:0/3650620842 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba8c096240 con 0x7fba98100cd0 2023-07-21T22:20:31.450 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:20:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.407+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fba800a11e0 msgr2=0x7fba800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.407+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fba800a11e0 0x7fba800a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba98100b30 tx=0x7fba88009260 comp rx=0 tx=0).stop 2023-07-21T22:20:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.407+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 msgr2=0x7fba98072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.407+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 0x7fba98072900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00e9a0 tx=0x7fba8c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 shutdown_connections 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fba800a11e0 0x7fba800a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba980ffae0 0x7fba980742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba98101660 0x7fba98072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 --2- 172.21.15.66:0/3650620842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fba98100cd0 0x7fba98072900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 >> 172.21.15.66:0/3650620842 conn(0x7fba980fb0e0 msgr2=0x7fba980fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 shutdown_connections 2023-07-21T22:20:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.408+0000 7fba9ddff700 1 -- 172.21.15.66:0/3650620842 wait complete. 2023-07-21T22:20:31.471 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:20:31.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:31 smithi104 ceph-mon[129021]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:31.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2109862698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:31.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2434926072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:31.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.636+0000 7f430fbff700 1 -- 172.21.15.66:0/213133471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 msgr2=0x7f43080733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.636+0000 7f430fbff700 1 --2- 172.21.15.66:0/213133471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f43080733e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f42f8009990 tx=0x7f42f8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.636+0000 7f430fbff700 1 -- 172.21.15.66:0/213133471 shutdown_connections 2023-07-21T22:20:31.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.636+0000 7f430fbff700 1 --2- 172.21.15.66:0/213133471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 0x7f43080729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.637+0000 7f430fbff700 1 --2- 172.21.15.66:0/213133471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43081079a0 0x7f4308109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.637+0000 7f430fbff700 1 --2- 172.21.15.66:0/213133471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f43080733e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.637+0000 7f430fbff700 1 -- 172.21.15.66:0/213133471 >> 172.21.15.66:0/213133471 conn(0x7f43080fb0e0 msgr2=0x7f43080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:31.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.637+0000 7f430fbff700 1 -- 172.21.15.66:0/213133471 shutdown_connections 2023-07-21T22:20:31.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.637+0000 7f430fbff700 1 -- 172.21.15.66:0/213133471 wait complete. 2023-07-21T22:20:31.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 Processor -- start 2023-07-21T22:20:31.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 -- start start 2023-07-21T22:20:31.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f4308197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 0x7f4308197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43081079a0 0x7f430819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819f630 con 0x7f4308072f70 2023-07-21T22:20:31.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819f770 con 0x7f4308074550 2023-07-21T22:20:31.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.638+0000 7f430fbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819f900 con 0x7f43081079a0 2023-07-21T22:20:31.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d19a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 0x7f4308197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f4308197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d19a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 0x7f4308197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42752/0 (socket says 172.21.15.66:42752) 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f4308197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:39932/0 (socket says 172.21.15.66:39932) 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d19a700 1 -- 172.21.15.66:0/2678930372 learned_addr learned my addr 172.21.15.66:0/2678930372 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430e19c700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43081079a0 0x7f430819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 -- 172.21.15.66:0/2678930372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43081079a0 msgr2=0x7f430819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43081079a0 0x7f430819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 -- 172.21.15.66:0/2678930372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 msgr2=0x7f4308197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 0x7f4308197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 -- 172.21.15.66:0/2678930372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4304009520 con 0x7f4308072f70 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f430d99b700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f4308197ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9c0 tx=0x7f430400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.639+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430400cb70 con 0x7f4308072f70 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.640+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f430400ee60 con 0x7f4308072f70 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.640+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304003c60 con 0x7f4308072f70 2023-07-21T22:20:31.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.640+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f80095f0 con 0x7f4308072f70 2023-07-21T22:20:31.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.640+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430819ff10 con 0x7f4308072f70 2023-07-21T22:20:31.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.641+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4308106050 con 0x7f4308072f70 2023-07-21T22:20:31.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.643+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4304003e00 con 0x7f4308072f70 2023-07-21T22:20:31.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.644+0000 7f42feffd700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42f40a11e0 0x7f42f40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:31.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.645+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4304014070 con 0x7f4308072f70 2023-07-21T22:20:31.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.645+0000 7f430d19a700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42f40a11e0 0x7f42f40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:31.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.645+0000 7f430d19a700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42f40a11e0 0x7f42f40a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42f800b2e0 tx=0x7f42f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:31.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.648+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4304096360 con 0x7f4308072f70 2023-07-21T22:20:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:31 smithi066 ceph-mon[126379]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2109862698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:31.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2434926072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:31.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:31 smithi149 ceph-mon[129268]: pgmap v318: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:31.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2109862698' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:31.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2434926072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:31.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.859+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f430819fac0 con 0x7f4308072f70 2023-07-21T22:20:31.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.859+0000 7f42feffd700 1 -- 172.21.15.66:0/2678930372 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4304099980 con 0x7f4308072f70 2023-07-21T22:20:31.904 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:20:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.861+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42f40a11e0 msgr2=0x7f42f40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.861+0000 7f430fbff700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42f40a11e0 0x7f42f40a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42f800b2e0 tx=0x7f42f8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.861+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 msgr2=0x7f4308197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.861+0000 7f430fbff700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f4308197ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f430400e9c0 tx=0x7f430400ecd0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 shutdown_connections 2023-07-21T22:20:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42f40a11e0 0x7f42f40a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f43081079a0 0x7f430819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4308072f70 0x7f4308197ab0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 --2- 172.21.15.66:0/2678930372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4308074550 0x7f4308197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 >> 172.21.15.66:0/2678930372 conn(0x7f43080fb0e0 msgr2=0x7f43080fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 shutdown_connections 2023-07-21T22:20:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:31.862+0000 7f430fbff700 1 -- 172.21.15.66:0/2678930372 wait complete. 2023-07-21T22:20:31.925 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:20:32.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 -- 172.21.15.66:0/160971668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75940ffae0 msgr2=0x7f75940ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 --2- 172.21.15.66:0/160971668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75940ffae0 0x7f75940ffef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75840098f0 tx=0x7f7584009c00 comp rx=0 tx=0).stop 2023-07-21T22:20:32.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 -- 172.21.15.66:0/160971668 shutdown_connections 2023-07-21T22:20:32.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 --2- 172.21.15.66:0/160971668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75940ffae0 0x7f75940ffef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 --2- 172.21.15.66:0/160971668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7594101660 0x7f7594107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 --2- 172.21.15.66:0/160971668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7594100cd0 0x7f7594101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 -- 172.21.15.66:0/160971668 >> 172.21.15.66:0/160971668 conn(0x7f75940fb0e0 msgr2=0x7f75940fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:32.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 -- 172.21.15.66:0/160971668 shutdown_connections 2023-07-21T22:20:32.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.092+0000 7f7599bff700 1 -- 172.21.15.66:0/160971668 wait complete. 2023-07-21T22:20:32.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.093+0000 7f7599bff700 1 Processor -- start 2023-07-21T22:20:32.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 -- start start 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75940ffae0 0x7f7594197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7594100cd0 0x7f7594198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 0x7f759419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759419f640 con 0x7f7594100cd0 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759419f780 con 0x7f7594101660 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7599bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759419f910 con 0x7f75940ffae0 2023-07-21T22:20:32.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7593fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 0x7f759419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7593fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 0x7f759419f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53558/0 (socket says 172.21.15.66:53558) 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f7593fff700 1 -- 172.21.15.66:0/3013451197 learned_addr learned my addr 172.21.15.66:0/3013451197 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f758bfff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7594100cd0 0x7f7594198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.094+0000 7f75937fe700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75940ffae0 0x7f7594197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7593fff700 1 -- 172.21.15.66:0/3013451197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75940ffae0 msgr2=0x7f7594197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7593fff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75940ffae0 0x7f7594197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7593fff700 1 -- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7594100cd0 msgr2=0x7f7594198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7593fff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7594100cd0 0x7f7594198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7593fff700 1 -- 172.21.15.66:0/3013451197 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7584009520 con 0x7f7594101660 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7593fff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 0x7f759419f070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f758c00c110 tx=0x7f758c00c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c017070 con 0x7f7594101660 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759419fb90 con 0x7f7594101660 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f758c00e050 con 0x7f7594101660 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c0078f0 con 0x7f7594101660 2023-07-21T22:20:32.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.095+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75941a00a0 con 0x7f7594101660 2023-07-21T22:20:32.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.096+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7594192170 con 0x7f7594101660 2023-07-21T22:20:32.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.099+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f758c007a50 con 0x7f7594101660 2023-07-21T22:20:32.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.100+0000 7f75917fa700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f75800a11e0 0x7f75800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.100+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f758c0cd250 con 0x7f7594101660 2023-07-21T22:20:32.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.100+0000 7f75937fe700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f75800a11e0 0x7f75800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.101+0000 7f75937fe700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f75800a11e0 0x7f75800a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75840098c0 tx=0x7f7584019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:32.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.102+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f758c0964d0 con 0x7f7594101660 2023-07-21T22:20:32.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.318+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f759402c9e0 con 0x7f7594101660 2023-07-21T22:20:32.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.319+0000 7f75917fa700 1 -- 172.21.15.66:0/3013451197 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f758c099af0 con 0x7f7594101660 2023-07-21T22:20:32.363 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:20:32.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f75800a11e0 msgr2=0x7f75800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f75800a11e0 0x7f75800a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75840098c0 tx=0x7f7584019040 comp rx=0 tx=0).stop 2023-07-21T22:20:32.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 msgr2=0x7f759419f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 0x7f759419f070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f758c00c110 tx=0x7f758c00c420 comp rx=0 tx=0).stop 2023-07-21T22:20:32.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 shutdown_connections 2023-07-21T22:20:32.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f75800a11e0 0x7f75800a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7594101660 0x7f759419f070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75940ffae0 0x7f7594197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 --2- 172.21.15.66:0/3013451197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7594100cd0 0x7f7594198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.321+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 >> 172.21.15.66:0/3013451197 conn(0x7f75940fb0e0 msgr2=0x7f7594106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:32.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.322+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 shutdown_connections 2023-07-21T22:20:32.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.322+0000 7f7599bff700 1 -- 172.21.15.66:0/3013451197 wait complete. 2023-07-21T22:20:32.385 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:20:32.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 -- 172.21.15.66:0/2690451064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 msgr2=0x7fe538109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 --2- 172.21.15.66:0/2690451064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe538109e40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe53000b0a0 tx=0x7fe53000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 -- 172.21.15.66:0/2690451064 shutdown_connections 2023-07-21T22:20:32.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 --2- 172.21.15.66:0/2690451064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe538074550 0x7fe5380729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 --2- 172.21.15.66:0/2690451064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe538109e40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 --2- 172.21.15.66:0/2690451064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe538072f70 0x7fe5380733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 -- 172.21.15.66:0/2690451064 >> 172.21.15.66:0/2690451064 conn(0x7fe5380fb0e0 msgr2=0x7fe5380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:32.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 -- 172.21.15.66:0/2690451064 shutdown_connections 2023-07-21T22:20:32.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.548+0000 7fe53cdff700 1 -- 172.21.15.66:0/2690451064 wait complete. 2023-07-21T22:20:32.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.549+0000 7fe53cdff700 1 Processor -- start 2023-07-21T22:20:32.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.549+0000 7fe53cdff700 1 -- start start 2023-07-21T22:20:32.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53cdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe538072f70 0x7fe538197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53cdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe538074550 0x7fe538198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53cdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe53819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53cdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53819f6c0 con 0x7fe5381079c0 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53cdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53819f800 con 0x7fe538072f70 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53cdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53819f990 con 0x7fe538074550 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe53819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe53819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49100/0 (socket says 172.21.15.66:49100) 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 -- 172.21.15.66:0/3858757524 learned_addr learned my addr 172.21.15.66:0/3858757524 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe52ffff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe538074550 0x7fe538198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe53659c700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe538072f70 0x7fe538197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 -- 172.21.15.66:0/3858757524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe538074550 msgr2=0x7fe538198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe538074550 0x7fe538198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 -- 172.21.15.66:0/3858757524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe538072f70 msgr2=0x7fe538197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe538072f70 0x7fe538197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.550+0000 7fe536d9d700 1 -- 172.21.15.66:0/3858757524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530009d00 con 0x7fe5381079c0 2023-07-21T22:20:32.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.551+0000 7fe536d9d700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe53819f0f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fd0 tx=0x7fe53000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:32.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.551+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000e040 con 0x7fe5381079c0 2023-07-21T22:20:32.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.551+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53819fc10 con 0x7fe5381079c0 2023-07-21T22:20:32.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.551+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5300094a0 con 0x7fe5381079c0 2023-07-21T22:20:32.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.551+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53001b6c0 con 0x7fe5381079c0 2023-07-21T22:20:32.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.551+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5381a0030 con 0x7fe5381079c0 2023-07-21T22:20:32.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.552+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe538106070 con 0x7fe5381079c0 2023-07-21T22:20:32.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.554+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe53001d470 con 0x7fe5381079c0 2023-07-21T22:20:32.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.556+0000 7fe52f7fe700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5240a12a0 0x7fe5240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:32.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.556+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5300cdfb0 con 0x7fe5381079c0 2023-07-21T22:20:32.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.556+0000 7fe53659c700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5240a12a0 0x7fe5240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:32.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.557+0000 7fe53659c700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5240a12a0 0x7fe5240a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe538074920 tx=0x7fe528009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:32.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.560+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe5300972c0 con 0x7fe5381079c0 2023-07-21T22:20:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:32 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3650620842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:32.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:32 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2678930372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:32.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:32 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3650620842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:32.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:32 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2678930372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:32.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:32 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3650620842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:32.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:32 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2678930372' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:32.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.819+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe53802cc70 con 0x7fe5381079c0 2023-07-21T22:20:32.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.820+0000 7fe52f7fe700 1 -- 172.21.15.66:0/3858757524 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7fe53009a8e0 con 0x7fe5381079c0 2023-07-21T22:20:32.864 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:32.864 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:20:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5240a12a0 msgr2=0x7fe5240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5240a12a0 0x7fe5240a3750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe538074920 tx=0x7fe528009260 comp rx=0 tx=0).stop 2023-07-21T22:20:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 msgr2=0x7fe53819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:32.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe53819f0f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe530009fd0 tx=0x7fe53000bce0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 shutdown_connections 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5240a12a0 0x7fe5240a3750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe538072f70 0x7fe538197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5381079c0 0x7fe53819f0f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 --2- 172.21.15.66:0/3858757524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe538074550 0x7fe538198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 >> 172.21.15.66:0/3858757524 conn(0x7fe5380fb0e0 msgr2=0x7fe5380fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:32.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 shutdown_connections 2023-07-21T22:20:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:32.822+0000 7fe53cdff700 1 -- 172.21.15.66:0/3858757524 wait complete. 2023-07-21T22:20:32.868 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:20:32.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:20:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 -- 172.21.15.66:0/2314465489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28000ffb00 msgr2=0x7f28000fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 --2- 172.21.15.66:0/2314465489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28000ffb00 0x7f28000fff10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f27f0009990 tx=0x7f27f0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 -- 172.21.15.66:0/2314465489 shutdown_connections 2023-07-21T22:20:33.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 --2- 172.21.15.66:0/2314465489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2800101680 0x7f2800107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 --2- 172.21.15.66:0/2314465489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f28000ffb00 0x7f28000fff10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 --2- 172.21.15.66:0/2314465489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2800100cf0 0x7f2800101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 -- 172.21.15.66:0/2314465489 >> 172.21.15.66:0/2314465489 conn(0x7f28000fb0e0 msgr2=0x7f28000fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:33.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 -- 172.21.15.66:0/2314465489 shutdown_connections 2023-07-21T22:20:33.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.047+0000 7f28055ff700 1 -- 172.21.15.66:0/2314465489 wait complete. 2023-07-21T22:20:33.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.048+0000 7f28055ff700 1 Processor -- start 2023-07-21T22:20:33.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 -- start start 2023-07-21T22:20:33.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000ffb00 0x7f2800197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 0x7f2800197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2800101680 0x7f280019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280019f5d0 con 0x7f2800100cf0 2023-07-21T22:20:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280019f710 con 0x7f28000ffb00 2023-07-21T22:20:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f28055ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280019f8a0 con 0x7f2800101680 2023-07-21T22:20:33.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f27fe7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 0x7f2800197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f27fe7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 0x7f2800197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49112/0 (socket says 172.21.15.66:49112) 2023-07-21T22:20:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f27fe7fc700 1 -- 172.21.15.66:0/2654059752 learned_addr learned my addr 172.21.15.66:0/2654059752 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f27feffd700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000ffb00 0x7f2800197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.049+0000 7f27ff7fe700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2800101680 0x7f280019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27fe7fc700 1 -- 172.21.15.66:0/2654059752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2800101680 msgr2=0x7f280019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27fe7fc700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2800101680 0x7f280019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27fe7fc700 1 -- 172.21.15.66:0/2654059752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000ffb00 msgr2=0x7f2800197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27fe7fc700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000ffb00 0x7f2800197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27fe7fc700 1 -- 172.21.15.66:0/2654059752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f00095f0 con 0x7f2800100cf0 2023-07-21T22:20:33.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27fe7fc700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 0x7f2800197f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27f400e9a0 tx=0x7f27f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:33.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4009790 con 0x7f2800100cf0 2023-07-21T22:20:33.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f4010430 con 0x7f2800100cf0 2023-07-21T22:20:33.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280019fb80 con 0x7f2800100cf0 2023-07-21T22:20:33.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4018680 con 0x7f2800100cf0 2023-07-21T22:20:33.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.050+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28001a00e0 con 0x7f2800100cf0 2023-07-21T22:20:33.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.051+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28001921e0 con 0x7f2800100cf0 2023-07-21T22:20:33.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.053+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f27f40105a0 con 0x7f2800100cf0 2023-07-21T22:20:33.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.055+0000 7f27e7fff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27e80a12a0 0x7f27e80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.055+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f27f4014070 con 0x7f2800100cf0 2023-07-21T22:20:33.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.055+0000 7f27feffd700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27e80a12a0 0x7f27e80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.056+0000 7f27feffd700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27e80a12a0 0x7f27e80a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f27f000b2e0 tx=0x7f27f0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:33.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.059+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f27f4051770 con 0x7f2800100cf0 2023-07-21T22:20:33.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.273+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f280002c850 con 0x7f2800100cf0 2023-07-21T22:20:33.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.275+0000 7f27e7fff700 1 -- 172.21.15.66:0/2654059752 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24442 (secure 0 0 0) 0x7f27f40961e0 con 0x7f2800100cf0 2023-07-21T22:20:33.320 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:33.320 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":91,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:19:37.301201+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:20:33.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27e80a12a0 msgr2=0x7f27e80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27e80a12a0 0x7f27e80a3750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f27f000b2e0 tx=0x7f27f0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 msgr2=0x7f2800197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 0x7f2800197f90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f27f400e9a0 tx=0x7f27f400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 shutdown_connections 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f27e80a12a0 0x7f27e80a3750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28000ffb00 0x7f2800197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2800101680 0x7f280019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 --2- 172.21.15.66:0/2654059752 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2800100cf0 0x7f2800197f90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 >> 172.21.15.66:0/2654059752 conn(0x7f28000fb0e0 msgr2=0x7f28001065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 shutdown_connections 2023-07-21T22:20:33.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.278+0000 7f28055ff700 1 -- 172.21.15.66:0/2654059752 wait complete. 2023-07-21T22:20:33.342 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:20:33.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 -- 172.21.15.66:0/3941456824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 msgr2=0x7f14ec0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 --2- 172.21.15.66:0/3941456824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec0733e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0098f0 tx=0x7f14dc009c00 comp rx=0 tx=0).stop 2023-07-21T22:20:33.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 -- 172.21.15.66:0/3941456824 shutdown_connections 2023-07-21T22:20:33.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 --2- 172.21.15.66:0/3941456824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ec1079c0 0x7f14ec109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 --2- 172.21.15.66:0/3941456824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14ec074550 0x7f14ec0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 --2- 172.21.15.66:0/3941456824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec0733e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 -- 172.21.15.66:0/3941456824 >> 172.21.15.66:0/3941456824 conn(0x7f14ec0fb0e0 msgr2=0x7f14ec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:33.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.505+0000 7f14f45ff700 1 -- 172.21.15.66:0/3941456824 shutdown_connections 2023-07-21T22:20:33.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.506+0000 7f14f45ff700 1 -- 172.21.15.66:0/3941456824 wait complete. 2023-07-21T22:20:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.506+0000 7f14f45ff700 1 Processor -- start 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 -- start start 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ec074550 0x7f14ec197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14ec1079c0 0x7f14ec19f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19f630 con 0x7f14ec072f70 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19f770 con 0x7f14ec074550 2023-07-21T22:20:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f45ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec19f900 con 0x7f14ec1079c0 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f239b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f239b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49130/0 (socket says 172.21.15.66:49130) 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f239b700 1 -- 172.21.15.66:0/321084050 learned_addr learned my addr 172.21.15.66:0/321084050 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f1b9a700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ec074550 0x7f14ec197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.507+0000 7f14f2b9c700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14ec1079c0 0x7f14ec19f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f239b700 1 -- 172.21.15.66:0/321084050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14ec1079c0 msgr2=0x7f14ec19f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f239b700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14ec1079c0 0x7f14ec19f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f239b700 1 -- 172.21.15.66:0/321084050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ec074550 msgr2=0x7f14ec197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f239b700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ec074550 0x7f14ec197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f239b700 1 -- 172.21.15.66:0/321084050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009520 con 0x7f14ec072f70 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f239b700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec197ab0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e970 tx=0x7f14e800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8009790 con 0x7f14ec072f70 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14e800c7e0 con 0x7f14ec072f70 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e80054e0 con 0x7f14ec072f70 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec19fbe0 con 0x7f14ec072f70 2023-07-21T22:20:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.508+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec1a00a0 con 0x7f14ec072f70 2023-07-21T22:20:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.509+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec106070 con 0x7f14ec072f70 2023-07-21T22:20:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.511+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f14e800c950 con 0x7f14ec072f70 2023-07-21T22:20:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.513+0000 7f14e37fe700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14d80a1220 0x7f14d80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.513+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f14e8014070 con 0x7f14ec072f70 2023-07-21T22:20:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.513+0000 7f14f1b9a700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14d80a1220 0x7f14d80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:33.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.514+0000 7f14f1b9a700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14d80a1220 0x7f14d80a36d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009fd0 tx=0x7f14dc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:33.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.517+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f14e80962b0 con 0x7f14ec072f70 2023-07-21T22:20:33.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:33 smithi104 ceph-mon[129021]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:33 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3013451197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:33.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:33 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3858757524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:33.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:33 smithi066 ceph-mon[126379]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:33 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3013451197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:33.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:33 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3858757524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:33 smithi149 ceph-mon[129268]: pgmap v319: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:33 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3013451197' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:33.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:33 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3858757524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:33.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.770+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ec02cac0 con 0x7f14ec072f70 2023-07-21T22:20:33.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.770+0000 7f14e37fe700 1 -- 172.21.15.66:0/321084050 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f14e80998d0 con 0x7f14ec072f70 2023-07-21T22:20:33.815 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:33.815 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:20:33.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14d80a1220 msgr2=0x7f14d80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14d80a1220 0x7f14d80a36d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009fd0 tx=0x7f14dc019040 comp rx=0 tx=0).stop 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 msgr2=0x7f14ec197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec197ab0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14e800e970 tx=0x7f14e800ec80 comp rx=0 tx=0).stop 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 shutdown_connections 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f14d80a1220 0x7f14d80a36d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14ec1079c0 0x7f14ec19f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f14ec072f70 0x7f14ec197ab0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 --2- 172.21.15.66:0/321084050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14ec074550 0x7f14ec197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:33.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 >> 172.21.15.66:0/321084050 conn(0x7f14ec0fb0e0 msgr2=0x7f14ec0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:33.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 shutdown_connections 2023-07-21T22:20:33.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:33.773+0000 7f14f45ff700 1 -- 172.21.15.66:0/321084050 wait complete. 2023-07-21T22:20:33.819 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:20:33.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T22:19:06.899289+0000', 'modified': '2023-07-21T22:19:34.299634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24530, 'mds_2': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1642617037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 1642617037}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1642617037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/1273148910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 1273148910}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 1273148910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/1037928792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 1037928792}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 1037928792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.149:6839/2751755690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2751755690}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2751755690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.104:6837/1237242254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 1237242254}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 1237242254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/1299576617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1299576617}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1299576617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T22:20:33.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-21T22:20:33.838 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T22:20:34.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/2634155904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 msgr2=0x7f5c540fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/2634155904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c540fff00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009990 tx=0x7f5c44009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/2634155904 shutdown_connections 2023-07-21T22:20:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/2634155904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c540fff00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/2634155904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54101670 0x7f5c54107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/2634155904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c54100ce0 0x7f5c54101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/2634155904 >> 172.21.15.66:0/2634155904 conn(0x7f5c540fb130 msgr2=0x7f5c540fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/2634155904 shutdown_connections 2023-07-21T22:20:34.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.002+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/2634155904 wait complete. 2023-07-21T22:20:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.003+0000 7f5c5a7ff700 1 Processor -- start 2023-07-21T22:20:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.003+0000 7f5c5a7ff700 1 -- start start 2023-07-21T22:20:34.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c5a7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c5a7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c5a7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c5a7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f5a0 con 0x7f5c54101670 2023-07-21T22:20:34.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c5a7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f6e0 con 0x7f5c540ffaf0 2023-07-21T22:20:34.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c5a7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419f870 con 0x7f5c54100ce0 2023-07-21T22:20:34.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43650/0 (socket says 172.21.15.66:43650) 2023-07-21T22:20:34.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c537fe700 1 -- 172.21.15.66:0/1341956821 learned_addr learned my addr 172.21.15.66:0/1341956821 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c53fff700 1 -- 172.21.15.66:0/1341956821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 msgr2=0x7f5c54197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c53fff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c53fff700 1 -- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c54101670 msgr2=0x7f5c5419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c53fff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c53fff700 1 -- 172.21.15.66:0/1341956821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440095f0 con 0x7f5c540ffaf0 2023-07-21T22:20:34.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.004+0000 7f5c537fe700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.005+0000 7f5c53fff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c440059d0 tx=0x7f5c44005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.005+0000 7f5c517fa700 1 -- 172.21.15.66:0/1341956821 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4401d070 con 0x7f5c540ffaf0 2023-07-21T22:20:34.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.005+0000 7f5c517fa700 1 -- 172.21.15.66:0/1341956821 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c44022470 con 0x7f5c540ffaf0 2023-07-21T22:20:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.005+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5419faf0 con 0x7f5c540ffaf0 2023-07-21T22:20:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.005+0000 7f5c517fa700 1 -- 172.21.15.66:0/1341956821 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44017650 con 0x7f5c540ffaf0 2023-07-21T22:20:34.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.005+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5419ff10 con 0x7f5c540ffaf0 2023-07-21T22:20:34.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.009+0000 7f5c517fa700 1 -- 172.21.15.66:0/1341956821 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5c4400fad0 con 0x7f5c540ffaf0 2023-07-21T22:20:34.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.010+0000 7f5c6e031b80 1 Processor -- start 2023-07-21T22:20:34.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.010+0000 7f5c6e031b80 1 -- start start 2023-07-21T22:20:34.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7322bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d73213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d7321e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7226c60 con 0x55a8d73227e0 2023-07-21T22:20:34.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d72799f0 con 0x55a8d7323150 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7272e80 con 0x55a8d73219c0 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7322bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7322bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49148/0 (socket says 172.21.15.66:49148) 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c58d9c700 1 -- 172.21.15.66:0/3633475474 learned_addr learned my addr 172.21.15.66:0/3633475474 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c53fff700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d73213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c537fe700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d7321e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c517fa700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c3c0a11e0 0x7f5c3c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.011+0000 7f5c517fa700 1 -- 172.21.15.66:0/1341956821 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c440ce0f0 con 0x7f5c540ffaf0 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c537fe700 1 -- 172.21.15.66:0/3633475474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 msgr2=0x55a8d73213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c537fe700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d73213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c537fe700 1 -- 172.21.15.66:0/3633475474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 msgr2=0x55a8d7322bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c537fe700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7322bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c537fe700 1 -- 172.21.15.66:0/3633475474 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8d70a0d90 con 0x55a8d73219c0 2023-07-21T22:20:34.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c53fff700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d73213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c58d9c700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7322bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c58d9c700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c3c0a11e0 0x7f5c3c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c537fe700 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d7321e40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5c48011570 tx=0x7f5c48011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a2c0bc0bac2eb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c3a7fc700 1 -- 172.21.15.66:0/3633475474 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48004230 con 0x55a8d73219c0 2023-07-21T22:20:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c3a7fc700 1 -- 172.21.15.66:0/3633475474 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c48004390 con 0x55a8d73219c0 2023-07-21T22:20:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.012+0000 7f5c3a7fc700 1 -- 172.21.15.66:0/3633475474 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48019880 con 0x55a8d73219c0 2023-07-21T22:20:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 -- 172.21.15.66:0/3633475474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 msgr2=0x55a8d7321e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c58d9c700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c3c0a11e0 0x7f5c3c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55a8d7321370 tx=0x7f5c4c009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d7321e40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5c48011570 tx=0x7f5c48011880 comp rx=0 tx=0).stop 2023-07-21T22:20:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 -- 172.21.15.66:0/3633475474 shutdown_connections 2023-07-21T22:20:34.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d7321e40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7322bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/3633475474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d73213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 -- 172.21.15.66:0/3633475474 >> 172.21.15.66:0/3633475474 conn(0x55a8d7316ea0 msgr2=0x55a8d731c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 -- 172.21.15.66:0/3633475474 shutdown_connections 2023-07-21T22:20:34.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.013+0000 7f5c6e031b80 1 -- 172.21.15.66:0/3633475474 wait complete. 2023-07-21T22:20:34.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.014+0000 7f5c6e031b80 1 -- start start 2023-07-21T22:20:34.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.014+0000 7f5c6e031b80 1 Processor -- start 2023-07-21T22:20:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d73325a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7212a70 con 0x55a8d7323150 2023-07-21T22:20:34.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7212770 con 0x55a8d73227e0 2023-07-21T22:20:34.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d70f68d0 con 0x55a8d73219c0 2023-07-21T22:20:34.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c6e031b80 10 client.0 _ll_register_callbacks cb 0x55a8d7210710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:34.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d73325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d73325a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49162/0 (socket says 172.21.15.66:49162) 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c58d9c700 1 -- 172.21.15.66:0/1997769301 learned_addr learned my addr 172.21.15.66:0/1997769301 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.015+0000 7f5c537fe700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c53fff700 1 -- 172.21.15.66:0/1997769301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 msgr2=0x55a8d732b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c53fff700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c53fff700 1 -- 172.21.15.66:0/1997769301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 msgr2=0x55a8d73325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c53fff700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d73325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c53fff700 1 -- 172.21.15.66:0/1997769301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c011380 con 0x55a8d73219c0 2023-07-21T22:20:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c58d9c700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d73325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c53fff700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732b020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c440cb730 tx=0x7f5c440cbc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc63fc50df1e67 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c337fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:20:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c337fe700 1 -- 172.21.15.66:0/1997769301 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4409ac00 con 0x55a8d73219c0 2023-07-21T22:20:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c337fe700 1 -- 172.21.15.66:0/1997769301 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c4409f0f0 con 0x55a8d73219c0 2023-07-21T22:20:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a8d71cc4a0 con 0x55a8d73219c0 2023-07-21T22:20:34.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c337fe700 1 -- 172.21.15.66:0/1997769301 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c4409aa20 con 0x55a8d73219c0 2023-07-21T22:20:34.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c6e031b80 10 client.25117 fetch_fsmap learned FSMap version 23 2023-07-21T22:20:34.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.016+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f5c5419ff10 con 0x55a8d73219c0 2023-07-21T22:20:34.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c337fe700 1 -- 172.21.15.66:0/1997769301 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a8d71cc4a0 con 0x55a8d73219c0 2023-07-21T22:20:34.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c337fe700 1 -- 172.21.15.66:0/1997769301 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c440d0160 con 0x55a8d73219c0 2023-07-21T22:20:34.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c6e031b80 10 client.25117 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:20:34.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c6e031b80 10 client.25117 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:20:34.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c6e031b80 20 client.25117 populate_metadata read hostname 'smithi066' 2023-07-21T22:20:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d7335f50 0x55a8d73354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c6e031b80 4 client.25117 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8d726be60 con 0x55a8d7335f50 2023-07-21T22:20:34.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.017+0000 7f5c537fe700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d7335f50 0x55a8d73354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.018+0000 7f5c537fe700 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d7335f50 0x55a8d73354d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a45d68b34fb7e8e server_cookie=1e7e4191ec0d0b22 in_seq=0 out_seq=0 2023-07-21T22:20:34.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.018+0000 7f5c337fe700 10 client.25117 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:20:34.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c337fe700 1 -- 172.21.15.66:0/1997769301 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a8d726be60 con 0x55a8d7335f50 2023-07-21T22:20:34.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c337fe700 10 client.25117 handle_command_reply: tid=0 2023-07-21T22:20:34.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 client.25117 shutdown 2023-07-21T22:20:34.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 msgr2=0x55a8d732b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732b020 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c440cb730 tx=0x7f5c440cbc60 comp rx=0 tx=0).stop 2023-07-21T22:20:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 shutdown_connections 2023-07-21T22:20:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d7335f50 0x55a8d73354d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d73325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c337fe700 0 client.25117 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:20:34.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732b020 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/1997769301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 >> 172.21.15.66:0/1997769301 conn(0x55a8d7316ea0 msgr2=0x55a8d731b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 shutdown_connections 2023-07-21T22:20:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 1 -- 172.21.15.66:0/1997769301 wait complete. 2023-07-21T22:20:34.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.019+0000 7f5c6e031b80 20 client.25117 trim_cache size 0 max 16384 2023-07-21T22:20:34.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.038+0000 7f5c6e031b80 1 Processor -- start 2023-07-21T22:20:34.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.038+0000 7f5c6e031b80 1 -- start start 2023-07-21T22:20:34.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d73259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7332530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d7325ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d71cc4a0 con 0x55a8d73227e0 2023-07-21T22:20:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d721c470 con 0x55a8d7323150 2023-07-21T22:20:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7241790 con 0x55a8d73219c0 2023-07-21T22:20:34.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7332530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d73259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d73259b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43664/0 (socket says 172.21.15.66:43664) 2023-07-21T22:20:34.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 -- 172.21.15.66:0/2475915469 learned_addr learned my addr 172.21.15.66:0/2475915469 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c58d9c700 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d7325ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 -- 172.21.15.66:0/2475915469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 msgr2=0x55a8d7325ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d7325ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 -- 172.21.15.66:0/2475915469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 msgr2=0x55a8d7332530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7332530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c53fff700 1 -- 172.21.15.66:0/2475915469 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8d732a0c0 con 0x55a8d73219c0 2023-07-21T22:20:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.039+0000 7f5c537fe700 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7332530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.040+0000 7f5c53fff700 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d73259b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c44097ae0 tx=0x7f5c440ce330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd50fc9deab1532d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.040+0000 7f5c3affd700 1 -- 172.21.15.66:0/2475915469 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4409b5b0 con 0x55a8d73219c0 2023-07-21T22:20:34.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.040+0000 7f5c3affd700 1 -- 172.21.15.66:0/2475915469 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c4409aa20 con 0x55a8d73219c0 2023-07-21T22:20:34.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.040+0000 7f5c3affd700 1 -- 172.21.15.66:0/2475915469 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440a3480 con 0x55a8d73219c0 2023-07-21T22:20:34.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.040+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2475915469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 msgr2=0x55a8d73259b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.040+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d73259b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5c44097ae0 tx=0x7f5c440ce330 comp rx=0 tx=0).stop 2023-07-21T22:20:34.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2475915469 shutdown_connections 2023-07-21T22:20:34.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d7323150 0x55a8d7325ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d73259b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2475915469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d73227e0 0x55a8d7332530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2475915469 >> 172.21.15.66:0/2475915469 conn(0x55a8d7316ea0 msgr2=0x55a8d73354d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2475915469 shutdown_connections 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2475915469 wait complete. 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 -- start start 2023-07-21T22:20:34.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.041+0000 7f5c6e031b80 1 Processor -- start 2023-07-21T22:20:34.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d7330a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7330fc0 con 0x55a8d7323150 2023-07-21T22:20:34.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d7331130 con 0x55a8d73227e0 2023-07-21T22:20:34.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8d73312f0 con 0x55a8d73219c0 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c6e031b80 10 client.0 _ll_register_callbacks cb 0x7f5c44009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d7330a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c53fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c58d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d7330a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49186/0 (socket says 172.21.15.66:49186) 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c58d9c700 1 -- 172.21.15.66:0/2368539800 learned_addr learned my addr 172.21.15.66:0/2368539800 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53650/0 (socket says 172.21.15.66:53650) 2023-07-21T22:20:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 -- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 msgr2=0x55a8d732d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 -- 172.21.15.66:0/2368539800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 msgr2=0x55a8d7330a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d7330a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c537fe700 1 -- 172.21.15.66:0/2368539800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8d70a0d90 con 0x55a8d73227e0 2023-07-21T22:20:34.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.042+0000 7f5c58d9c700 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d7330a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c537fe700 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732da60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c48000c00 tx=0x7f5c48004210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68dc70941d55fa96 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c337fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:20:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c337fe700 1 -- 172.21.15.66:0/2368539800 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48029040 con 0x55a8d73227e0 2023-07-21T22:20:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c337fe700 1 -- 172.21.15.66:0/2368539800 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c48005ce0 con 0x55a8d73227e0 2023-07-21T22:20:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c48004390 con 0x55a8d73227e0 2023-07-21T22:20:34.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c337fe700 1 -- 172.21.15.66:0/2368539800 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c480051c0 con 0x55a8d73227e0 2023-07-21T22:20:34.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c6e031b80 10 client.25142 fetch_fsmap learned FSMap version 23 2023-07-21T22:20:34.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.043+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f5c5419ff10 con 0x55a8d73227e0 2023-07-21T22:20:34.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c337fe700 1 -- 172.21.15.66:0/2368539800 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c48004390 con 0x55a8d73227e0 2023-07-21T22:20:34.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c337fe700 1 -- 172.21.15.66:0/2368539800 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c480115a0 con 0x55a8d73227e0 2023-07-21T22:20:34.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c6e031b80 10 client.25142 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:20:34.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c6e031b80 10 client.25142 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:20:34.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c6e031b80 20 client.25142 populate_metadata read hostname 'smithi066' 2023-07-21T22:20:34.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d738f310 0x55a8d738be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c6e031b80 4 client.25142 mds_command: new command op to 24520 tid=0[{"prefix": "status"}] 2023-07-21T22:20:34.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a8d722e000 con 0x55a8d738f310 2023-07-21T22:20:34.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.044+0000 7f5c53fff700 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d738f310 0x55a8d738be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c53fff700 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d738f310 0x55a8d738be00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8eafbeab2d60fc55 server_cookie=13d7f60abf551630 in_seq=0 out_seq=0 2023-07-21T22:20:34.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c337fe700 10 client.25142 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:20:34.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c337fe700 1 -- 172.21.15.66:0/2368539800 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55a8d722e000 con 0x55a8d738f310 2023-07-21T22:20:34.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c337fe700 10 client.25142 handle_command_reply: tid=0 2023-07-21T22:20:34.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 client.25142 shutdown 2023-07-21T22:20:34.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 msgr2=0x55a8d732da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732da60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5c48000c00 tx=0x7f5c48004210 comp rx=0 tx=0).stop 2023-07-21T22:20:34.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 shutdown_connections 2023-07-21T22:20:34.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55a8d738f310 0x55a8d738be00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a8d7323150 0x55a8d7330a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c337fe700 0 client.25142 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:20:34.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a8d73219c0 0x55a8d732d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 --2- 172.21.15.66:0/2368539800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a8d73227e0 0x55a8d732da60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 >> 172.21.15.66:0/2368539800 conn(0x55a8d7316ea0 msgr2=0x55a8d73259b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.045+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 shutdown_connections 2023-07-21T22:20:34.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c6e031b80 1 -- 172.21.15.66:0/2368539800 wait complete. 2023-07-21T22:20:34.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c6e031b80 20 client.25142 trim_cache size 0 max 16384 2023-07-21T22:20:34.107 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:20:34.107 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:20:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2023-07-21T22:20:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24520, 2023-07-21T22:20:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:20:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:20:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:20:34.108 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 60.753871296, 2023-07-21T22:20:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 23, 2023-07-21T22:20:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 91, 2023-07-21T22:20:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:20:34.109 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 101.585685771 2023-07-21T22:20:34.109 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:20:34.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c3c0a11e0 msgr2=0x7f5c3c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c3c0a11e0 0x7f5c3c0a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55a8d7321370 tx=0x7f5c4c009040 comp rx=0 tx=0).stop 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 msgr2=0x7f5c54197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5c440059d0 tx=0x7f5c44005740 comp rx=0 tx=0).stop 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 shutdown_connections 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5c3c0a11e0 0x7f5c3c0a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c540ffaf0 0x7f5c54197a20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5c54101670 0x7f5c5419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 --2- 172.21.15.66:0/1341956821 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5c54100ce0 0x7f5c54197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 >> 172.21.15.66:0/1341956821 conn(0x7f5c540fb130 msgr2=0x7f5c541065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.046+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 shutdown_connections 2023-07-21T22:20:34.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.047+0000 7f5c5a7ff700 1 -- 172.21.15.66:0/1341956821 wait complete. 2023-07-21T22:20:34.112 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-21T22:20:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.283+0000 7f95ba7ff700 1 -- 172.21.15.66:0/2989457297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 msgr2=0x7f95b40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.283+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/2989457297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 0x7f95b40fff00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009990 tx=0x7f95a4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 -- 172.21.15.66:0/2989457297 shutdown_connections 2023-07-21T22:20:34.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/2989457297 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b4101670 0x7f95b4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/2989457297 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 0x7f95b40fff00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/2989457297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 -- 172.21.15.66:0/2989457297 >> 172.21.15.66:0/2989457297 conn(0x7f95b40fb130 msgr2=0x7f95b40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 -- 172.21.15.66:0/2989457297 shutdown_connections 2023-07-21T22:20:34.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.284+0000 7f95ba7ff700 1 -- 172.21.15.66:0/2989457297 wait complete. 2023-07-21T22:20:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 Processor -- start 2023-07-21T22:20:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 -- start start 2023-07-21T22:20:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 0x7f95b4195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b4101670 0x7f95b419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b419d3a0 con 0x7f95b4101670 2023-07-21T22:20:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b419d4e0 con 0x7f95b4100ce0 2023-07-21T22:20:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.285+0000 7f95ba7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b419d670 con 0x7f95b40ffaf0 2023-07-21T22:20:34.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53660/0 (socket says 172.21.15.66:53660) 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 -- 172.21.15.66:0/3337933064 learned_addr learned my addr 172.21.15.66:0/3337933064 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 0x7f95b4195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b4101670 0x7f95b419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 -- 172.21.15.66:0/3337933064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 msgr2=0x7f95b4195820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 0x7f95b4195820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 -- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b4101670 msgr2=0x7f95b419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b4101670 0x7f95b419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 -- 172.21.15.66:0/3337933064 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac009c40 con 0x7f95b4100ce0 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.286+0000 7f95b37fe700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4195d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e9a0 tx=0x7f95a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.287+0000 7f95b17fa700 1 -- 172.21.15.66:0/3337933064 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8009790 con 0x7f95b4100ce0 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.287+0000 7f95b17fa700 1 -- 172.21.15.66:0/3337933064 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95a800ce80 con 0x7f95b4100ce0 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.287+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a40095f0 con 0x7f95b4100ce0 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.287+0000 7f95b17fa700 1 -- 172.21.15.66:0/3337933064 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8018680 con 0x7f95b4100ce0 2023-07-21T22:20:34.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.287+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b419dd20 con 0x7f95b4100ce0 2023-07-21T22:20:34.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.290+0000 7f95b17fa700 1 -- 172.21.15.66:0/3337933064 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f95a8010430 con 0x7f95b4100ce0 2023-07-21T22:20:34.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.291+0000 7f95ce18bb80 1 Processor -- start 2023-07-21T22:20:34.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.291+0000 7f95b17fa700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f959c0a1260 0x7f959c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95b17fa700 1 -- 172.21.15.66:0/3337933064 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95a8014070 con 0x7f95b4100ce0 2023-07-21T22:20:34.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 -- start start 2023-07-21T22:20:34.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f959c0a1260 0x7f959c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ebce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ebd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ebe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28dbdc60 con 0x55df28ebd380 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28e109f0 con 0x55df28ebca30 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28e09e80 con 0x55df28ebdfa0 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ebd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95b3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ebce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ebd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49214/0 (socket says 172.21.15.66:49214) 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.292+0000 7f95b37fe700 1 -- 172.21.15.66:0/3345744656 learned_addr learned my addr 172.21.15.66:0/3345744656 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ebe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b37fe700 1 -- 172.21.15.66:0/3345744656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 msgr2=0x55df28ebe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b37fe700 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ebe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b37fe700 1 -- 172.21.15.66:0/3345744656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 msgr2=0x55df28ebce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b37fe700 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ebce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f959c0a1260 0x7f959c0a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009fd0 tx=0x7f95ac00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b37fe700 1 -- 172.21.15.66:0/3345744656 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df28c37d90 con 0x55df28ebd380 2023-07-21T22:20:34.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b3fff700 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ebce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f95b37fe700 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ebd9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95a80a1db0 tx=0x7f95a8099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be46384cef1e7d9f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f959a7fc700 1 -- 172.21.15.66:0/3345744656 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a809dec0 con 0x55df28ebd380 2023-07-21T22:20:34.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f959a7fc700 1 -- 172.21.15.66:0/3345744656 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95a80a22e0 con 0x55df28ebd380 2023-07-21T22:20:34.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.293+0000 7f959a7fc700 1 -- 172.21.15.66:0/3345744656 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80a1e50 con 0x55df28ebd380 2023-07-21T22:20:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 -- 172.21.15.66:0/3345744656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 msgr2=0x55df28ebd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ebd9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f95a80a1db0 tx=0x7f95a8099b00 comp rx=0 tx=0).stop 2023-07-21T22:20:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 -- 172.21.15.66:0/3345744656 shutdown_connections 2023-07-21T22:20:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ebce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ebe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/3345744656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ebd9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 -- 172.21.15.66:0/3345744656 >> 172.21.15.66:0/3345744656 conn(0x55df28eadea0 msgr2=0x55df28eb31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 -- 172.21.15.66:0/3345744656 shutdown_connections 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.294+0000 7f95ce18bb80 1 -- 172.21.15.66:0/3345744656 wait complete. 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.295+0000 7f95ce18bb80 1 -- start start 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.295+0000 7f95ce18bb80 1 Processor -- start 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ec64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ec6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ecda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28da9a70 con 0x55df28ebd380 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28da9770 con 0x55df28ebca30 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28c8d8d0 con 0x55df28ebdfa0 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95ce18bb80 10 client.0 _ll_register_callbacks cb 0x55df28da7710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ec6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b8d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ecda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ec64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b8d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ecda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43696/0 (socket says 172.21.15.66:43696) 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b8d9c700 1 -- 172.21.15.66:0/575482603 learned_addr learned my addr 172.21.15.66:0/575482603 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b37fe700 1 -- 172.21.15.66:0/575482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 msgr2=0x55df28ecda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b37fe700 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ecda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b37fe700 1 -- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 msgr2=0x55df28ec64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b37fe700 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ec64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b37fe700 1 -- 172.21.15.66:0/575482603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b4108860 con 0x55df28ebd380 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.296+0000 7f95b3fff700 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ec64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95b37fe700 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ec6a10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95a8099b90 tx=0x7f95a8095b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60861ce487e7ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95937fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95937fe700 1 -- 172.21.15.66:0/575482603 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80b82c0 con 0x55df28ebd380 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95937fe700 1 -- 172.21.15.66:0/575482603 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95a809dec0 con 0x55df28ebd380 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df28d634a0 con 0x55df28ebd380 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95937fe700 1 -- 172.21.15.66:0/575482603 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a80a2060 con 0x55df28ebd380 2023-07-21T22:20:34.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95ce18bb80 10 client.15093 fetch_fsmap learned FSMap version 23 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f95b419dd20 con 0x55df28ebd380 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95937fe700 1 -- 172.21.15.66:0/575482603 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95a807c070 con 0x55df28ebd380 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.297+0000 7f95937fe700 1 -- 172.21.15.66:0/575482603 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95a80adc20 con 0x55df28ebd380 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95ce18bb80 10 client.15093 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95ce18bb80 10 client.15093 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95ce18bb80 20 client.15093 populate_metadata read hostname 'smithi066' 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28ed19d0 0x55df28ed0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95ce18bb80 4 client.15093 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:34.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df28e02e60 con 0x55df28ed19d0 2023-07-21T22:20:34.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.298+0000 7f95b3fff700 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28ed19d0 0x55df28ed0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.299+0000 7f95937fe700 10 client.15093 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:20:34.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.299+0000 7f95b3fff700 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28ed19d0 0x55df28ed0980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a10af5d7a8fb092c server_cookie=2859c069976b19e in_seq=0 out_seq=0 2023-07-21T22:20:34.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95937fe700 1 -- 172.21.15.66:0/575482603 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55df28e02e60 con 0x55df28ed19d0 2023-07-21T22:20:34.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95937fe700 10 client.15093 handle_command_reply: tid=0 2023-07-21T22:20:34.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 client.15093 shutdown 2023-07-21T22:20:34.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 msgr2=0x55df28ec6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ec6a10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95a8099b90 tx=0x7f95a8095b30 comp rx=0 tx=0).stop 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 shutdown_connections 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28ed19d0 0x55df28ed0980 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ec64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebdfa0 0x55df28ecda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/575482603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebd380 0x55df28ec6a10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95937fe700 0 client.15093 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:20:34.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 >> 172.21.15.66:0/575482603 conn(0x55df28eadea0 msgr2=0x55df28ebb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 shutdown_connections 2023-07-21T22:20:34.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 1 -- 172.21.15.66:0/575482603 wait complete. 2023-07-21T22:20:34.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.300+0000 7f95ce18bb80 20 client.15093 trim_cache size 0 max 16384 2023-07-21T22:20:34.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.319+0000 7f95ce18bb80 1 Processor -- start 2023-07-21T22:20:34.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.319+0000 7f95ce18bb80 1 -- start start 2023-07-21T22:20:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 0x55df28ec4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebdfa0 0x55df28ec46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28e02e60 con 0x55df28ebca30 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28eb45c0 con 0x55df28ebdfa0 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28ec4c00 con 0x55df28ebd380 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 0x55df28ec4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 0x55df28ec4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49238/0 (socket says 172.21.15.66:49238) 2023-07-21T22:20:34.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b37fe700 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 -- 172.21.15.66:0/539881167 learned_addr learned my addr 172.21.15.66:0/539881167 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebdfa0 0x55df28ec46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 -- 172.21.15.66:0/539881167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 msgr2=0x55df28ecda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 -- 172.21.15.66:0/539881167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebdfa0 msgr2=0x55df28ec46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebdfa0 0x55df28ec46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b3fff700 1 -- 172.21.15.66:0/539881167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df28c20c10 con 0x55df28ebca30 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.320+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebdfa0 0x55df28ec46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95b3fff700 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 0x55df28ec4180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95b4100b40 tx=0x7f95a4005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e550407a5a4579d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f959affd700 1 -- 172.21.15.66:0/539881167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a401d040 con 0x55df28ebca30 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f959affd700 1 -- 172.21.15.66:0/539881167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95a401e040 con 0x55df28ebca30 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f959affd700 1 -- 172.21.15.66:0/539881167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4029610 con 0x55df28ebca30 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 -- 172.21.15.66:0/539881167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 msgr2=0x55df28ec4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 0x55df28ec4180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95b4100b40 tx=0x7f95a4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 -- 172.21.15.66:0/539881167 shutdown_connections 2023-07-21T22:20:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebdfa0 0x55df28ec46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebca30 0x55df28ec4180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/539881167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 -- 172.21.15.66:0/539881167 >> 172.21.15.66:0/539881167 conn(0x55df28eadea0 msgr2=0x55df28ec5120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.321+0000 7f95ce18bb80 1 -- 172.21.15.66:0/539881167 shutdown_connections 2023-07-21T22:20:34.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.322+0000 7f95ce18bb80 1 -- 172.21.15.66:0/539881167 wait complete. 2023-07-21T22:20:34.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.322+0000 7f95ce18bb80 1 -- start start 2023-07-21T22:20:34.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.322+0000 7f95ce18bb80 1 Processor -- start 2023-07-21T22:20:34.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.322+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ecafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.322+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.322+0000 7f95ce18bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebdfa0 0x55df28ecc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28ecca30 con 0x55df28ebdfa0 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28eccba0 con 0x55df28ebca30 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95ce18bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df28eccd60 con 0x55df28ebd380 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95ce18bb80 10 client.0 _ll_register_callbacks cb 0x7f95ac0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b8d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebdfa0 0x55df28ecc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ecafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43718/0 (socket says 172.21.15.66:43718) 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 -- 172.21.15.66:0/1201712598 learned_addr learned my addr 172.21.15.66:0/1201712598 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 -- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 msgr2=0x55df28ecafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ecafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 -- 172.21.15.66:0/1201712598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebdfa0 msgr2=0x55df28ecc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebdfa0 0x55df28ecc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b37fe700 1 -- 172.21.15.66:0/1201712598 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a4005040 con 0x55df28ebd380 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.323+0000 7f95b8d9c700 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebdfa0 0x55df28ecc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95b37fe700 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecb4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95a8096900 tx=0x7f95a80998a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6def956460b55590 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95937fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95937fe700 1 -- 172.21.15.66:0/1201712598 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8095d20 con 0x55df28ebd380 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95937fe700 1 -- 172.21.15.66:0/1201712598 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95a809dec0 con 0x55df28ebd380 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95a80a22e0 con 0x55df28ebd380 2023-07-21T22:20:34.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95937fe700 1 -- 172.21.15.66:0/1201712598 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a80b82c0 con 0x55df28ebd380 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95ce18bb80 10 client.25141 fetch_fsmap learned FSMap version 23 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55df28c5e200 con 0x55df28ebd380 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.324+0000 7f95937fe700 1 -- 172.21.15.66:0/1201712598 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95a80a22e0 con 0x55df28ebd380 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95937fe700 1 -- 172.21.15.66:0/1201712598 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f95a8018970 con 0x55df28ebd380 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95ce18bb80 10 client.25141 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95ce18bb80 10 client.25141 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95ce18bb80 20 client.25141 populate_metadata read hostname 'smithi066' 2023-07-21T22:20:34.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28f2a630 0x55df28f27250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95ce18bb80 4 client.25141 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55df28dc5000 con 0x55df28f2a630 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.325+0000 7f95b3fff700 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28f2a630 0x55df28f27250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95b3fff700 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28f2a630 0x55df28f27250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3b092cf4c2fb2e server_cookie=12fec9605eb2ec88 in_seq=0 out_seq=0 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95937fe700 10 client.25141 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95937fe700 1 -- 172.21.15.66:0/1201712598 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55df28dc5000 con 0x55df28f2a630 2023-07-21T22:20:34.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95937fe700 10 client.25141 handle_command_reply: tid=0 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 client.25141 shutdown 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 msgr2=0x55df28ecb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecb4e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f95a8096900 tx=0x7f95a80998a0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 shutdown_connections 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55df28f2a630 0x55df28f27250 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55df28ebca30 0x55df28ecafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55df28ebdfa0 0x55df28ecc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 --2- 172.21.15.66:0/1201712598 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55df28ebd380 0x55df28ecb4e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95937fe700 0 client.25141 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:20:34.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 >> 172.21.15.66:0/1201712598 conn(0x55df28eadea0 msgr2=0x55df28ec4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.326+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 shutdown_connections 2023-07-21T22:20:34.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ce18bb80 1 -- 172.21.15.66:0/1201712598 wait complete. 2023-07-21T22:20:34.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ce18bb80 20 client.25141 trim_cache size 0 max 16384 2023-07-21T22:20:34.379 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24530, 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:20:34.380 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 62.044583899999999, 2023-07-21T22:20:34.381 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 23, 2023-07-21T22:20:34.381 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 91, 2023-07-21T22:20:34.381 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:20:34.381 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 104.216249555 2023-07-21T22:20:34.381 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:20:34.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f959c0a1260 msgr2=0x7f959c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f959c0a1260 0x7f959c0a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009fd0 tx=0x7f95ac00b040 comp rx=0 tx=0).stop 2023-07-21T22:20:34.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 msgr2=0x7f95b4195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4195d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95a800e9a0 tx=0x7f95a800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 shutdown_connections 2023-07-21T22:20:34.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f959c0a1260 0x7f959c0a3710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f95b4101670 0x7f95b419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95b40ffaf0 0x7f95b4195820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 --2- 172.21.15.66:0/3337933064 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95b4100ce0 0x7f95b4195d60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 >> 172.21.15.66:0/3337933064 conn(0x7f95b40fb130 msgr2=0x7f95b41041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.327+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 shutdown_connections 2023-07-21T22:20:34.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.328+0000 7f95ba7ff700 1 -- 172.21.15.66:0/3337933064 wait complete. 2023-07-21T22:20:34.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2654059752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/321084050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:20:34.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 -- 172.21.15.66:0/223085186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5540ffae0 msgr2=0x7fd5540ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 --2- 172.21.15.66:0/223085186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5540ffae0 0x7fd5540ffef0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd550009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 -- 172.21.15.66:0/223085186 shutdown_connections 2023-07-21T22:20:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 --2- 172.21.15.66:0/223085186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 0x7fd554107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 --2- 172.21.15.66:0/223085186 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd5540ffae0 0x7fd5540ffef0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 --2- 172.21.15.66:0/223085186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd554100cd0 0x7fd554101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 -- 172.21.15.66:0/223085186 >> 172.21.15.66:0/223085186 conn(0x7fd5540fb0e0 msgr2=0x7fd5540fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 -- 172.21.15.66:0/223085186 shutdown_connections 2023-07-21T22:20:34.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.554+0000 7fd55bbff700 1 -- 172.21.15.66:0/223085186 wait complete. 2023-07-21T22:20:34.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.555+0000 7fd55bbff700 1 Processor -- start 2023-07-21T22:20:34.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.555+0000 7fd55bbff700 1 -- start start 2023-07-21T22:20:34.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.555+0000 7fd55bbff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5540ffae0 0x7fd554195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.555+0000 7fd55bbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 0x7fd554195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55bbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 0x7fd55419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55bbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419d3c0 con 0x7fd554100cd0 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55bbff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419d500 con 0x7fd5540ffae0 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55bbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55419d690 con 0x7fd554101660 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 0x7fd554195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 0x7fd554195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49266/0 (socket says 172.21.15.66:49266) 2023-07-21T22:20:34.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 -- 172.21.15.66:0/667517351 learned_addr learned my addr 172.21.15.66:0/667517351 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55999b700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5540ffae0 0x7fd554195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55a19c700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 0x7fd55419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 -- 172.21.15.66:0/667517351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 msgr2=0x7fd55419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 0x7fd55419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 -- 172.21.15.66:0/667517351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5540ffae0 msgr2=0x7fd554195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5540ffae0 0x7fd554195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55919a700 1 -- 172.21.15.66:0/667517351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500095f0 con 0x7fd554100cd0 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.556+0000 7fd55a19c700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 0x7fd55419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.557+0000 7fd55919a700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 0x7fd554195d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e9a0 tx=0x7fd54400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.557+0000 7fd54affd700 1 -- 172.21.15.66:0/667517351 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544009790 con 0x7fd554100cd0 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.557+0000 7fd54affd700 1 -- 172.21.15.66:0/667517351 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd544010430 con 0x7fd554100cd0 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.557+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55419d970 con 0x7fd554100cd0 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.557+0000 7fd54affd700 1 -- 172.21.15.66:0/667517351 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544018680 con 0x7fd554100cd0 2023-07-21T22:20:34.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.557+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55419de80 con 0x7fd554100cd0 2023-07-21T22:20:34.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.560+0000 7fd54affd700 1 -- 172.21.15.66:0/667517351 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd5440105a0 con 0x7fd554100cd0 2023-07-21T22:20:34.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.561+0000 7fd56f43fb80 1 Processor -- start 2023-07-21T22:20:34.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 -- start start 2023-07-21T22:20:34.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd54affd700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd5400a11e0 0x7fd5400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 0x55bc0cb14bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb13e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ca18c60 con 0x55bc0cb139c0 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ca6b9f0 con 0x55bc0cb15150 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ca64e80 con 0x55bc0cb147e0 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd54affd700 1 -- 172.21.15.66:0/667517351 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd544014070 con 0x7fd554100cd0 2023-07-21T22:20:34.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd55a19c700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd5400a11e0 0x7fd5400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd55919a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd55999b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 0x55bc0cb14bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55999b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 0x55bc0cb14bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43730/0 (socket says 172.21.15.66:43730) 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.562+0000 7fd55919a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb133e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53722/0 (socket says 172.21.15.66:53722) 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55999b700 1 -- 172.21.15.66:0/1324366489 learned_addr learned my addr 172.21.15.66:0/1324366489 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55a19c700 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb13e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55919a700 1 -- 172.21.15.66:0/1324366489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 msgr2=0x55bc0cb14bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55919a700 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 0x55bc0cb14bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55919a700 1 -- 172.21.15.66:0/1324366489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 msgr2=0x55bc0cb13e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55919a700 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb13e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55919a700 1 -- 172.21.15.66:0/1324366489 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0c892d90 con 0x55bc0cb15150 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55999b700 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 0x55bc0cb14bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55919a700 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb133e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd5440ca610 tx=0x7fd544099b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5adfdda253219c49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd55a19c700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd5400a11e0 0x7fd5400a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009fd0 tx=0x7fd54c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd53ffff700 1 -- 172.21.15.66:0/1324366489 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54409df40 con 0x55bc0cb15150 2023-07-21T22:20:34.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd53ffff700 1 -- 172.21.15.66:0/1324366489 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5440a2360 con 0x55bc0cb15150 2023-07-21T22:20:34.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.563+0000 7fd53ffff700 1 -- 172.21.15.66:0/1324366489 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440ca860 con 0x55bc0cb15150 2023-07-21T22:20:34.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 -- 172.21.15.66:0/1324366489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 msgr2=0x55bc0cb133e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb133e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd5440ca610 tx=0x7fd544099b60 comp rx=0 tx=0).stop 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 -- 172.21.15.66:0/1324366489 shutdown_connections 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb13e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb147e0 0x55bc0cb14bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/1324366489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb133e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 -- 172.21.15.66:0/1324366489 >> 172.21.15.66:0/1324366489 conn(0x55bc0cb08ea0 msgr2=0x55bc0cb0e310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 -- 172.21.15.66:0/1324366489 shutdown_connections 2023-07-21T22:20:34.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.564+0000 7fd56f43fb80 1 -- 172.21.15.66:0/1324366489 wait complete. 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.565+0000 7fd56f43fb80 1 -- start start 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.565+0000 7fd56f43fb80 1 Processor -- start 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb147e0 0x55bc0cb219e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ca04a70 con 0x55bc0cb147e0 2023-07-21T22:20:34.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0ca04770 con 0x55bc0cb15150 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0c8e88d0 con 0x55bc0cb139c0 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd56f43fb80 10 client.0 _ll_register_callbacks cb 0x55bc0ca02710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd55a19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd55a19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb28a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53732/0 (socket says 172.21.15.66:53732) 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd55919a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb147e0 0x55bc0cb219e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd55a19c700 1 -- 172.21.15.66:0/2916999985 learned_addr learned my addr 172.21.15.66:0/2916999985 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.566+0000 7fd55999b700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55a19c700 1 -- 172.21.15.66:0/2916999985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 msgr2=0x55bc0cb214a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55a19c700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb214a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55a19c700 1 -- 172.21.15.66:0/2916999985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb147e0 msgr2=0x55bc0cb219e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55a19c700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb147e0 0x55bc0cb219e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55a19c700 1 -- 172.21.15.66:0/2916999985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd554108850 con 0x55bc0cb15150 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55999b700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd55a19c700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb28a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00efa0 tx=0x7fd54c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a0dabf0c6f1cc1b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd53d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd53d7fa700 1 -- 172.21.15.66:0/2916999985 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c01be50 con 0x55bc0cb15150 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.567+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc0c9be4a0 con 0x55bc0cb15150 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd53d7fa700 1 -- 172.21.15.66:0/2916999985 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd54c021040 con 0x55bc0cb15150 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd53d7fa700 1 -- 172.21.15.66:0/2916999985 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd54c027460 con 0x55bc0cb15150 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd56f43fb80 10 client.25172 fetch_fsmap learned FSMap version 23 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd55419de80 con 0x55bc0cb15150 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd53d7fa700 1 -- 172.21.15.66:0/2916999985 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc0c9be4a0 con 0x55bc0cb15150 2023-07-21T22:20:34.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd53d7fa700 1 -- 172.21.15.66:0/2916999985 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd54c025590 con 0x55bc0cb15150 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd56f43fb80 10 client.25172 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd56f43fb80 10 client.25172 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.568+0000 7fd56f43fb80 20 client.25172 populate_metadata read hostname 'smithi066' 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.569+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb2c770 0x55bc0cb2bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.569+0000 7fd56f43fb80 4 client.25172 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.569+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc0ca5de60 con 0x55bc0cb2c770 2023-07-21T22:20:34.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.569+0000 7fd55999b700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb2c770 0x55bc0cb2bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.569+0000 7fd55999b700 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb2c770 0x55bc0cb2bb20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec4f724cf6a2028 server_cookie=21c206410b49b7b7 in_seq=0 out_seq=0 2023-07-21T22:20:34.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.569+0000 7fd53d7fa700 10 client.25172 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:20:34.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2654059752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/321084050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.570+0000 7fd53d7fa700 1 -- 172.21.15.66:0/2916999985 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bc0ca5de60 con 0x55bc0cb2c770 2023-07-21T22:20:34.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.570+0000 7fd53d7fa700 10 client.25172 handle_command_reply: tid=0 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.570+0000 7fd56f43fb80 1 client.25172 shutdown 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 msgr2=0x55bc0cb28a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb28a20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00efa0 tx=0x7fd54c00b040 comp rx=0 tx=0).stop 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 shutdown_connections 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb15150 0x55bc0cb28a20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb2c770 0x55bc0cb2bb20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb214a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd53d7fa700 0 client.25172 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:20:34.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2916999985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb147e0 0x55bc0cb219e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 >> 172.21.15.66:0/2916999985 conn(0x55bc0cb08ea0 msgr2=0x55bc0cb0d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 shutdown_connections 2023-07-21T22:20:34.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2916999985 wait complete. 2023-07-21T22:20:34.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.571+0000 7fd56f43fb80 20 client.25172 trim_cache size 0 max 16384 2023-07-21T22:20:34.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.589+0000 7fd56f43fb80 1 Processor -- start 2023-07-21T22:20:34.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.589+0000 7fd56f43fb80 1 -- start start 2023-07-21T22:20:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb289e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb15150 0x55bc0cb219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0c9be4a0 con 0x55bc0cb15150 2023-07-21T22:20:34.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cb22690 con 0x55bc0cb147e0 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cb21ef0 con 0x55bc0cb139c0 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55a19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb15150 0x55bc0cb219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55919a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55919a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb289e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53734/0 (socket says 172.21.15.66:53734) 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55919a700 1 -- 172.21.15.66:0/2522761265 learned_addr learned my addr 172.21.15.66:0/2522761265 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55999b700 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55a19c700 1 -- 172.21.15.66:0/2522761265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 msgr2=0x55bc0cb21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55a19c700 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55a19c700 1 -- 172.21.15.66:0/2522761265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 msgr2=0x55bc0cb289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55a19c700 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55a19c700 1 -- 172.21.15.66:0/2522761265 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc0c87bc10 con 0x55bc0cb15150 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.590+0000 7fd55919a700 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.591+0000 7fd55a19c700 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb15150 0x55bc0cb219b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0258f0 tx=0x7fd54c0089b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438a381940aa4e1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.591+0000 7fd548bff700 1 -- 172.21.15.66:0/2522761265 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c025ea0 con 0x55bc0cb15150 2023-07-21T22:20:34.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.591+0000 7fd548bff700 1 -- 172.21.15.66:0/2522761265 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd54c00f040 con 0x55bc0cb15150 2023-07-21T22:20:34.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.591+0000 7fd548bff700 1 -- 172.21.15.66:0/2522761265 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54c0082d0 con 0x55bc0cb15150 2023-07-21T22:20:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2522761265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb15150 msgr2=0x55bc0cb219b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb15150 0x55bc0cb219b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0258f0 tx=0x7fd54c0089b0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2522761265 shutdown_connections 2023-07-21T22:20:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb15150 0x55bc0cb219b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb139c0 0x55bc0cb21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/2522761265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2522761265 >> 172.21.15.66:0/2522761265 conn(0x55bc0cb08ea0 msgr2=0x55bc0cb1c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2522761265 shutdown_connections 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 -- 172.21.15.66:0/2522761265 wait complete. 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 -- start start 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.592+0000 7fd56f43fb80 1 Processor -- start 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb15150 0x55bc0cb27560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cb27aa0 con 0x55bc0cb139c0 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cb27c10 con 0x55bc0cb147e0 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc0cb27dd0 con 0x55bc0cb15150 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd56f43fb80 10 client.0 _ll_register_callbacks cb 0x7fd54c008650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb24000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49298/0 (socket says 172.21.15.66:49298) 2023-07-21T22:20:34.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 -- 172.21.15.66:0/3046435980 learned_addr learned my addr 172.21.15.66:0/3046435980 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55919a700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55a19c700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb15150 0x55bc0cb27560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 -- 172.21.15.66:0/3046435980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb15150 msgr2=0x55bc0cb27560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb15150 0x55bc0cb27560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 -- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 msgr2=0x55bc0cb24540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.593+0000 7fd55999b700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb24540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd55999b700 1 -- 172.21.15.66:0/3046435980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc0c892d90 con 0x55bc0cb139c0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd55a19c700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb15150 0x55bc0cb27560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd55999b700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb24000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd550000c00 tx=0x7fd550005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff909738dd31b9c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd53d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd53d7fa700 1 -- 172.21.15.66:0/3046435980 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55001d070 con 0x55bc0cb139c0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd53d7fa700 1 -- 172.21.15.66:0/3046435980 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd5500056c0 con 0x55bc0cb139c0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5440a2360 con 0x55bc0cb139c0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd53d7fa700 1 -- 172.21.15.66:0/3046435980 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd55002a070 con 0x55bc0cb139c0 2023-07-21T22:20:34.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd56f43fb80 10 client.15117 fetch_fsmap learned FSMap version 23 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.594+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd55419de80 con 0x55bc0cb139c0 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd53d7fa700 1 -- 172.21.15.66:0/3046435980 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5440a2360 con 0x55bc0cb139c0 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd53d7fa700 1 -- 172.21.15.66:0/3046435980 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd55000f460 con 0x55bc0cb139c0 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd56f43fb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd56f43fb80 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd56f43fb80 20 client.15117 populate_metadata read hostname 'smithi066' 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb86270 0x55bc0cb82c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:34.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd56f43fb80 4 client.15117 mds_command: new command op to 24572 tid=0[{"prefix": "status"}] 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bc0ca20000 con 0x55bc0cb86270 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.595+0000 7fd55919a700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb86270 0x55bc0cb82c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd53d7fa700 10 client.15117 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd55919a700 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb86270 0x55bc0cb82c00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92c7aca01b10285a server_cookie=8c5b9b99f31d2408 in_seq=0 out_seq=0 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd53d7fa700 1 -- 172.21.15.66:0/3046435980 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55bc0ca20000 con 0x55bc0cb86270 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd53d7fa700 10 client.15117 handle_command_reply: tid=0 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 client.15117 shutdown 2023-07-21T22:20:34.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 msgr2=0x55bc0cb24000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb24000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd550000c00 tx=0x7fd550005960 comp rx=0 tx=0).stop 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 shutdown_connections 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55bc0cb86270 0x55bc0cb82c00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc0cb15150 0x55bc0cb27560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bc0cb139c0 0x55bc0cb24000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd53d7fa700 0 client.15117 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:20:34.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 --2- 172.21.15.66:0/3046435980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc0cb147e0 0x55bc0cb24540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 >> 172.21.15.66:0/3046435980 conn(0x55bc0cb08ea0 msgr2=0x55bc0cb1c370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 shutdown_connections 2023-07-21T22:20:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 1 -- 172.21.15.66:0/3046435980 wait complete. 2023-07-21T22:20:34.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.596+0000 7fd56f43fb80 20 client.15117 trim_cache size 0 max 16384 2023-07-21T22:20:34.649 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24572, 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 87.194802346000003, 2023-07-21T22:20:34.650 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 23, 2023-07-21T22:20:34.651 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 91, 2023-07-21T22:20:34.651 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:20:34.651 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 93.820900186000003 2023-07-21T22:20:34.651 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd5400a11e0 msgr2=0x7fd5400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd5400a11e0 0x7fd5400a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009fd0 tx=0x7fd54c009310 comp rx=0 tx=0).stop 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 msgr2=0x7fd554195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 0x7fd554195d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e9a0 tx=0x7fd54400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 shutdown_connections 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd5400a11e0 0x7fd5400a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5540ffae0 0x7fd554195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd554101660 0x7fd55419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 --2- 172.21.15.66:0/667517351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd554100cd0 0x7fd554195d80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:34.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 >> 172.21.15.66:0/667517351 conn(0x7fd5540fb0e0 msgr2=0x7fd554104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:34.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 shutdown_connections 2023-07-21T22:20:34.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:34.597+0000 7fd55bbff700 1 -- 172.21.15.66:0/667517351 wait complete. 2023-07-21T22:20:34.660 INFO:tasks.cephadm:Setup complete, yielding 2023-07-21T22:20:34.660 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T22:20:34.669 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:20:34.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-21T22:20:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2654059752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:34.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/321084050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:35.410 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:35.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:35 smithi104 ceph-mon[129021]: pgmap v320: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:35.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:35 smithi066 ceph-mon[126379]: pgmap v320: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:35 smithi149 ceph-mon[129268]: pgmap v320: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:36.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 -- 172.21.15.66:0/2273699454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20101720 msgr2=0x7f9c20107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:36.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2273699454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20101720 0x7f9c20107fe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800b0a0 tx=0x7f9c1800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:36.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 -- 172.21.15.66:0/2273699454 shutdown_connections 2023-07-21T22:20:36.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2273699454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c200ffb90 0x7f9c200fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:36.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2273699454 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20101720 0x7f9c20107fe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2273699454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20100d90 0x7f9c201011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 -- 172.21.15.66:0/2273699454 >> 172.21.15.66:0/2273699454 conn(0x7f9c200fb1a0 msgr2=0x7f9c200fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 -- 172.21.15.66:0/2273699454 shutdown_connections 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.947+0000 7f9c2837b700 1 -- 172.21.15.66:0/2273699454 wait complete. 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.948+0000 7f9c2837b700 1 Processor -- start 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.948+0000 7f9c2837b700 1 -- start start 2023-07-21T22:20:36.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c2837b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c200ffb90 0x7f9c20197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c2837b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 0x7f9c20198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c2837b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20101720 0x7f9c2019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c2837b700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019f750 con 0x7f9c20100d90 2023-07-21T22:20:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c2837b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019f890 con 0x7f9c200ffb90 2023-07-21T22:20:36.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c2837b700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2019fa20 con 0x7f9c20101720 2023-07-21T22:20:36.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 0x7f9c20198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 0x7f9c20198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49316/0 (socket says 172.21.15.66:49316) 2023-07-21T22:20:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 -- 172.21.15.66:0/2630890547 learned_addr learned my addr 172.21.15.66:0/2630890547 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c26918700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20101720 0x7f9c2019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c26117700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c200ffb90 0x7f9c20197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 -- 172.21.15.66:0/2630890547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20101720 msgr2=0x7f9c2019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:36.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20101720 0x7f9c2019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 -- 172.21.15.66:0/2630890547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c200ffb90 msgr2=0x7f9c20197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c200ffb90 0x7f9c20197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.949+0000 7f9c25916700 1 -- 172.21.15.66:0/2630890547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c18009d00 con 0x7f9c20100d90 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.950+0000 7f9c25916700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 0x7f9c20198100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000e9d0 tx=0x7f9c1000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.950+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10009790 con 0x7f9c20100d90 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.950+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c1000ee70 con 0x7f9c20100d90 2023-07-21T22:20:36.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.950+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2019fd00 con 0x7f9c20100d90 2023-07-21T22:20:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.950+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10010ea0 con 0x7f9c20100d90 2023-07-21T22:20:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.950+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c201a01b0 con 0x7f9c20100d90 2023-07-21T22:20:36.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.951+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2010b3b0 con 0x7f9c20100d90 2023-07-21T22:20:37.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.953+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9c1000cbd0 con 0x7f9c20100d90 2023-07-21T22:20:37.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.955+0000 7f9c177fe700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9c0c0a11e0 0x7f9c0c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:37.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.955+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9c10014070 con 0x7f9c20100d90 2023-07-21T22:20:37.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.955+0000 7f9c26117700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9c0c0a11e0 0x7f9c0c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:37.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.956+0000 7f9c26117700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9c0c0a11e0 0x7f9c0c0a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c20100bf0 tx=0x7f9c1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:37.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:36.958+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9c10051770 con 0x7f9c20100d90 2023-07-21T22:20:37.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.184+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c20107d30 con 0x7f9c0c0a11e0 2023-07-21T22:20:37.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.185+0000 7f9c177fe700 1 -- 172.21.15.66:0/2630890547 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9c20107d30 con 0x7f9c0c0a11e0 2023-07-21T22:20:37.230 INFO:teuthology.orchestra.run.smithi066.stdout:Backend: cephadm 2023-07-21T22:20:37.230 INFO:teuthology.orchestra.run.smithi066.stdout:Available: Yes 2023-07-21T22:20:37.230 INFO:teuthology.orchestra.run.smithi066.stdout:Paused: No 2023-07-21T22:20:37.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9c0c0a11e0 msgr2=0x7f9c0c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:37.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9c0c0a11e0 0x7f9c0c0a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9c20100bf0 tx=0x7f9c1c009260 comp rx=0 tx=0).stop 2023-07-21T22:20:37.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 msgr2=0x7f9c20198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 0x7f9c20198100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000e9d0 tx=0x7f9c1000ece0 comp rx=0 tx=0).stop 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 shutdown_connections 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f9c0c0a11e0 0x7f9c0c0a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c200ffb90 0x7f9c20197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9c20101720 0x7f9c2019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 --2- 172.21.15.66:0/2630890547 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9c20100d90 0x7f9c20198100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 >> 172.21.15.66:0/2630890547 conn(0x7f9c200fb1a0 msgr2=0x7f9c20106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:37.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 shutdown_connections 2023-07-21T22:20:37.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:37.188+0000 7f9c2837b700 1 -- 172.21.15.66:0/2630890547 wait complete. 2023-07-21T22:20:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:37 smithi104 ceph-mon[129021]: pgmap v321: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-07-21T22:20:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-07-21T22:20:37.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:37 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2023-07-21T22:20:37.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:37 smithi066 ceph-mon[126379]: pgmap v321: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:37.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-07-21T22:20:37.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-07-21T22:20:37.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:37 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2023-07-21T22:20:37.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-21T22:20:37.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:37 smithi149 ceph-mon[129268]: pgmap v321: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:37.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-07-21T22:20:37.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2023-07-21T22:20:37.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:37 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2023-07-21T22:20:38.461 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:38.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:38 smithi104 ceph-mon[129021]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:38.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-07-21T22:20:38.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-07-21T22:20:38.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:38 smithi104 ceph-mon[129021]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2023-07-21T22:20:38.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:38 smithi104 ceph-mon[129021]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T22:20:38.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:38 smithi066 ceph-mon[126379]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:38.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-07-21T22:20:38.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-07-21T22:20:38.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:38 smithi066 ceph-mon[126379]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2023-07-21T22:20:38.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:38 smithi066 ceph-mon[126379]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T22:20:38.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:38 smithi149 ceph-mon[129268]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:38.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-07-21T22:20:38.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2023-07-21T22:20:38.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:38 smithi149 ceph-mon[129268]: from='mgr.14150 172.21.15.66:0/1725301826' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2023-07-21T22:20:38.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:38 smithi149 ceph-mon[129268]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T22:20:39.614 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:39 smithi066 ceph-mon[126379]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-21T22:20:39.614 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:39 smithi066 ceph-mon[126379]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T22:20:39.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:39 smithi104 ceph-mon[129021]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-21T22:20:39.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:39 smithi104 ceph-mon[129021]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T22:20:39.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:39 smithi149 ceph-mon[129268]: pgmap v323: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-21T22:20:39.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:39 smithi149 ceph-mon[129268]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T22:20:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 -- 172.21.15.66:0/2538437295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28100610 msgr2=0x7f3d281029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 --2- 172.21.15.66:0/2538437295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28100610 0x7f3d281029f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009990 tx=0x7f3d18009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 -- 172.21.15.66:0/2538437295 shutdown_connections 2023-07-21T22:20:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 --2- 172.21.15.66:0/2538437295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d28105850 0x7f3d28107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 --2- 172.21.15.66:0/2538437295 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28100610 0x7f3d281029f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 --2- 172.21.15.66:0/2538437295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28102f30 0x7f3d28105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 -- 172.21.15.66:0/2538437295 >> 172.21.15.66:0/2538437295 conn(0x7f3d280fa0b0 msgr2=0x7f3d280fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 -- 172.21.15.66:0/2538437295 shutdown_connections 2023-07-21T22:20:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.354+0000 7f3d2d324700 1 -- 172.21.15.66:0/2538437295 wait complete. 2023-07-21T22:20:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.355+0000 7f3d2d324700 1 Processor -- start 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 -- start start 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 0x7f3d28195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d28102f30 0x7f3d28195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28105850 0x7f3d2819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2819d400 con 0x7f3d28105850 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2819d540 con 0x7f3d28100610 2023-07-21T22:20:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d2d324700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2819d6d0 con 0x7f3d28102f30 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d26ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 0x7f3d28195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28105850 0x7f3d2819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d26ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 0x7f3d28195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53794/0 (socket says 172.21.15.66:53794) 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d26ffd700 1 -- 172.21.15.66:0/3445230328 learned_addr learned my addr 172.21.15.66:0/3445230328 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.356+0000 7f3d267fc700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d28102f30 0x7f3d28195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d26ffd700 1 -- 172.21.15.66:0/3445230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d28102f30 msgr2=0x7f3d28195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d26ffd700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d28102f30 0x7f3d28195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d26ffd700 1 -- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28105850 msgr2=0x7f3d2819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d26ffd700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28105850 0x7f3d2819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d26ffd700 1 -- 172.21.15.66:0/3445230328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180095f0 con 0x7f3d28100610 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d26ffd700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 0x7f3d28195870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d180059d0 tx=0x7f3d18004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1801d070 con 0x7f3d28100610 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d1800fb90 con 0x7f3d28100610 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d180173f0 con 0x7f3d28100610 2023-07-21T22:20:40.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.357+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2819d950 con 0x7f3d28100610 2023-07-21T22:20:40.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.358+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2819dd10 con 0x7f3d28100610 2023-07-21T22:20:40.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.359+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2810b360 con 0x7f3d28100610 2023-07-21T22:20:40.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.361+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3d1800fd00 con 0x7f3d28100610 2023-07-21T22:20:40.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.362+0000 7f3d0ffff700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3d100a11e0 0x7f3d100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:40.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.362+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3d180ce380 con 0x7f3d28100610 2023-07-21T22:20:40.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.362+0000 7f3d267fc700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3d100a11e0 0x7f3d100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:40.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.363+0000 7f3d267fc700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3d100a11e0 0x7f3d100a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009c60 tx=0x7f3d1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:40.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.364+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3d18097630 con 0x7f3d28100610 2023-07-21T22:20:40.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.581+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d28102180 con 0x7f3d100a11e0 2023-07-21T22:20:40.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.595+0000 7f3d0ffff700 1 -- 172.21.15.66:0/3445230328 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f3d28102180 con 0x7f3d100a11e0 2023-07-21T22:20:40.639 INFO:teuthology.orchestra.run.smithi066.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.a smithi066 running (102s) 21s ago 101s 13.7M - 18.0.0-4735-gbffe049d fa7da5dee971 037fdd3da291 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.b smithi104 running (100s) 28s ago 99s 30.1M - 18.0.0-4735-gbffe049d fa7da5dee971 ebcdcf444018 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.c smithi149 running (113s) 28s ago 112s 20.0M - 18.0.0-4735-gbffe049d fa7da5dee971 8df0d8a15017 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.d smithi066 running (2m) 21s ago 119s 13.9M - 18.0.0-4735-gbffe049d fa7da5dee971 ed25230da3ce 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.e smithi104 running (117s) 28s ago 117s 16.4M - 18.0.0-4735-gbffe049d fa7da5dee971 1930cab105f5 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.f smithi149 running (115s) 28s ago 114s 18.8M - 18.0.0-4735-gbffe049d fa7da5dee971 dd9c9d461778 2023-07-21T22:20:40.640 INFO:teuthology.orchestra.run.smithi066.stdout:mds.g smithi066 running (105s) 21s ago 105s 13.7M - 18.0.0-4735-gbffe049d fa7da5dee971 fb49a61277a9 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mds.h smithi104 running (110s) 28s ago 110s 21.2M - 18.0.0-4735-gbffe049d fa7da5dee971 4ac07dcbef39 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mds.i smithi149 running (108s) 28s ago 108s 20.8M - 18.0.0-4735-gbffe049d fa7da5dee971 0107abbd2666 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mds.j smithi066 running (2m) 21s ago 2m 13.8M - 18.0.0-4735-gbffe049d fa7da5dee971 7e707ab90db1 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mds.k smithi104 running (2m) 28s ago 2m 15.0M - 18.0.0-4735-gbffe049d fa7da5dee971 d7a1a1904531 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mds.l smithi149 running (2m) 28s ago 2m 17.5M - 18.0.0-4735-gbffe049d fa7da5dee971 91ff1ef556eb 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.x smithi066 *:9283,8765 running (8m) 21s ago 8m 522M - 18.0.0-4735-gbffe049d fa7da5dee971 cb1b8fddfef5 2023-07-21T22:20:40.641 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.y smithi104 *:8443,8765 running (6m) 28s ago 6m 449M - 18.0.0-4735-gbffe049d fa7da5dee971 83d91cef7640 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.z smithi149 *:8443,8765 running (7m) 28s ago 7m 446M - 18.0.0-4735-gbffe049d fa7da5dee971 d12e652ab94c 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:mon.a smithi066 running (8m) 21s ago 9m 93.6M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 084af29f85fd 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:mon.b smithi104 running (7m) 28s ago 7m 63.5M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 b858e006bfc8 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:mon.c smithi149 running (7m) 28s ago 7m 65.2M 2048M 18.0.0-4735-gbffe049d fa7da5dee971 72aaf61f4d18 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 smithi066 running (6m) 21s ago 6m 72.2M 4316M 18.0.0-4735-gbffe049d fa7da5dee971 3f199cc07862 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 smithi104 running (6m) 28s ago 6m 77.4M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 6a91f2ebebaf 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 smithi149 running (5m) 28s ago 5m 74.9M 1469M 18.0.0-4735-gbffe049d fa7da5dee971 342c954d7a55 2023-07-21T22:20:40.642 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 smithi066 running (5m) 21s ago 5m 67.6M 4316M 18.0.0-4735-gbffe049d fa7da5dee971 58bae7267249 2023-07-21T22:20:40.643 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 smithi104 running (5m) 28s ago 5m 67.9M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 88a111cb692b 2023-07-21T22:20:40.643 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 smithi149 running (4m) 28s ago 4m 69.3M 1469M 18.0.0-4735-gbffe049d fa7da5dee971 d5ecb5e93d59 2023-07-21T22:20:40.643 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 smithi066 running (4m) 21s ago 4m 61.6M 4316M 18.0.0-4735-gbffe049d fa7da5dee971 d9e1d6b6ca5a 2023-07-21T22:20:40.643 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 smithi104 running (3m) 28s ago 3m 66.4M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 c82377a3159d 2023-07-21T22:20:40.643 INFO:teuthology.orchestra.run.smithi066.stdout:osd.8 smithi149 running (3m) 28s ago 3m 62.9M 1469M 18.0.0-4735-gbffe049d fa7da5dee971 6b1c6a33da21 2023-07-21T22:20:40.644 INFO:teuthology.orchestra.run.smithi066.stdout:osd.9 smithi066 running (3m) 21s ago 3m 60.6M 4316M 18.0.0-4735-gbffe049d fa7da5dee971 f4d5f7a86302 2023-07-21T22:20:40.644 INFO:teuthology.orchestra.run.smithi066.stdout:osd.10 smithi104 running (2m) 28s ago 2m 62.0M 4292M 18.0.0-4735-gbffe049d fa7da5dee971 dc4473c1c7c3 2023-07-21T22:20:40.644 INFO:teuthology.orchestra.run.smithi066.stdout:osd.11 smithi149 running (2m) 28s ago 2m 55.4M 1469M 18.0.0-4735-gbffe049d fa7da5dee971 9ffac39d8b33 2023-07-21T22:20:40.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3d100a11e0 msgr2=0x7f3d100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:40.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3d100a11e0 0x7f3d100a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009c60 tx=0x7f3d1c009260 comp rx=0 tx=0).stop 2023-07-21T22:20:40.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 msgr2=0x7f3d28195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:40.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 0x7f3d28195870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3d180059d0 tx=0x7f3d18004730 comp rx=0 tx=0).stop 2023-07-21T22:20:40.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 shutdown_connections 2023-07-21T22:20:40.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3d100a11e0 0x7f3d100a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d28100610 0x7f3d28195870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3d28105850 0x7f3d2819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 --2- 172.21.15.66:0/3445230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d28102f30 0x7f3d28195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:40.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.597+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 >> 172.21.15.66:0/3445230328 conn(0x7f3d280fa0b0 msgr2=0x7f3d28101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.598+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 shutdown_connections 2023-07-21T22:20:40.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:40.598+0000 7f3d2d324700 1 -- 172.21.15.66:0/3445230328 wait complete. 2023-07-21T22:20:41.161 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-21T22:20:41.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:41 smithi104 ceph-mon[129021]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:41.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:41 smithi066 ceph-mon[126379]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:41.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:41 smithi149 ceph-mon[129268]: pgmap v325: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:41.906 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:42.438 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:42 smithi066 ceph-mon[126379]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:42.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:42 smithi104 ceph-mon[129021]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:42.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:42 smithi149 ceph-mon[129268]: from='client.25190 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 -- 172.21.15.66:0/799040200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234107a60 msgr2=0x7f1234109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 --2- 172.21.15.66:0/799040200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234107a60 0x7f1234109ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b0a0 tx=0x7f122c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 -- 172.21.15.66:0/799040200 shutdown_connections 2023-07-21T22:20:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 --2- 172.21.15.66:0/799040200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234107a60 0x7f1234109ea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 --2- 172.21.15.66:0/799040200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234074540 0x7f12340729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 --2- 172.21.15.66:0/799040200 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234072f70 0x7f12340733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 -- 172.21.15.66:0/799040200 >> 172.21.15.66:0/799040200 conn(0x7f12340fb1a0 msgr2=0x7f12340fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 -- 172.21.15.66:0/799040200 shutdown_connections 2023-07-21T22:20:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.297+0000 7f123a674700 1 -- 172.21.15.66:0/799040200 wait complete. 2023-07-21T22:20:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 Processor -- start 2023-07-21T22:20:43.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 -- start start 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 0x7f1234197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 0x7f1234198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234107a60 0x7f123419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419f770 con 0x7f1234074540 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419f8b0 con 0x7f1234072f70 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.298+0000 7f123a674700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123419fa40 con 0x7f1234107a60 2023-07-21T22:20:43.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 0x7f1234198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f1233fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 0x7f1234197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 0x7f1234198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33272/0 (socket says 172.21.15.66:33272) 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f1233fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 0x7f1234197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49598/0 (socket says 172.21.15.66:49598) 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 -- 172.21.15.66:0/502837284 learned_addr learned my addr 172.21.15.66:0/502837284 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f1238c11700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234107a60 0x7f123419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 -- 172.21.15.66:0/502837284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234107a60 msgr2=0x7f123419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234107a60 0x7f123419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 -- 172.21.15.66:0/502837284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 msgr2=0x7f1234197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:43.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 0x7f1234197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 -- 172.21.15.66:0/502837284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c009d00 con 0x7f1234074540 2023-07-21T22:20:43.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f1238c11700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234107a60 0x7f123419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:43.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f1233fff700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 0x7f1234197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.299+0000 7f12337fe700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 0x7f1234198120 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f122800e9d0 tx=0x7f122800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.300+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228009790 con 0x7f1234074540 2023-07-21T22:20:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.300+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122800cbd0 con 0x7f1234074540 2023-07-21T22:20:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.300+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228018680 con 0x7f1234074540 2023-07-21T22:20:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.300+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123419fd20 con 0x7f1234074540 2023-07-21T22:20:43.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.300+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12341a0170 con 0x7f1234074540 2023-07-21T22:20:43.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.301+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1234106820 con 0x7f1234074540 2023-07-21T22:20:43.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.303+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f122800cd40 con 0x7f1234074540 2023-07-21T22:20:43.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.304+0000 7f12317fa700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f121c0a1260 0x7f121c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:43.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.305+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1228014070 con 0x7f1234074540 2023-07-21T22:20:43.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.305+0000 7f1233fff700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f121c0a1260 0x7f121c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:43.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.305+0000 7f1233fff700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f121c0a1260 0x7f121c0a3710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1224011490 tx=0x7f1224009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:43.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.309+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1228051770 con 0x7f1234074540 2023-07-21T22:20:43.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.528+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1234060950 con 0x7f121c0a1260 2023-07-21T22:20:43.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.534+0000 7f12317fa700 1 -- 172.21.15.66:0/502837284 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f1234060950 con 0x7f121c0a1260 2023-07-21T22:20:43.578 INFO:teuthology.orchestra.run.smithi066.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-21T22:20:43.579 INFO:teuthology.orchestra.run.smithi066.stdout:mds.all 12/12 31s ago 2m smithi066=a;smithi066=d;smithi066=g;smithi066=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi149=c;smithi149=f;smithi149=i;smithi149=l;count:12 2023-07-21T22:20:43.579 INFO:teuthology.orchestra.run.smithi066.stdout:mgr 3/3 31s ago 7m smithi066=x;smithi104=y;smithi149=z;count:3 2023-07-21T22:20:43.579 INFO:teuthology.orchestra.run.smithi066.stdout:mon 3/3 31s ago 7m smithi066:172.21.15.66=a;smithi104:172.21.15.104=b;smithi149:172.21.15.149=c;count:3 2023-07-21T22:20:43.579 INFO:teuthology.orchestra.run.smithi066.stdout:osd 12 31s ago - 2023-07-21T22:20:43.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f121c0a1260 msgr2=0x7f121c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:43.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f121c0a1260 0x7f121c0a3710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1224011490 tx=0x7f1224009260 comp rx=0 tx=0).stop 2023-07-21T22:20:43.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 msgr2=0x7f1234198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:43.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 0x7f1234198120 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f122800e9d0 tx=0x7f122800ece0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 shutdown_connections 2023-07-21T22:20:43.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f121c0a1260 0x7f121c0a3710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1234072f70 0x7f1234197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1234107a60 0x7f123419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 --2- 172.21.15.66:0/502837284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1234074540 0x7f1234198120 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:43.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 >> 172.21.15.66:0/502837284 conn(0x7f12340fb1a0 msgr2=0x7f1234103960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:43.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 shutdown_connections 2023-07-21T22:20:43.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:43.536+0000 7f123a674700 1 -- 172.21.15.66:0/502837284 wait complete. 2023-07-21T22:20:43.605 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:43 smithi066 ceph-mon[126379]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-07-21T22:20:43.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:43 smithi104 ceph-mon[129021]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-07-21T22:20:43.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:43 smithi149 ceph-mon[129268]: pgmap v326: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2023-07-21T22:20:44.103 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-21T22:20:44.845 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:45.413 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:45 smithi066 ceph-mon[126379]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:45.413 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:45 smithi066 ceph-mon[126379]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:45.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:45 smithi104 ceph-mon[129021]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:45.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:45 smithi104 ceph-mon[129021]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:45.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:45 smithi149 ceph-mon[129268]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:45.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:45 smithi149 ceph-mon[129268]: pgmap v327: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-07-21T22:20:46.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 -- 172.21.15.66:0/1821941601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24100d90 msgr2=0x7f8d241011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:46.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 --2- 172.21.15.66:0/1821941601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24100d90 0x7f8d241011e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009990 tx=0x7f8d14009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 -- 172.21.15.66:0/1821941601 shutdown_connections 2023-07-21T22:20:46.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 --2- 172.21.15.66:0/1821941601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240ffb90 0x7f8d240fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 --2- 172.21.15.66:0/1821941601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24101720 0x7f8d24107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 --2- 172.21.15.66:0/1821941601 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24100d90 0x7f8d241011e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 -- 172.21.15.66:0/1821941601 >> 172.21.15.66:0/1821941601 conn(0x7f8d240fb1a0 msgr2=0x7f8d240fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:46.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 -- 172.21.15.66:0/1821941601 shutdown_connections 2023-07-21T22:20:46.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.339+0000 7f8d2b691700 1 -- 172.21.15.66:0/1821941601 wait complete. 2023-07-21T22:20:46.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.340+0000 7f8d2b691700 1 Processor -- start 2023-07-21T22:20:46.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.340+0000 7f8d2b691700 1 -- start start 2023-07-21T22:20:46.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2b691700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240ffb90 0x7f8d24197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2b691700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 0x7f8d24198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2b691700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24101720 0x7f8d2419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2b691700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2419f670 con 0x7f8d24101720 2023-07-21T22:20:46.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2b691700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2419f7b0 con 0x7f8d240ffb90 2023-07-21T22:20:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2b691700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2419f940 con 0x7f8d24100d90 2023-07-21T22:20:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 0x7f8d24198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:46.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 0x7f8d24198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35792/0 (socket says 172.21.15.66:35792) 2023-07-21T22:20:46.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 -- 172.21.15.66:0/2383835367 learned_addr learned my addr 172.21.15.66:0/2383835367 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:46.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d29c2e700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24101720 0x7f8d2419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:46.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d2942d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240ffb90 0x7f8d24197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:46.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 -- 172.21.15.66:0/2383835367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240ffb90 msgr2=0x7f8d24197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:46.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240ffb90 0x7f8d24197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 -- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24101720 msgr2=0x7f8d2419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:46.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24101720 0x7f8d2419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d28c2c700 1 -- 172.21.15.66:0/2383835367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140095f0 con 0x7f8d24100d90 2023-07-21T22:20:46.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.341+0000 7f8d29c2e700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24101720 0x7f8d2419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:46.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.342+0000 7f8d28c2c700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 0x7f8d24198020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d14004760 tx=0x7f8d14004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:46.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.342+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1401d070 con 0x7f8d24100d90 2023-07-21T22:20:46.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.342+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d14022470 con 0x7f8d24100d90 2023-07-21T22:20:46.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.342+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14017650 con 0x7f8d24100d90 2023-07-21T22:20:46.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.342+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2419fbc0 con 0x7f8d24100d90 2023-07-21T22:20:46.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.342+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2419ffd0 con 0x7f8d24100d90 2023-07-21T22:20:46.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.343+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d24192260 con 0x7f8d24100d90 2023-07-21T22:20:46.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.345+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8d14022870 con 0x7f8d24100d90 2023-07-21T22:20:46.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.347+0000 7f8d1a7fc700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8d100a1260 0x7f8d100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.347+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d140ce3b0 con 0x7f8d24100d90 2023-07-21T22:20:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.347+0000 7f8d2942d700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8d100a1260 0x7f8d100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.348+0000 7f8d2942d700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8d100a1260 0x7f8d100a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d24100bf0 tx=0x7f8d20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:46.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.349+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8d140977e0 con 0x7f8d24100d90 2023-07-21T22:20:46.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.564+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d24060950 con 0x7f8d100a1260 2023-07-21T22:20:46.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.565+0000 7f8d1a7fc700 1 -- 172.21.15.66:0/2383835367 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f8d24060950 con 0x7f8d100a1260 2023-07-21T22:20:46.610 INFO:teuthology.orchestra.run.smithi066.stdout:HOST ADDR LABELS STATUS 2023-07-21T22:20:46.610 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 172.21.15.66 2023-07-21T22:20:46.610 INFO:teuthology.orchestra.run.smithi066.stdout:smithi104 172.21.15.104 2023-07-21T22:20:46.610 INFO:teuthology.orchestra.run.smithi066.stdout:smithi149 172.21.15.149 2023-07-21T22:20:46.610 INFO:teuthology.orchestra.run.smithi066.stdout:3 hosts in cluster 2023-07-21T22:20:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.567+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8d100a1260 msgr2=0x7f8d100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.567+0000 7f8d2b691700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8d100a1260 0x7f8d100a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d24100bf0 tx=0x7f8d20009260 comp rx=0 tx=0).stop 2023-07-21T22:20:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.567+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 msgr2=0x7f8d24198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.567+0000 7f8d2b691700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 0x7f8d24198020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d14004760 tx=0x7f8d14004790 comp rx=0 tx=0).stop 2023-07-21T22:20:46.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 shutdown_connections 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f8d100a1260 0x7f8d100a3710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d240ffb90 0x7f8d24197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d24101720 0x7f8d2419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 --2- 172.21.15.66:0/2383835367 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d24100d90 0x7f8d24198020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 >> 172.21.15.66:0/2383835367 conn(0x7f8d240fb1a0 msgr2=0x7f8d241066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 shutdown_connections 2023-07-21T22:20:46.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:46.568+0000 7f8d2b691700 1 -- 172.21.15.66:0/2383835367 wait complete. 2023-07-21T22:20:47.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-21T22:20:47.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:47 smithi066 ceph-mon[126379]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-07-21T22:20:47.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:47 smithi104 ceph-mon[129021]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-07-21T22:20:47.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:47 smithi149 ceph-mon[129268]: pgmap v328: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2023-07-21T22:20:47.896 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:48.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:48 smithi066 ceph-mon[126379]: from='client.25183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:48.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:48 smithi104 ceph-mon[129021]: from='client.25183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:48.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:48 smithi149 ceph-mon[129268]: from='client.25183 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:49.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 -- 172.21.15.66:0/3643242414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c900ffb90 msgr2=0x7f3c900fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:49.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 --2- 172.21.15.66:0/3643242414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c900ffb90 0x7f3c900fffa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009990 tx=0x7f3c80009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 -- 172.21.15.66:0/3643242414 shutdown_connections 2023-07-21T22:20:49.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 --2- 172.21.15.66:0/3643242414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 0x7f3c90107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 --2- 172.21.15.66:0/3643242414 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c900ffb90 0x7f3c900fffa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 --2- 172.21.15.66:0/3643242414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c90100d90 0x7f3c901011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 -- 172.21.15.66:0/3643242414 >> 172.21.15.66:0/3643242414 conn(0x7f3c900fb1a0 msgr2=0x7f3c900fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:49.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 -- 172.21.15.66:0/3643242414 shutdown_connections 2023-07-21T22:20:49.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.396+0000 7f3c95df6700 1 -- 172.21.15.66:0/3643242414 wait complete. 2023-07-21T22:20:49.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 Processor -- start 2023-07-21T22:20:49.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 -- start start 2023-07-21T22:20:49.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c900ffb90 0x7f3c900742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 0x7f3c90072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 0x7f3c90072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900735c0 con 0x7f3c90100d90 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90073730 con 0x7f3c90101720 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.398+0000 7f3c95df6700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9019f950 con 0x7f3c900ffb90 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 0x7f3c90072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:49.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 0x7f3c90072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c900ffb90 0x7f3c900742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 0x7f3c90072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:33322/0 (socket says 172.21.15.66:33322) 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8effd700 1 -- 172.21.15.66:0/43460904 learned_addr learned my addr 172.21.15.66:0/43460904 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8f7fe700 1 -- 172.21.15.66:0/43460904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 msgr2=0x7f3c90072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8f7fe700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 0x7f3c90072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8f7fe700 1 -- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 msgr2=0x7f3c90072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:49.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8f7fe700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 0x7f3c90072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8f7fe700 1 -- 172.21.15.66:0/43460904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800095f0 con 0x7f3c900ffb90 2023-07-21T22:20:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8ffff700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 0x7f3c90072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:49.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.399+0000 7f3c8effd700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 0x7f3c90072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:20:49.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.400+0000 7f3c8f7fe700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c900ffb90 0x7f3c900742b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b680 tx=0x7f3c80005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:49.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.400+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8001d070 con 0x7f3c900ffb90 2023-07-21T22:20:49.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.400+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9019fbd0 con 0x7f3c900ffb90 2023-07-21T22:20:49.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.400+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c80021410 con 0x7f3c900ffb90 2023-07-21T22:20:49.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.400+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80021ea0 con 0x7f3c900ffb90 2023-07-21T22:20:49.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.400+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c901a0040 con 0x7f3c900ffb90 2023-07-21T22:20:49.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:49 smithi066 ceph-mon[126379]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-21T22:20:49.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.401+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c901071d0 con 0x7f3c900ffb90 2023-07-21T22:20:49.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.403+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3c800217f0 con 0x7f3c900ffb90 2023-07-21T22:20:49.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.404+0000 7f3c8cff9700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3c780a1260 0x7f3c780a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:49.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.405+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3c800cde80 con 0x7f3c900ffb90 2023-07-21T22:20:49.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.405+0000 7f3c8effd700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3c780a1260 0x7f3c780a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:49.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.406+0000 7f3c8effd700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3c780a1260 0x7f3c780a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c840104e0 tx=0x7f3c84007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:49.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.407+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3c80097190 con 0x7f3c900ffb90 2023-07-21T22:20:49.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:49 smithi104 ceph-mon[129021]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-21T22:20:49.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.623+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c90060950 con 0x7f3c780a1260 2023-07-21T22:20:49.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.624+0000 7f3c8cff9700 1 -- 172.21.15.66:0/43460904 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f3c90060950 con 0x7f3c780a1260 2023-07-21T22:20:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-21T22:20:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 3m ago LVM detected, locked 2023-07-21T22:20:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 2m ago LVM detected, locked 2023-07-21T22:20:49.669 INFO:teuthology.orchestra.run.smithi066.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 2m ago LVM detected, locked 2023-07-21T22:20:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3c780a1260 msgr2=0x7f3c780a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3c780a1260 0x7f3c780a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3c840104e0 tx=0x7f3c84007480 comp rx=0 tx=0).stop 2023-07-21T22:20:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c900ffb90 msgr2=0x7f3c900742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:49.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c900ffb90 0x7f3c900742b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b680 tx=0x7f3c80005740 comp rx=0 tx=0).stop 2023-07-21T22:20:49.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 shutdown_connections 2023-07-21T22:20:49.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3c780a1260 0x7f3c780a3710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c90101720 0x7f3c90072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c900ffb90 0x7f3c900742b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 --2- 172.21.15.66:0/43460904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3c90100d90 0x7f3c90072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 >> 172.21.15.66:0/43460904 conn(0x7f3c900fb1a0 msgr2=0x7f3c90104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 shutdown_connections 2023-07-21T22:20:49.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:49.627+0000 7f3c95df6700 1 -- 172.21.15.66:0/43460904 wait complete. 2023-07-21T22:20:49.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:49 smithi149 ceph-mon[129268]: pgmap v329: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-07-21T22:20:50.299 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T22:20:50.313 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:20:50.314 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-21T22:20:51.055 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:51 smithi104 ceph-mon[129021]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:51.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:51 smithi104 ceph-mon[129021]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:51.688 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:51 smithi066 ceph-mon[126379]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:51.688 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:51 smithi066 ceph-mon[126379]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:51.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:51 smithi149 ceph-mon[129268]: from='client.25189 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:20:51.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:51 smithi149 ceph-mon[129268]: pgmap v330: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:20:52.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.562+0000 7fde79f59700 1 -- 172.21.15.66:0/1387258008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74100610 msgr2=0x7fde741029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.562+0000 7fde79f59700 1 --2- 172.21.15.66:0/1387258008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74100610 0x7fde741029f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fde64009990 tx=0x7fde64009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 -- 172.21.15.66:0/1387258008 shutdown_connections 2023-07-21T22:20:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 --2- 172.21.15.66:0/1387258008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde74105850 0x7fde74107cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 --2- 172.21.15.66:0/1387258008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74100610 0x7fde741029f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 --2- 172.21.15.66:0/1387258008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde74102f30 0x7fde74105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 -- 172.21.15.66:0/1387258008 >> 172.21.15.66:0/1387258008 conn(0x7fde740fa0b0 msgr2=0x7fde740fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 -- 172.21.15.66:0/1387258008 shutdown_connections 2023-07-21T22:20:52.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.563+0000 7fde79f59700 1 -- 172.21.15.66:0/1387258008 wait complete. 2023-07-21T22:20:52.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.564+0000 7fde79f59700 1 Processor -- start 2023-07-21T22:20:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.564+0000 7fde79f59700 1 -- start start 2023-07-21T22:20:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde79f59700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde74100610 0x7fde740fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde79f59700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 0x7fde740fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde79f59700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde74105850 0x7fde740feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde79f59700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde740ff280 con 0x7fde74102f30 2023-07-21T22:20:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde79f59700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde740ff3f0 con 0x7fde74105850 2023-07-21T22:20:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde79f59700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7419f8e0 con 0x7fde74100610 2023-07-21T22:20:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde72ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 0x7fde740fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde737fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde74100610 0x7fde740fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde72ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 0x7fde740fe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48596/0 (socket says 172.21.15.66:48596) 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde72ffd700 1 -- 172.21.15.66:0/3369075537 learned_addr learned my addr 172.21.15.66:0/3369075537 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.565+0000 7fde73fff700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde74105850 0x7fde740feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde72ffd700 1 -- 172.21.15.66:0/3369075537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde74100610 msgr2=0x7fde740fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde72ffd700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde74100610 0x7fde740fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde72ffd700 1 -- 172.21.15.66:0/3369075537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde74105850 msgr2=0x7fde740feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde72ffd700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde74105850 0x7fde740feb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde72ffd700 1 -- 172.21.15.66:0/3369075537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640095f0 con 0x7fde74102f30 2023-07-21T22:20:52.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde72ffd700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 0x7fde740fe5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fde6800e9a0 tx=0x7fde6800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68009790 con 0x7fde74102f30 2023-07-21T22:20:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde68010430 con 0x7fde74102f30 2023-07-21T22:20:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7419fbc0 con 0x7fde74102f30 2023-07-21T22:20:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68018680 con 0x7fde74102f30 2023-07-21T22:20:52.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.566+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde741a00c0 con 0x7fde74102f30 2023-07-21T22:20:52.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.567+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7410b360 con 0x7fde74102f30 2023-07-21T22:20:52.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.570+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fde680105a0 con 0x7fde74102f30 2023-07-21T22:20:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.571+0000 7fde70ff9700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fde5c0a11e0 0x7fde5c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:52.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.571+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fde68014070 con 0x7fde74102f30 2023-07-21T22:20:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.571+0000 7fde737fe700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fde5c0a11e0 0x7fde5c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:52.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.572+0000 7fde737fe700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fde5c0a11e0 0x7fde5c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fde6400b2e0 tx=0x7fde64005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:52.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.575+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fde68051770 con 0x7fde74102f30 2023-07-21T22:20:52.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.831+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fde741a0670 con 0x7fde74102f30 2023-07-21T22:20:52.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.832+0000 7fde70ff9700 1 -- 172.21.15.66:0/3369075537 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2909 (secure 0 0 0) 0x7fde68096240 con 0x7fde74102f30 2023-07-21T22:20:52.876 INFO:teuthology.orchestra.run.smithi066.stdout:e23 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout:legacy client fscid: 1 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout:Filesystem 'cephfs' (1) 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout:fs_name cephfs 2023-07-21T22:20:52.877 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 23 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:created 2023-07-21T22:19:06.899289+0000 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:modified 2023-07-21T22:19:34.299634+0000 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:tableserver 0 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:root 0 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:session_timeout 300 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:session_autoclose 300 2023-07-21T22:20:52.878 INFO:teuthology.orchestra.run.smithi066.stdout:max_file_size 1099511627776 2023-07-21T22:20:52.879 INFO:teuthology.orchestra.run.smithi066.stdout:max_xattr_size 65536 2023-07-21T22:20:52.879 INFO:teuthology.orchestra.run.smithi066.stdout:required_client_features {} 2023-07-21T22:20:52.879 INFO:teuthology.orchestra.run.smithi066.stdout:last_failure 0 2023-07-21T22:20:52.879 INFO:teuthology.orchestra.run.smithi066.stdout:last_failure_osd_epoch 0 2023-07-21T22:20:52.879 INFO:teuthology.orchestra.run.smithi066.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T22:20:52.879 INFO:teuthology.orchestra.run.smithi066.stdout:max_mds 3 2023-07-21T22:20:52.880 INFO:teuthology.orchestra.run.smithi066.stdout:in 0,1,2 2023-07-21T22:20:52.880 INFO:teuthology.orchestra.run.smithi066.stdout:up {0=24572,1=24530,2=24520} 2023-07-21T22:20:52.880 INFO:teuthology.orchestra.run.smithi066.stdout:failed 2023-07-21T22:20:52.880 INFO:teuthology.orchestra.run.smithi066.stdout:damaged 2023-07-21T22:20:52.880 INFO:teuthology.orchestra.run.smithi066.stdout:stopped 2023-07-21T22:20:52.880 INFO:teuthology.orchestra.run.smithi066.stdout:data_pools [3,4] 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:metadata_pool 2 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:inline_data disabled 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:balancer 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:bal_rank_mask -1 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:standby_count_wanted 0 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.b{0:24572} state up:active seq 3 addr [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.c{0:24502} state up:standby-replay seq 1 addr [v2:172.21.15.149:6836/1037928792,v1:172.21.15.149:6837/1037928792] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.881 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.h{1:24530} state up:active seq 12 addr [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.e{1:24497} state up:standby-replay seq 1 addr [v2:172.21.15.104:6834/1273148910,v1:172.21.15.104:6835/1273148910] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.i{2:24520} state up:active seq 12 addr [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.f{2:24490} state up:standby-replay seq 1 addr [v2:172.21.15.149:6834/1642617037,v1:172.21.15.149:6835/1642617037] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout:Standby daemons: 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:52.882 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.j{-1:14433} state up:standby seq 1 addr [v2:172.21.15.66:6834/1071874116,v1:172.21.15.66:6835/1071874116] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.883 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.g{-1:14496} state up:standby seq 1 addr [v2:172.21.15.66:6838/1234257491,v1:172.21.15.66:6839/1234257491] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.883 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.a{-1:14508} state up:standby seq 1 addr [v2:172.21.15.66:6840/3476312972,v1:172.21.15.66:6841/3476312972] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.883 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.l{-1:24439} state up:standby seq 1 addr [v2:172.21.15.149:6832/2009309286,v1:172.21.15.149:6833/2009309286] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.883 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.k{-1:24461} state up:standby seq 1 addr [v2:172.21.15.104:6832/3341651114,v1:172.21.15.104:6833/3341651114] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.883 INFO:teuthology.orchestra.run.smithi066.stdout:[mds.d{-1:24485} state up:standby seq 1 addr [v2:172.21.15.66:6836/1892710488,v1:172.21.15.66:6837/1892710488] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T22:20:52.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fde5c0a11e0 msgr2=0x7fde5c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fde5c0a11e0 0x7fde5c0a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fde6400b2e0 tx=0x7fde64005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 msgr2=0x7fde740fe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 0x7fde740fe5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fde6800e9a0 tx=0x7fde6800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 shutdown_connections 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fde5c0a11e0 0x7fde5c0a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde74105850 0x7fde740feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde74100610 0x7fde740fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 --2- 172.21.15.66:0/3369075537 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fde74102f30 0x7fde740fe5c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:52.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 >> 172.21.15.66:0/3369075537 conn(0x7fde740fa0b0 msgr2=0x7fde74101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:52.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 shutdown_connections 2023-07-21T22:20:52.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:52.834+0000 7fde79f59700 1 -- 172.21.15.66:0/3369075537 wait complete. 2023-07-21T22:20:52.886 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:20:53.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-21T22:20:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:53 smithi104 ceph-mon[129021]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:53 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3369075537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:20:53.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:53 smithi066 ceph-mon[126379]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:53.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:53 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3369075537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:20:53.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:53 smithi149 ceph-mon[129268]: pgmap v331: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:53.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:53 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3369075537' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T22:20:54.103 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:20:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 -- 172.21.15.66:0/105309803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3228100d90 msgr2=0x7f32281011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 --2- 172.21.15.66:0/105309803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3228100d90 0x7f32281011e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f321c009990 tx=0x7f321c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 -- 172.21.15.66:0/105309803 shutdown_connections 2023-07-21T22:20:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 --2- 172.21.15.66:0/105309803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f32280ffb90 0x7f32280fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 --2- 172.21.15.66:0/105309803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3228101720 0x7f3228107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 --2- 172.21.15.66:0/105309803 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3228100d90 0x7f32281011e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 -- 172.21.15.66:0/105309803 >> 172.21.15.66:0/105309803 conn(0x7f32280fb1a0 msgr2=0x7f32280fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 -- 172.21.15.66:0/105309803 shutdown_connections 2023-07-21T22:20:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.594+0000 7f322dbd7700 1 -- 172.21.15.66:0/105309803 wait complete. 2023-07-21T22:20:55.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 Processor -- start 2023-07-21T22:20:55.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 -- start start 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32280ffb90 0x7f3228197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3228100d90 0x7f3228198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3228101720 0x7f322819f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819f670 con 0x7f32280ffb90 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819f7b0 con 0x7f3228100d90 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.595+0000 7f322dbd7700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322819f940 con 0x7f3228101720 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f32277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32280ffb90 0x7f3228197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f32277fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32280ffb90 0x7f3228197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48628/0 (socket says 172.21.15.66:48628) 2023-07-21T22:20:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3228100d90 0x7f3228198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f32277fe700 1 -- 172.21.15.66:0/46868049 learned_addr learned my addr 172.21.15.66:0/46868049 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3227fff700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3228101720 0x7f322819f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 -- 172.21.15.66:0/46868049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3228101720 msgr2=0x7f322819f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3228101720 0x7f322819f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 -- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32280ffb90 msgr2=0x7f3228197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32280ffb90 0x7f3228197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 -- 172.21.15.66:0/46868049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0095f0 con 0x7f3228100d90 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.596+0000 7f3226ffd700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3228100d90 0x7f3228198020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b680 tx=0x7f321c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.597+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c01d070 con 0x7f3228100d90 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.597+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322819fbc0 con 0x7f3228100d90 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.597+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321c021410 con 0x7f3228100d90 2023-07-21T22:20:55.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.597+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c017650 con 0x7f3228100d90 2023-07-21T22:20:55.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.597+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32281a0030 con 0x7f3228100d90 2023-07-21T22:20:55.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.598+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3228192260 con 0x7f3228100d90 2023-07-21T22:20:55.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.600+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f321c0217f0 con 0x7f3228100d90 2023-07-21T22:20:55.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.601+0000 7f3224ff9700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f32100a11e0 0x7f32100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:55.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.602+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f321c0ce8d0 con 0x7f3228100d90 2023-07-21T22:20:55.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.602+0000 7f32277fe700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f32100a11e0 0x7f32100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:55.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.602+0000 7f32277fe700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f32100a11e0 0x7f32100a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3218005950 tx=0x7f3218009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:55.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.603+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f321c097b80 con 0x7f3228100d90 2023-07-21T22:20:55.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:55 smithi066 ceph-mon[126379]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:55.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:55 smithi149 ceph-mon[129268]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:55.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.815+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f322802c850 con 0x7f3228100d90 2023-07-21T22:20:55.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.816+0000 7f3224ff9700 1 -- 172.21.15.66:0/46868049 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) v1 ==== 56+0+5682 (secure 0 0 0) 0x7f321c09b1a0 con 0x7f3228100d90 2023-07-21T22:20:55.860 INFO:teuthology.orchestra.run.smithi066.stdout:epoch 93 2023-07-21T22:20:55.861 INFO:teuthology.orchestra.run.smithi066.stdout:fsid 6bcb348c-2813-11ee-9b34-001a4aab830c 2023-07-21T22:20:55.861 INFO:teuthology.orchestra.run.smithi066.stdout:created 2023-07-21T22:11:36.433200+0000 2023-07-21T22:20:55.861 INFO:teuthology.orchestra.run.smithi066.stdout:modified 2023-07-21T22:20:38.285621+0000 2023-07-21T22:20:55.861 INFO:teuthology.orchestra.run.smithi066.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-21T22:20:55.861 INFO:teuthology.orchestra.run.smithi066.stdout:crush_version 27 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:full_ratio 0.95 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:backfillfull_ratio 0.9 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:nearfull_ratio 0.85 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:require_min_compat_client luminous 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:min_compat_client luminous 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:require_osd_release reef 2023-07-21T22:20:55.862 INFO:teuthology.orchestra.run.smithi066.stdout:stretch_mode_enabled false 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:max_osd 12 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6802/1273732852,v1:172.21.15.66:6803/1273732852] [v2:172.21.15.66:6804/1273732852,v1:172.21.15.66:6805/1273732852] exists,up 165f7676-76ef-4250-b74b-48159136d422 2023-07-21T22:20:55.863 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/1799730222,v1:172.21.15.104:6801/1799730222] [v2:172.21.15.104:6802/1799730222,v1:172.21.15.104:6803/1799730222] exists,up 77b37050-b8b8-4218-b605-b362f6375262 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 up in weight 1 up_from 18 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/943388416,v1:172.21.15.149:6801/943388416] [v2:172.21.15.149:6802/943388416,v1:172.21.15.149:6803/943388416] exists,up 502c7d03-af4c-4cbb-ab10-7bd70bb07c12 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 up in weight 1 up_from 26 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6810/4147082736,v1:172.21.15.66:6811/4147082736] [v2:172.21.15.66:6812/4147082736,v1:172.21.15.66:6813/4147082736] exists,up b3f495e8-3d23-4125-96ea-0e4459f72a13 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 up in weight 1 up_from 34 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/3974235171,v1:172.21.15.104:6809/3974235171] [v2:172.21.15.104:6810/3974235171,v1:172.21.15.104:6811/3974235171] exists,up 0864a013-76a0-4e35-91a1-2405091d259e 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 up in weight 1 up_from 39 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/2736316265,v1:172.21.15.149:6809/2736316265] [v2:172.21.15.149:6810/2736316265,v1:172.21.15.149:6811/2736316265] exists,up d8b253aa-c95f-488c-af70-0d2f6a83e718 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6818/2608304075,v1:172.21.15.66:6819/2608304075] [v2:172.21.15.66:6820/2608304075,v1:172.21.15.66:6821/2608304075] exists,up 60ab1523-87ee-4f6c-a457-cc311b885a18 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/1910018308,v1:172.21.15.104:6817/1910018308] [v2:172.21.15.104:6818/1910018308,v1:172.21.15.104:6819/1910018308] exists,up 8896ebbe-b361-411e-a5fc-f818267ad8e2 2023-07-21T22:20:55.864 INFO:teuthology.orchestra.run.smithi066.stdout:osd.8 up in weight 1 up_from 60 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/1996550641,v1:172.21.15.149:6817/1996550641] [v2:172.21.15.149:6818/1996550641,v1:172.21.15.149:6819/1996550641] exists,up f7e61e60-51c6-484e-a89b-3e079f2bdb15 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:osd.9 up in weight 1 up_from 68 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.66:6826/825018601,v1:172.21.15.66:6827/825018601] [v2:172.21.15.66:6828/825018601,v1:172.21.15.66:6829/825018601] exists,up a6812986-f4de-4f88-8aec-7f112164c075 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:osd.10 up in weight 1 up_from 73 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/3265864999,v1:172.21.15.104:6825/3265864999] [v2:172.21.15.104:6826/3265864999,v1:172.21.15.104:6827/3265864999] exists,up 24a4c4d2-8441-4081-a35b-b758c5753b8b 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/2237094047,v1:172.21.15.149:6825/2237094047] [v2:172.21.15.149:6826/2237094047,v1:172.21.15.149:6827/2237094047] exists,up f3e27952-c49b-4512-a879-f0b53e0aea22 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.4 [2,11] 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.a [2,11] 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 2.1a [8,4] 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.6 [4,7] 2023-07-21T22:20:55.865 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.1d [4,6] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.20 [5,9] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.21 [5,9] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.27 [4,8] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 3.29 [3,9] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.10 [9,10] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.17 [9,8] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.1d [5,8] 2023-07-21T22:20:55.866 INFO:teuthology.orchestra.run.smithi066.stdout:pg_upmap_items 4.1f [9,8] 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/662946744 expires 2023-07-22T22:12:36.098613+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6800/3065709937 expires 2023-07-22T22:12:36.098613+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/1244827991 expires 2023-07-22T22:12:36.098613+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6800/979491945 expires 2023-07-22T22:12:09.674594+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6801/3065709937 expires 2023-07-22T22:12:36.098613+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/3161416321 expires 2023-07-22T22:12:09.674594+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/3483118396 expires 2023-07-22T22:12:09.674594+0000 2023-07-21T22:20:55.867 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:6801/979491945 expires 2023-07-22T22:12:09.674594+0000 2023-07-21T22:20:55.868 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/690385827 expires 2023-07-22T22:12:09.674594+0000 2023-07-21T22:20:55.868 INFO:teuthology.orchestra.run.smithi066.stdout:blocklist 172.21.15.66:0/3486180168 expires 2023-07-22T22:12:36.098613+0000 2023-07-21T22:20:55.868 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f32100a11e0 msgr2=0x7f32100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f32100a11e0 0x7f32100a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3218005950 tx=0x7f3218009260 comp rx=0 tx=0).stop 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3228100d90 msgr2=0x7f3228198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3228100d90 0x7f3228198020 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f321c00b680 tx=0x7f321c005740 comp rx=0 tx=0).stop 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 shutdown_connections 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f32100a11e0 0x7f32100a3690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3228101720 0x7f322819f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f32280ffb90 0x7f3228197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 --2- 172.21.15.66:0/46868049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3228100d90 0x7f3228198020 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:55.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.818+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 >> 172.21.15.66:0/46868049 conn(0x7f32280fb1a0 msgr2=0x7f32281066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:55.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.819+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 shutdown_connections 2023-07-21T22:20:55.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:55.819+0000 7f322dbd7700 1 -- 172.21.15.66:0/46868049 wait complete. 2023-07-21T22:20:55.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:55 smithi104 ceph-mon[129021]: pgmap v332: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:20:56.305 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-21T22:20:56.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:20:56.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 -- 172.21.15.66:0/1400384344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc101680 msgr2=0x7fc3cc107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/1400384344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc101680 0x7fc3cc107f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b0a0 tx=0x7fc3c400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 -- 172.21.15.66:0/1400384344 shutdown_connections 2023-07-21T22:20:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/1400384344 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc101680 0x7fc3cc107f40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/1400384344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 0x7fc3cc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/1400384344 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 -- 172.21.15.66:0/1400384344 >> 172.21.15.66:0/1400384344 conn(0x7fc3cc0fb0e0 msgr2=0x7fc3cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 -- 172.21.15.66:0/1400384344 shutdown_connections 2023-07-21T22:20:56.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.485+0000 7fc3d3bff700 1 -- 172.21.15.66:0/1400384344 wait complete. 2023-07-21T22:20:56.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.486+0000 7fc3d3bff700 1 Processor -- start 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.486+0000 7fc3d3bff700 1 -- start start 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d3bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 0x7fc3cc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d3bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d3bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3cc101680 0x7fc3cc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d3bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc19f6c0 con 0x7fc3cc100cf0 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d3bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc19f800 con 0x7fc3cc101680 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d3bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc19f990 con 0x7fc3cc0ffb00 2023-07-21T22:20:56.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d199b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 0x7fc3cc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48634/0 (socket says 172.21.15.66:48634) 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d199b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 0x7fc3cc197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:34904/0 (socket says 172.21.15.66:34904) 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 -- 172.21.15.66:0/382647836 learned_addr learned my addr 172.21.15.66:0/382647836 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d219c700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3cc101680 0x7fc3cc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 -- 172.21.15.66:0/382647836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 msgr2=0x7fc3cc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 0x7fc3cc197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 -- 172.21.15.66:0/382647836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3cc101680 msgr2=0x7fc3cc19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3cc101680 0x7fc3cc19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.487+0000 7fc3d119a700 1 -- 172.21.15.66:0/382647836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4009d00 con 0x7fc3cc100cf0 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.488+0000 7fc3d119a700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc198080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e9a0 tx=0x7fc3bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.488+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc009790 con 0x7fc3cc100cf0 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.488+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3bc010430 con 0x7fc3cc100cf0 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.488+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc018680 con 0x7fc3cc100cf0 2023-07-21T22:20:56.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.488+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc19fc70 con 0x7fc3cc100cf0 2023-07-21T22:20:56.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.488+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc1a0130 con 0x7fc3cc100cf0 2023-07-21T22:20:56.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.489+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc10b330 con 0x7fc3cc100cf0 2023-07-21T22:20:56.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.493+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc3bc0105a0 con 0x7fc3cc100cf0 2023-07-21T22:20:56.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.494+0000 7fc3c2ffd700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc3b80a1220 0x7fc3b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.494+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc3bc014070 con 0x7fc3cc100cf0 2023-07-21T22:20:56.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.494+0000 7fc3d199b700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc3b80a1220 0x7fc3b80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.495+0000 7fc3d199b700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc3b80a1220 0x7fc3b80a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc100b50 tx=0x7fc3c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:56.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.496+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc3bc096430 con 0x7fc3cc100cf0 2023-07-21T22:20:56.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:56 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/46868049' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:20:56.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:56 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/46868049' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:20:56.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.707+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3cc10b5c0 con 0x7fc3cc100cf0 2023-07-21T22:20:56.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.709+0000 7fc3c2ffd700 1 -- 172.21.15.66:0/382647836 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7fc3cc10b5c0 con 0x7fc3cc100cf0 2023-07-21T22:20:56.754 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:56.754 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:20:56.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc3b80a1220 msgr2=0x7fc3b80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc3b80a1220 0x7fc3b80a36d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc100b50 tx=0x7fc3c8009260 comp rx=0 tx=0).stop 2023-07-21T22:20:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 msgr2=0x7fc3cc198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc198080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc00e9a0 tx=0x7fc3bc00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 shutdown_connections 2023-07-21T22:20:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fc3b80a1220 0x7fc3b80a36d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3cc101680 0x7fc3cc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3cc0ffb00 0x7fc3cc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 --2- 172.21.15.66:0/382647836 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3cc100cf0 0x7fc3cc198080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.711+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 >> 172.21.15.66:0/382647836 conn(0x7fc3cc0fb0e0 msgr2=0x7fc3cc1064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:56.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.712+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 shutdown_connections 2023-07-21T22:20:56.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.712+0000 7fc3d3bff700 1 -- 172.21.15.66:0/382647836 wait complete. 2023-07-21T22:20:56.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:20:56.776 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:20:56.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:56 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/46868049' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T22:20:56.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 -- 172.21.15.66:0/1745269440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc101660 msgr2=0x7fb0cc107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/1745269440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc101660 0x7fb0cc107f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400b0a0 tx=0x7fb0c400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 -- 172.21.15.66:0/1745269440 shutdown_connections 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/1745269440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc101660 0x7fb0cc107f20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/1745269440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc0ffae0 0x7fb0cc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/1745269440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0cc100cd0 0x7fb0cc101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 -- 172.21.15.66:0/1745269440 >> 172.21.15.66:0/1745269440 conn(0x7fb0cc0fb0e0 msgr2=0x7fb0cc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:56.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 -- 172.21.15.66:0/1745269440 shutdown_connections 2023-07-21T22:20:56.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.942+0000 7fb0d0bff700 1 -- 172.21.15.66:0/1745269440 wait complete. 2023-07-21T22:20:56.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.943+0000 7fb0d0bff700 1 Processor -- start 2023-07-21T22:20:56.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.943+0000 7fb0d0bff700 1 -- start start 2023-07-21T22:20:56.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0d0bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc0ffae0 0x7fb0cc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0d0bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0cc100cd0 0x7fb0cc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0d0bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 0x7fb0cc19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0d0bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19f6f0 con 0x7fb0cc101660 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0d0bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19f830 con 0x7fb0cc0ffae0 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0d0bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19f9c0 con 0x7fb0cc100cd0 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 0x7fb0cc19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 0x7fb0cc19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48650/0 (socket says 172.21.15.66:48650) 2023-07-21T22:20:56.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 -- 172.21.15.66:0/630884504 learned_addr learned my addr 172.21.15.66:0/630884504 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0c3fff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0cc100cd0 0x7fb0cc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0ca59c700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc0ffae0 0x7fb0cc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 -- 172.21.15.66:0/630884504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0cc100cd0 msgr2=0x7fb0cc1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0cc100cd0 0x7fb0cc1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 -- 172.21.15.66:0/630884504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc0ffae0 msgr2=0x7fb0cc197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc0ffae0 0x7fb0cc197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:56.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0cad9d700 1 -- 172.21.15.66:0/630884504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4009d00 con 0x7fb0cc101660 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.944+0000 7fb0ca59c700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc0ffae0 0x7fb0cc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.945+0000 7fb0cad9d700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 0x7fb0cc19f120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400d010 tx=0x7fb0c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.945+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c400e040 con 0x7fb0cc101660 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.945+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc19fc40 con 0x7fb0cc101660 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.945+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0c4003e10 con 0x7fb0cc101660 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.945+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40052a0 con 0x7fb0cc101660 2023-07-21T22:20:56.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.945+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc1a0060 con 0x7fb0cc101660 2023-07-21T22:20:56.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.946+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0cc10b310 con 0x7fb0cc101660 2023-07-21T22:20:56.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.950+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb0c402e020 con 0x7fb0cc101660 2023-07-21T22:20:56.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.951+0000 7fb0c37fe700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0b80a1260 0x7fb0b80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:56.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.951+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb0c40ce270 con 0x7fb0cc101660 2023-07-21T22:20:56.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.951+0000 7fb0ca59c700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0b80a1260 0x7fb0b80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:56.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.952+0000 7fb0ca59c700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0b80a1260 0x7fb0b80a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc100b30 tx=0x7fb0bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:56.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:56.953+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb0c40975a0 con 0x7fb0cc101660 2023-07-21T22:20:57.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.162+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0cc1a06d0 con 0x7fb0cc101660 2023-07-21T22:20:57.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.163+0000 7fb0c37fe700 1 -- 172.21.15.66:0/630884504 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0c409abc0 con 0x7fb0cc101660 2023-07-21T22:20:57.208 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:20:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0b80a1260 msgr2=0x7fb0b80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0b80a1260 0x7fb0b80a3710 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc100b30 tx=0x7fb0bc009260 comp rx=0 tx=0).stop 2023-07-21T22:20:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 msgr2=0x7fb0cc19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 0x7fb0cc19f120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb0c400d010 tx=0x7fb0c400bc80 comp rx=0 tx=0).stop 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 shutdown_connections 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb0b80a1260 0x7fb0b80a3710 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0cc0ffae0 0x7fb0cc197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0cc101660 0x7fb0cc19f120 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 --2- 172.21.15.66:0/630884504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0cc100cd0 0x7fb0cc1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 >> 172.21.15.66:0/630884504 conn(0x7fb0cc0fb0e0 msgr2=0x7fb0cc0fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 shutdown_connections 2023-07-21T22:20:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.165+0000 7fb0d0bff700 1 -- 172.21.15.66:0/630884504 wait complete. 2023-07-21T22:20:57.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:20:57.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 -- 172.21.15.66:0/4155928656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d80ffae0 msgr2=0x7f31d80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 --2- 172.21.15.66:0/4155928656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d80ffae0 0x7f31d80ffef0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009990 tx=0x7f31c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 -- 172.21.15.66:0/4155928656 shutdown_connections 2023-07-21T22:20:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 --2- 172.21.15.66:0/4155928656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31d8101660 0x7f31d8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 --2- 172.21.15.66:0/4155928656 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d80ffae0 0x7f31d80ffef0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 --2- 172.21.15.66:0/4155928656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31d8100cd0 0x7f31d8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 -- 172.21.15.66:0/4155928656 >> 172.21.15.66:0/4155928656 conn(0x7f31d80fb0e0 msgr2=0x7f31d80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:57.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 -- 172.21.15.66:0/4155928656 shutdown_connections 2023-07-21T22:20:57.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.399+0000 7f31ddfff700 1 -- 172.21.15.66:0/4155928656 wait complete. 2023-07-21T22:20:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.400+0000 7f31ddfff700 1 Processor -- start 2023-07-21T22:20:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.400+0000 7f31ddfff700 1 -- start start 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31ddfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31d80ffae0 0x7f31d8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31ddfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 0x7f31d8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31ddfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31d8101660 0x7f31d819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31ddfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d819f5d0 con 0x7f31d8100cd0 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31ddfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d819f710 con 0x7f31d80ffae0 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31ddfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d819f8a0 con 0x7f31d8101660 2023-07-21T22:20:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 0x7f31d8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 0x7f31d8197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48656/0 (socket says 172.21.15.66:48656) 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 -- 172.21.15.66:0/2296062771 learned_addr learned my addr 172.21.15.66:0/2296062771 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31d77fe700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31d80ffae0 0x7f31d8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31d7fff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31d8101660 0x7f31d819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 -- 172.21.15.66:0/2296062771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31d8101660 msgr2=0x7f31d819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31d8101660 0x7f31d819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 -- 172.21.15.66:0/2296062771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31d80ffae0 msgr2=0x7f31d8197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31d80ffae0 0x7f31d8197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.401+0000 7f31cffff700 1 -- 172.21.15.66:0/2296062771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c80095f0 con 0x7f31d8100cd0 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.402+0000 7f31cffff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 0x7f31d8197f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f31c000e9a0 tx=0x7f31c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.402+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0009790 con 0x7f31d8100cd0 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.402+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d819fb80 con 0x7f31d8100cd0 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.402+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31c0010430 con 0x7f31d8100cd0 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.402+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0018680 con 0x7f31d8100cd0 2023-07-21T22:20:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.402+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d81a0090 con 0x7f31d8100cd0 2023-07-21T22:20:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.403+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d810b2b0 con 0x7f31d8100cd0 2023-07-21T22:20:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.405+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f31c00105a0 con 0x7f31d8100cd0 2023-07-21T22:20:57.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.407+0000 7f31d57fa700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f31c40a11e0 0x7f31c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.407+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f31c0014070 con 0x7f31d8100cd0 2023-07-21T22:20:57.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.407+0000 7f31d77fe700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f31c40a11e0 0x7f31c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.408+0000 7f31d77fe700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f31c40a11e0 0x7f31c40a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f31c800b2e0 tx=0x7f31c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:57.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.411+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f31c0051770 con 0x7f31d8100cd0 2023-07-21T22:20:57.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.621+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31d81a0640 con 0x7f31d8100cd0 2023-07-21T22:20:57.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.621+0000 7f31d57fa700 1 -- 172.21.15.66:0/2296062771 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31c0096240 con 0x7f31d8100cd0 2023-07-21T22:20:57.666 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:20:57.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f31c40a11e0 msgr2=0x7f31c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f31c40a11e0 0x7f31c40a3690 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f31c800b2e0 tx=0x7f31c8005960 comp rx=0 tx=0).stop 2023-07-21T22:20:57.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 msgr2=0x7f31d8197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 0x7f31d8197f90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f31c000e9a0 tx=0x7f31c000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 shutdown_connections 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f31c40a11e0 0x7f31c40a3690 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31d80ffae0 0x7f31d8197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31d8101660 0x7f31d819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 --2- 172.21.15.66:0/2296062771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f31d8100cd0 0x7f31d8197f90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 >> 172.21.15.66:0/2296062771 conn(0x7f31d80fb0e0 msgr2=0x7f31d80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 shutdown_connections 2023-07-21T22:20:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.624+0000 7f31ddfff700 1 -- 172.21.15.66:0/2296062771 wait complete. 2023-07-21T22:20:57.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:57 smithi066 ceph-mon[126379]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:57.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:57 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/382647836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:57.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:57 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/630884504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:57.688 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:20:57.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:57 smithi149 ceph-mon[129268]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:57.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:57 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/382647836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:57.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:57 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/630884504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:57.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:57 smithi104 ceph-mon[129021]: pgmap v333: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:20:57.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:57 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/382647836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:20:57.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:57 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/630884504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:20:57.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 -- 172.21.15.66:0/4053271024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c102e80 msgr2=0x7fd47c105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 --2- 172.21.15.66:0/4053271024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c102e80 0x7fd47c105260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd468009990 tx=0x7fd468009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 -- 172.21.15.66:0/4053271024 shutdown_connections 2023-07-21T22:20:57.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 --2- 172.21.15.66:0/4053271024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 0x7fd47c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 --2- 172.21.15.66:0/4053271024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd47c1057a0 0x7fd47c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 --2- 172.21.15.66:0/4053271024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c102e80 0x7fd47c105260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 -- 172.21.15.66:0/4053271024 >> 172.21.15.66:0/4053271024 conn(0x7fd47c0fa000 msgr2=0x7fd47c0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:57.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 -- 172.21.15.66:0/4053271024 shutdown_connections 2023-07-21T22:20:57.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.855+0000 7fd4831ff700 1 -- 172.21.15.66:0/4053271024 wait complete. 2023-07-21T22:20:57.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 Processor -- start 2023-07-21T22:20:57.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 -- start start 2023-07-21T22:20:57.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 0x7fd47c0ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd47c102e80 0x7fd47c0fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 0x7fd47c0fea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0ff1f0 con 0x7fd47c1057a0 2023-07-21T22:20:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0ff360 con 0x7fd47c100560 2023-07-21T22:20:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.856+0000 7fd4831ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c19f8c0 con 0x7fd47c102e80 2023-07-21T22:20:57.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 0x7fd47c0fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 0x7fd47c0fea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48664/0 (socket says 172.21.15.66:48664) 2023-07-21T22:20:57.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 -- 172.21.15.66:0/976756628 learned_addr learned my addr 172.21.15.66:0/976756628 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:57.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd480f9b700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 0x7fd47c0ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd47bfff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd47c102e80 0x7fd47c0fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 -- 172.21.15.66:0/976756628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd47c102e80 msgr2=0x7fd47c0fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd47c102e80 0x7fd47c0fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 -- 172.21.15.66:0/976756628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 msgr2=0x7fd47c0ffed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:57.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 0x7fd47c0ffed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:57.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 -- 172.21.15.66:0/976756628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680095f0 con 0x7fd47c1057a0 2023-07-21T22:20:57.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd480f9b700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 0x7fd47c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:20:57.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.857+0000 7fd48179c700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 0x7fd47c0fea50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd47400ef30 tx=0x7fd47400c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:57.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.858+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474018070 con 0x7fd47c1057a0 2023-07-21T22:20:57.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.858+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c19fb40 con 0x7fd47c1057a0 2023-07-21T22:20:57.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.858+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd47400f040 con 0x7fd47c1057a0 2023-07-21T22:20:57.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.858+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474003dc0 con 0x7fd47c1057a0 2023-07-21T22:20:57.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.858+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c1a00a0 con 0x7fd47c1057a0 2023-07-21T22:20:57.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.859+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c10b2f0 con 0x7fd47c1057a0 2023-07-21T22:20:57.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.861+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd47400caa0 con 0x7fd47c1057a0 2023-07-21T22:20:57.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.862+0000 7fd479ffb700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd46c0a11e0 0x7fd46c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:57.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.862+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd4740ccaf0 con 0x7fd47c1057a0 2023-07-21T22:20:57.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.863+0000 7fd480f9b700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd46c0a11e0 0x7fd46c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:57.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.863+0000 7fd480f9b700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd46c0a11e0 0x7fd46c0a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd470009cb0 tx=0x7fd470009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:57.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:57.866+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd474095da0 con 0x7fd47c1057a0 2023-07-21T22:20:58.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.076+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd47c1a0650 con 0x7fd47c1057a0 2023-07-21T22:20:58.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.077+0000 7fd479ffb700 1 -- 172.21.15.66:0/976756628 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4740993c0 con 0x7fd47c1057a0 2023-07-21T22:20:58.121 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:20:58.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd46c0a11e0 msgr2=0x7fd46c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd46c0a11e0 0x7fd46c0a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd470009cb0 tx=0x7fd470009260 comp rx=0 tx=0).stop 2023-07-21T22:20:58.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 msgr2=0x7fd47c0fea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 0x7fd47c0fea50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd47400ef30 tx=0x7fd47400c450 comp rx=0 tx=0).stop 2023-07-21T22:20:58.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 shutdown_connections 2023-07-21T22:20:58.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fd46c0a11e0 0x7fd46c0a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd47c100560 0x7fd47c0ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd47c1057a0 0x7fd47c0fea50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 --2- 172.21.15.66:0/976756628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd47c102e80 0x7fd47c0fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.079+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 >> 172.21.15.66:0/976756628 conn(0x7fd47c0fa000 msgr2=0x7fd47c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:58.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.080+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 shutdown_connections 2023-07-21T22:20:58.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.080+0000 7fd4831ff700 1 -- 172.21.15.66:0/976756628 wait complete. 2023-07-21T22:20:58.143 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:20:58.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 -- 172.21.15.66:0/4217285306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f80ffae0 msgr2=0x7f84f80ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/4217285306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f80ffae0 0x7f84f80ffef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009990 tx=0x7f84e8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 -- 172.21.15.66:0/4217285306 shutdown_connections 2023-07-21T22:20:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/4217285306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f80ffae0 0x7f84f80ffef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/4217285306 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8101660 0x7f84f8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/4217285306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8100cd0 0x7f84f8101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 -- 172.21.15.66:0/4217285306 >> 172.21.15.66:0/4217285306 conn(0x7f84f80fb0e0 msgr2=0x7f84f80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 -- 172.21.15.66:0/4217285306 shutdown_connections 2023-07-21T22:20:58.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.329+0000 7f84fe1ff700 1 -- 172.21.15.66:0/4217285306 wait complete. 2023-07-21T22:20:58.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.330+0000 7f84fe1ff700 1 Processor -- start 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.330+0000 7f84fe1ff700 1 -- start start 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84fe1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f80ffae0 0x7f84f8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84fe1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 0x7f84f8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84fe1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f8101660 0x7f84f819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84fe1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f819f610 con 0x7f84f8100cd0 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84fe1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f819f750 con 0x7f84f8101660 2023-07-21T22:20:58.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84fe1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f819f8e0 con 0x7f84f80ffae0 2023-07-21T22:20:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 0x7f84f8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f80ffae0 0x7f84f8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 0x7f84f8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48674/0 (socket says 172.21.15.66:48674) 2023-07-21T22:20:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 -- 172.21.15.66:0/823918011 learned_addr learned my addr 172.21.15.66:0/823918011 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f7fff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f8101660 0x7f84f819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 -- 172.21.15.66:0/823918011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f80ffae0 msgr2=0x7f84f8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f80ffae0 0x7f84f8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 -- 172.21.15.66:0/823918011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f8101660 msgr2=0x7f84f819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f8101660 0x7f84f819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.331+0000 7f84f6ffd700 1 -- 172.21.15.66:0/823918011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80095f0 con 0x7f84f8100cd0 2023-07-21T22:20:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.332+0000 7f84f6ffd700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 0x7f84f8197fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e9a0 tx=0x7f84ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.332+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec009790 con 0x7f84f8100cd0 2023-07-21T22:20:58.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.332+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f819fbc0 con 0x7f84f8100cd0 2023-07-21T22:20:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.332+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ec00cbd0 con 0x7f84f8100cd0 2023-07-21T22:20:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.332+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec018680 con 0x7f84f8100cd0 2023-07-21T22:20:58.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.332+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f81a0120 con 0x7f84f8100cd0 2023-07-21T22:20:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.333+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f810b300 con 0x7f84f8100cd0 2023-07-21T22:20:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.335+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f84ec00cd40 con 0x7f84f8100cd0 2023-07-21T22:20:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.337+0000 7f84f4ff9700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84e00a11e0 0x7f84e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.337+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f84ec014070 con 0x7f84f8100cd0 2023-07-21T22:20:58.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.337+0000 7f84f77fe700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84e00a11e0 0x7f84e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.338+0000 7f84f77fe700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84e00a11e0 0x7f84e00a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f84e800b2e0 tx=0x7f84e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:58.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.342+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f84ec051770 con 0x7f84f8100cd0 2023-07-21T22:20:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:58 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2296062771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:58.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:58 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/976756628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:58.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.556+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f84f81a06d0 con 0x7f84f8100cd0 2023-07-21T22:20:58.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.556+0000 7f84f4ff9700 1 -- 172.21.15.66:0/823918011 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f84ec096330 con 0x7f84f8100cd0 2023-07-21T22:20:58.601 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84e00a11e0 msgr2=0x7f84e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84e00a11e0 0x7f84e00a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f84e800b2e0 tx=0x7f84e8005fb0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 msgr2=0x7f84f8197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 0x7f84f8197fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00e9a0 tx=0x7f84ec00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 shutdown_connections 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f84e00a11e0 0x7f84e00a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84f8101660 0x7f84f819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f80ffae0 0x7f84f8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.604 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 --2- 172.21.15.66:0/823918011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f84f8100cd0 0x7f84f8197fd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 >> 172.21.15.66:0/823918011 conn(0x7f84f80fb0e0 msgr2=0x7f84f80fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 shutdown_connections 2023-07-21T22:20:58.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.559+0000 7f84fe1ff700 1 -- 172.21.15.66:0/823918011 wait complete. 2023-07-21T22:20:58.622 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:20:58.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:58 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2296062771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:58.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:58 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/976756628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:58.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 -- 172.21.15.66:0/723069372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c101680 msgr2=0x7f707c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 --2- 172.21.15.66:0/723069372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c101680 0x7f707c107f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f707400b0a0 tx=0x7f707400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 -- 172.21.15.66:0/723069372 shutdown_connections 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 --2- 172.21.15.66:0/723069372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c101680 0x7f707c107f40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 --2- 172.21.15.66:0/723069372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c0ffb00 0x7f707c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 --2- 172.21.15.66:0/723069372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c100cf0 0x7f707c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 -- 172.21.15.66:0/723069372 >> 172.21.15.66:0/723069372 conn(0x7f707c0fb0e0 msgr2=0x7f707c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 -- 172.21.15.66:0/723069372 shutdown_connections 2023-07-21T22:20:58.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.786+0000 7f7083fff700 1 -- 172.21.15.66:0/723069372 wait complete. 2023-07-21T22:20:58.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.787+0000 7f7083fff700 1 Processor -- start 2023-07-21T22:20:58.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.787+0000 7f7083fff700 1 -- start start 2023-07-21T22:20:58.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7083fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7083fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c100cf0 0x7f707c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7083fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c101680 0x7f707c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7083fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c19f690 con 0x7f707c0ffb00 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7083fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c19f7d0 con 0x7f707c100cf0 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7083fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c19f960 con 0x7f707c101680 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48680/0 (socket says 172.21.15.66:48680) 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 -- 172.21.15.66:0/474477562 learned_addr learned my addr 172.21.15.66:0/474477562 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f708259c700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c101680 0x7f707c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f708159a700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c100cf0 0x7f707c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 -- 172.21.15.66:0/474477562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c101680 msgr2=0x7f707c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c101680 0x7f707c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 -- 172.21.15.66:0/474477562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c100cf0 msgr2=0x7f707c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c100cf0 0x7f707c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.788+0000 7f7081d9b700 1 -- 172.21.15.66:0/474477562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078009520 con 0x7f707c0ffb00 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.789+0000 7f7081d9b700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c197b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f707800ea50 tx=0x7f707800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.789+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800cbd0 con 0x7f707c0ffb00 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.789+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f707800cd30 con 0x7f707c0ffb00 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.789+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7074009d00 con 0x7f707c0ffb00 2023-07-21T22:20:58.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.789+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078018680 con 0x7f707c0ffb00 2023-07-21T22:20:58.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.789+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c1a0010 con 0x7f707c0ffb00 2023-07-21T22:20:58.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.790+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707c10b330 con 0x7f707c0ffb00 2023-07-21T22:20:58.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.792+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7078010430 con 0x7f707c0ffb00 2023-07-21T22:20:58.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.794+0000 7f7072ffd700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70680a12a0 0x7f70680a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:58.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.794+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7078014070 con 0x7f707c0ffb00 2023-07-21T22:20:58.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.794+0000 7f708159a700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70680a12a0 0x7f70680a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:58.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.794+0000 7f708159a700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70680a12a0 0x7f70680a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f706c009c60 tx=0x7f706c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:58.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:58.797+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f70780964a0 con 0x7f707c0ffb00 2023-07-21T22:20:58.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:58 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2296062771' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:20:58.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:58 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/976756628' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:20:59.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.055+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f707c10b550 con 0x7f707c0ffb00 2023-07-21T22:20:59.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.056+0000 7f7072ffd700 1 -- 172.21.15.66:0/474477562 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f70780ade60 con 0x7f707c0ffb00 2023-07-21T22:20:59.100 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:59.100 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:20:59.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70680a12a0 msgr2=0x7f70680a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70680a12a0 0x7f70680a3750 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f706c009c60 tx=0x7f706c009260 comp rx=0 tx=0).stop 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 msgr2=0x7f707c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c197b10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f707800ea50 tx=0x7f707800ed60 comp rx=0 tx=0).stop 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 shutdown_connections 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f70680a12a0 0x7f70680a3750 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c101680 0x7f707c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c197b10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 --2- 172.21.15.66:0/474477562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c100cf0 0x7f707c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.058+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 >> 172.21.15.66:0/474477562 conn(0x7f707c0fb0e0 msgr2=0x7f707c0fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:59.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.059+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 shutdown_connections 2023-07-21T22:20:59.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.059+0000 7f7083fff700 1 -- 172.21.15.66:0/474477562 wait complete. 2023-07-21T22:20:59.104 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:20:59.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:20:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 -- 172.21.15.66:0/2307228236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f340ffb00 msgr2=0x7f0f340fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2307228236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f340ffb00 0x7f0f340fff10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009990 tx=0x7f0f24009ca0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 -- 172.21.15.66:0/2307228236 shutdown_connections 2023-07-21T22:20:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2307228236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f340ffb00 0x7f0f340fff10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2307228236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f34101680 0x7f0f34107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2307228236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f34100cf0 0x7f0f34101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 -- 172.21.15.66:0/2307228236 >> 172.21.15.66:0/2307228236 conn(0x7f0f340fb0e0 msgr2=0x7f0f340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 -- 172.21.15.66:0/2307228236 shutdown_connections 2023-07-21T22:20:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.286+0000 7f0f39bff700 1 -- 172.21.15.66:0/2307228236 wait complete. 2023-07-21T22:20:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.287+0000 7f0f39bff700 1 Processor -- start 2023-07-21T22:20:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.287+0000 7f0f39bff700 1 -- start start 2023-07-21T22:20:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.287+0000 7f0f39bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 0x7f0f34195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.287+0000 7f0f39bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f34100cf0 0x7f0f34195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.287+0000 7f0f39bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f34101680 0x7f0f3419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f39bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3419d400 con 0x7f0f340ffb00 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f39bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3419d540 con 0x7f0f34100cf0 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f39bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3419d6d0 con 0x7f0f34101680 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 0x7f0f34195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 0x7f0f34195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48688/0 (socket says 172.21.15.66:48688) 2023-07-21T22:20:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 -- 172.21.15.66:0/2424611093 learned_addr learned my addr 172.21.15.66:0/2424611093 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f32ffd700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f34100cf0 0x7f0f34195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f33fff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f34101680 0x7f0f3419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 -- 172.21.15.66:0/2424611093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f34101680 msgr2=0x7f0f3419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f34101680 0x7f0f3419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 -- 172.21.15.66:0/2424611093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f34100cf0 msgr2=0x7f0f34195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f34100cf0 0x7f0f34195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 -- 172.21.15.66:0/2424611093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f240095f0 con 0x7f0f340ffb00 2023-07-21T22:20:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.288+0000 7f0f337fe700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 0x7f0f34195880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f24005740 tx=0x7f0f24005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.289+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2401d070 con 0x7f0f340ffb00 2023-07-21T22:20:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.289+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3419d950 con 0x7f0f340ffb00 2023-07-21T22:20:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.289+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f24021410 con 0x7f0f340ffb00 2023-07-21T22:20:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.289+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24017650 con 0x7f0f340ffb00 2023-07-21T22:20:59.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.289+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3419dd80 con 0x7f0f340ffb00 2023-07-21T22:20:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.290+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f34107090 con 0x7f0f340ffb00 2023-07-21T22:20:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.292+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0f24021580 con 0x7f0f340ffb00 2023-07-21T22:20:59.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.293+0000 7f0f30ff9700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0f1c0a1220 0x7f0f1c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.293+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0f240ce1e0 con 0x7f0f340ffb00 2023-07-21T22:20:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.294+0000 7f0f32ffd700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0f1c0a1220 0x7f0f1c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.294+0000 7f0f32ffd700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0f1c0a1220 0x7f0f1c0a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009c60 tx=0x7f0f28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:59.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.297+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0f24097520 con 0x7f0f340ffb00 2023-07-21T22:20:59.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.513+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f3402c850 con 0x7f0f340ffb00 2023-07-21T22:20:59.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.515+0000 7f0f30ff9700 1 -- 172.21.15.66:0/2424611093 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7f0f2409ab40 con 0x7f0f340ffb00 2023-07-21T22:20:59.559 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:20:59.560 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:20:59.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0f1c0a1220 msgr2=0x7f0f1c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0f1c0a1220 0x7f0f1c0a36d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009c60 tx=0x7f0f28009260 comp rx=0 tx=0).stop 2023-07-21T22:20:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 msgr2=0x7f0f34195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 0x7f0f34195880 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f24005740 tx=0x7f0f24005770 comp rx=0 tx=0).stop 2023-07-21T22:20:59.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 shutdown_connections 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0f1c0a1220 0x7f0f1c0a36d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f34101680 0x7f0f3419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0f340ffb00 0x7f0f34195880 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 --2- 172.21.15.66:0/2424611093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f34100cf0 0x7f0f34195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.517+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 >> 172.21.15.66:0/2424611093 conn(0x7f0f340fb0e0 msgr2=0x7f0f341041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.518+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 shutdown_connections 2023-07-21T22:20:59.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.518+0000 7f0f39bff700 1 -- 172.21.15.66:0/2424611093 wait complete. 2023-07-21T22:20:59.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:20:59.583 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:20:59.645 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:59 smithi066 ceph-mon[126379]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:59.646 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:59 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/823918011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:59.646 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:20:59 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/474477562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:59.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:59 smithi149 ceph-mon[129268]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:59.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:59 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/823918011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:59.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:20:59 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/474477562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:20:59.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.749+0000 7f73aa7ff700 1 -- 172.21.15.66:0/1538195755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 msgr2=0x7f73a4107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.749+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/1538195755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a4107f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f739c00b0a0 tx=0x7f739c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 -- 172.21.15.66:0/1538195755 shutdown_connections 2023-07-21T22:20:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/1538195755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73a40ffae0 0x7f73a40ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/1538195755 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a4107f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/1538195755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73a4100cd0 0x7f73a4101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 -- 172.21.15.66:0/1538195755 >> 172.21.15.66:0/1538195755 conn(0x7f73a40fb0e0 msgr2=0x7f73a40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:20:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 -- 172.21.15.66:0/1538195755 shutdown_connections 2023-07-21T22:20:59.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.750+0000 7f73aa7ff700 1 -- 172.21.15.66:0/1538195755 wait complete. 2023-07-21T22:20:59.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 Processor -- start 2023-07-21T22:20:59.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 -- start start 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73a40ffae0 0x7f73a41958e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73a4100cd0 0x7f73a4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a419d460 con 0x7f73a4101660 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a419d5a0 con 0x7f73a40ffae0 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73aa7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a419d730 con 0x7f73a4100cd0 2023-07-21T22:20:59.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.751+0000 7f73a8d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48710/0 (socket says 172.21.15.66:48710) 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 -- 172.21.15.66:0/842187618 learned_addr learned my addr 172.21.15.66:0/842187618 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73a40ffae0 0x7f73a41958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a37fe700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73a4100cd0 0x7f73a4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 -- 172.21.15.66:0/842187618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73a4100cd0 msgr2=0x7f73a4195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73a4100cd0 0x7f73a4195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 -- 172.21.15.66:0/842187618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73a40ffae0 msgr2=0x7f73a41958e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73a40ffae0 0x7f73a41958e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 -- 172.21.15.66:0/842187618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c009d00 con 0x7f73a4101660 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a8d9c700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a419ce90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f739c00b780 tx=0x7f739c00bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c00e040 con 0x7f73a4101660 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.752+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f739c004e60 con 0x7f73a4101660 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.753+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a419d9b0 con 0x7f73a4101660 2023-07-21T22:20:59.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.753+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739c01b6b0 con 0x7f73a4101660 2023-07-21T22:20:59.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.753+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a419de80 con 0x7f73a4101660 2023-07-21T22:20:59.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.754+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a4106ff0 con 0x7f73a4101660 2023-07-21T22:20:59.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.756+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f739c01b810 con 0x7f73a4101660 2023-07-21T22:20:59.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.757+0000 7f73a17fa700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f738c0a1260 0x7f738c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:20:59.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.757+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f739c0ce5c0 con 0x7f73a4101660 2023-07-21T22:20:59.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.758+0000 7f73a3fff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f738c0a1260 0x7f738c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:20:59.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.758+0000 7f73a3fff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f738c0a1260 0x7f738c0a3710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f73a4100b30 tx=0x7f7394007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:20:59.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.761+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f739c097870 con 0x7f73a4101660 2023-07-21T22:20:59.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:59 smithi104 ceph-mon[129021]: pgmap v334: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T22:20:59.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:59 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/823918011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:20:59.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:20:59 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/474477562' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:00.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.969+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73a402ca60 con 0x7f73a4101660 2023-07-21T22:21:00.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.970+0000 7f73a17fa700 1 -- 172.21.15.66:0/842187618 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f739c09ae90 con 0x7f73a4101660 2023-07-21T22:21:00.014 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:21:00.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f738c0a1260 msgr2=0x7f738c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f738c0a1260 0x7f738c0a3710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f73a4100b30 tx=0x7f7394007400 comp rx=0 tx=0).stop 2023-07-21T22:21:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 msgr2=0x7f73a419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a419ce90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f739c00b780 tx=0x7f739c00bd00 comp rx=0 tx=0).stop 2023-07-21T22:21:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 shutdown_connections 2023-07-21T22:21:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f738c0a1260 0x7f738c0a3710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73a40ffae0 0x7f73a41958e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f73a4101660 0x7f73a419ce90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 --2- 172.21.15.66:0/842187618 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73a4100cd0 0x7f73a4195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 >> 172.21.15.66:0/842187618 conn(0x7f73a40fb0e0 msgr2=0x7f73a4104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.972+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 shutdown_connections 2023-07-21T22:21:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:20:59.973+0000 7f73aa7ff700 1 -- 172.21.15.66:0/842187618 wait complete. 2023-07-21T22:21:00.035 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:21:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 -- 172.21.15.66:0/2259625699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8100cd0 msgr2=0x7f58f8101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 --2- 172.21.15.66:0/2259625699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8100cd0 0x7f58f8101120 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009990 tx=0x7f58e8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 -- 172.21.15.66:0/2259625699 shutdown_connections 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 --2- 172.21.15.66:0/2259625699 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f80ffae0 0x7f58f80ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 --2- 172.21.15.66:0/2259625699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58f8101660 0x7f58f8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 --2- 172.21.15.66:0/2259625699 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8100cd0 0x7f58f8101120 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.201+0000 7f59003ff700 1 -- 172.21.15.66:0/2259625699 >> 172.21.15.66:0/2259625699 conn(0x7f58f80fb0e0 msgr2=0x7f58f80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.202+0000 7f59003ff700 1 -- 172.21.15.66:0/2259625699 shutdown_connections 2023-07-21T22:21:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.202+0000 7f59003ff700 1 -- 172.21.15.66:0/2259625699 wait complete. 2023-07-21T22:21:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.203+0000 7f59003ff700 1 Processor -- start 2023-07-21T22:21:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.203+0000 7f59003ff700 1 -- start start 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f59003ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f80ffae0 0x7f58f8195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f59003ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58f8100cd0 0x7f58f8195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f59003ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 0x7f58f819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f59003ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f819d390 con 0x7f58f8101660 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f59003ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f819d4d0 con 0x7f58f80ffae0 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f59003ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f819d660 con 0x7f58f8100cd0 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 0x7f58f819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fd99a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58f8100cd0 0x7f58f8195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 0x7f58f819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48732/0 (socket says 172.21.15.66:48732) 2023-07-21T22:21:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 -- 172.21.15.66:0/45985426 learned_addr learned my addr 172.21.15.66:0/45985426 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe19b700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f80ffae0 0x7f58f8195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 -- 172.21.15.66:0/45985426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58f8100cd0 msgr2=0x7f58f8195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58f8100cd0 0x7f58f8195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 -- 172.21.15.66:0/45985426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f80ffae0 msgr2=0x7f58f8195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f80ffae0 0x7f58f8195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.204+0000 7f58fe99c700 1 -- 172.21.15.66:0/45985426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009520 con 0x7f58f8101660 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.205+0000 7f58fe99c700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 0x7f58f819cdc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ef30 tx=0x7f58f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.205+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0018070 con 0x7f58f8101660 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.205+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f000f040 con 0x7f58f8101660 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.205+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e80095f0 con 0x7f58f8101660 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.205+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000cd00 con 0x7f58f8101660 2023-07-21T22:21:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.205+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f819dc60 con 0x7f58f8101660 2023-07-21T22:21:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.206+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f810b2b0 con 0x7f58f8101660 2023-07-21T22:21:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.208+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f58f0003a00 con 0x7f58f8101660 2023-07-21T22:21:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.210+0000 7f58ef7fe700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f58e40a11e0 0x7f58e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.210+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f58f00cc990 con 0x7f58f8101660 2023-07-21T22:21:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.210+0000 7f58fe19b700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f58e40a11e0 0x7f58e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.211+0000 7f58fe19b700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f58e40a11e0 0x7f58e40a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58f8100b30 tx=0x7f58f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.214+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f58f0095c40 con 0x7f58f8101660 2023-07-21T22:21:00.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.424+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58f8106a10 con 0x7f58f8101660 2023-07-21T22:21:00.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.424+0000 7f58ef7fe700 1 -- 172.21.15.66:0/45985426 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58f0099260 con 0x7f58f8101660 2023-07-21T22:21:00.469 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f58e40a11e0 msgr2=0x7f58e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f58e40a11e0 0x7f58e40a3690 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f58f8100b30 tx=0x7f58f4009260 comp rx=0 tx=0).stop 2023-07-21T22:21:00.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 msgr2=0x7f58f819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 0x7f58f819cdc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f58f000ef30 tx=0x7f58f000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 shutdown_connections 2023-07-21T22:21:00.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f58e40a11e0 0x7f58e40a3690 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58f80ffae0 0x7f58f8195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f58f8101660 0x7f58f819cdc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 --2- 172.21.15.66:0/45985426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58f8100cd0 0x7f58f8195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 >> 172.21.15.66:0/45985426 conn(0x7f58f80fb0e0 msgr2=0x7f58f80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:00.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.426+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 shutdown_connections 2023-07-21T22:21:00.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.427+0000 7f59003ff700 1 -- 172.21.15.66:0/45985426 wait complete. 2023-07-21T22:21:00.489 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:21:00.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2424611093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:00.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/842187618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 -- 172.21.15.66:0/1835007281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0101670 msgr2=0x7f99e0107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 --2- 172.21.15.66:0/1835007281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0101670 0x7f99e0107f30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f99d800b0a0 tx=0x7f99d800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 -- 172.21.15.66:0/1835007281 shutdown_connections 2023-07-21T22:21:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 --2- 172.21.15.66:0/1835007281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e00ffaf0 0x7f99e00fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 --2- 172.21.15.66:0/1835007281 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0101670 0x7f99e0107f30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 --2- 172.21.15.66:0/1835007281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99e0100ce0 0x7f99e0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 -- 172.21.15.66:0/1835007281 >> 172.21.15.66:0/1835007281 conn(0x7f99e00fb130 msgr2=0x7f99e00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.654+0000 7f99e7bff700 1 -- 172.21.15.66:0/1835007281 shutdown_connections 2023-07-21T22:21:00.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.655+0000 7f99e7bff700 1 -- 172.21.15.66:0/1835007281 wait complete. 2023-07-21T22:21:00.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 Processor -- start 2023-07-21T22:21:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 -- start start 2023-07-21T22:21:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99e00ffaf0 0x7f99e0195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 0x7f99e0195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e0101670 0x7f99e019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e019d480 con 0x7f99e0100ce0 2023-07-21T22:21:00.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e019d5c0 con 0x7f99e0101670 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e7bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e019d750 con 0x7f99e00ffaf0 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e519a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 0x7f99e0195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e519a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 0x7f99e0195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48740/0 (socket says 172.21.15.66:48740) 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.656+0000 7f99e519a700 1 -- 172.21.15.66:0/454756221 learned_addr learned my addr 172.21.15.66:0/454756221 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e599b700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99e00ffaf0 0x7f99e0195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e619c700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e0101670 0x7f99e019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e519a700 1 -- 172.21.15.66:0/454756221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99e00ffaf0 msgr2=0x7f99e0195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e519a700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99e00ffaf0 0x7f99e0195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e519a700 1 -- 172.21.15.66:0/454756221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e0101670 msgr2=0x7f99e019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e519a700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e0101670 0x7f99e019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e519a700 1 -- 172.21.15.66:0/454756221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d8009d00 con 0x7f99e0100ce0 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e519a700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 0x7f99e0195e40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99d000e9a0 tx=0x7f99d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0009790 con 0x7f99e0100ce0 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e019da30 con 0x7f99e0100ce0 2023-07-21T22:21:00.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99d0010430 con 0x7f99e0100ce0 2023-07-21T22:21:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.657+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0018680 con 0x7f99e0100ce0 2023-07-21T22:21:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.658+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e019df90 con 0x7f99e0100ce0 2023-07-21T22:21:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.659+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e0106fc0 con 0x7f99e0100ce0 2023-07-21T22:21:00.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.661+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f99d00105a0 con 0x7f99e0100ce0 2023-07-21T22:21:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.662+0000 7f99d6ffd700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f99cc0a11e0 0x7f99cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:00.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.662+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f99d0014070 con 0x7f99e0100ce0 2023-07-21T22:21:00.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.662+0000 7f99e599b700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f99cc0a11e0 0x7f99cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:00.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.663+0000 7f99e599b700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f99cc0a11e0 0x7f99cc0a3690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f99e0100b40 tx=0x7f99dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:00.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.666+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f99d0051770 con 0x7f99e0100ce0 2023-07-21T22:21:00.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2424611093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:00.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/842187618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:00.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2424611093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:00.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/842187618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:00.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.875+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f99e002cc50 con 0x7f99e0100ce0 2023-07-21T22:21:00.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.876+0000 7f99d6ffd700 1 -- 172.21.15.66:0/454756221 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99d0096250 con 0x7f99e0100ce0 2023-07-21T22:21:00.972 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:21:00.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f99cc0a11e0 msgr2=0x7f99cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f99cc0a11e0 0x7f99cc0a3690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f99e0100b40 tx=0x7f99dc009260 comp rx=0 tx=0).stop 2023-07-21T22:21:00.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 msgr2=0x7f99e0195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:00.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 0x7f99e0195e40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f99d000e9a0 tx=0x7f99d000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 shutdown_connections 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f99cc0a11e0 0x7f99cc0a3690 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99e0101670 0x7f99e019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f99e00ffaf0 0x7f99e0195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 --2- 172.21.15.66:0/454756221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f99e0100ce0 0x7f99e0195e40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.878+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 >> 172.21.15.66:0/454756221 conn(0x7f99e00fb130 msgr2=0x7f99e0104100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.879+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 shutdown_connections 2023-07-21T22:21:00.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:00.879+0000 7f99e7bff700 1 -- 172.21.15.66:0/454756221 wait complete. 2023-07-21T22:21:00.975 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:21:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 -- 172.21.15.66:0/3397566168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b0100cd0 msgr2=0x7f40b0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 --2- 172.21.15.66:0/3397566168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b0100cd0 0x7f40b0101120 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f40a40098f0 tx=0x7f40a4009c00 comp rx=0 tx=0).stop 2023-07-21T22:21:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 -- 172.21.15.66:0/3397566168 shutdown_connections 2023-07-21T22:21:01.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 --2- 172.21.15.66:0/3397566168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40b00ffae0 0x7f40b00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 --2- 172.21.15.66:0/3397566168 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40b0101660 0x7f40b0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 --2- 172.21.15.66:0/3397566168 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b0100cd0 0x7f40b0101120 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 -- 172.21.15.66:0/3397566168 >> 172.21.15.66:0/3397566168 conn(0x7f40b00fb0e0 msgr2=0x7f40b00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 -- 172.21.15.66:0/3397566168 shutdown_connections 2023-07-21T22:21:01.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.138+0000 7f40b53ff700 1 -- 172.21.15.66:0/3397566168 wait complete. 2023-07-21T22:21:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.139+0000 7f40b53ff700 1 Processor -- start 2023-07-21T22:21:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 -- start start 2023-07-21T22:21:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 0x7f40b0195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40b0100cd0 0x7f40b0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40b0101660 0x7f40b019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019d400 con 0x7f40b0100cd0 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019d540 con 0x7f40b0101660 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40b53ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019d6d0 con 0x7f40b00ffae0 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40aeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 0x7f40b0195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40aeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 0x7f40b0195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35088/0 (socket says 172.21.15.66:35088) 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40ae7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40b0100cd0 0x7f40b0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40aeffd700 1 -- 172.21.15.66:0/1586831176 learned_addr learned my addr 172.21.15.66:0/1586831176 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:01.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.140+0000 7f40af7fe700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40b0101660 0x7f40b019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40aeffd700 1 -- 172.21.15.66:0/1586831176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40b0101660 msgr2=0x7f40b019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40aeffd700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40b0101660 0x7f40b019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40aeffd700 1 -- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40b0100cd0 msgr2=0x7f40b0195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40aeffd700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40b0100cd0 0x7f40b0195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40aeffd700 1 -- 172.21.15.66:0/1586831176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8009d00 con 0x7f40b00ffae0 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40aeffd700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 0x7f40b0195880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e970 tx=0x7f40a000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0009790 con 0x7f40b00ffae0 2023-07-21T22:21:01.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a000c7e0 con 0x7f40b00ffae0 2023-07-21T22:21:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a4009520 con 0x7f40b00ffae0 2023-07-21T22:21:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a0003c70 con 0x7f40b00ffae0 2023-07-21T22:21:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.141+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b019dd80 con 0x7f40b00ffae0 2023-07-21T22:21:01.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.142+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b010b2a0 con 0x7f40b00ffae0 2023-07-21T22:21:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.144+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f40a000cc50 con 0x7f40b00ffae0 2023-07-21T22:21:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.146+0000 7f4097fff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f40980a1260 0x7f40980a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.146+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f40a0014070 con 0x7f40b00ffae0 2023-07-21T22:21:01.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.146+0000 7f40ae7fc700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f40980a1260 0x7f40980a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.147+0000 7f40ae7fc700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f40980a1260 0x7f40980a3710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005e90 tx=0x7f40a4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:01.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.148+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f40a0096810 con 0x7f40b00ffae0 2023-07-21T22:21:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.361+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f40b0106a30 con 0x7f40b00ffae0 2023-07-21T22:21:01.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.361+0000 7f4097fff700 1 -- 172.21.15.66:0/1586831176 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f40a0099e30 con 0x7f40b00ffae0 2023-07-21T22:21:01.407 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:01.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f40980a1260 msgr2=0x7f40980a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f40980a1260 0x7f40980a3710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005e90 tx=0x7f40a4019040 comp rx=0 tx=0).stop 2023-07-21T22:21:01.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 msgr2=0x7f40b0195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 0x7f40b0195880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f40a000e970 tx=0x7f40a000ec80 comp rx=0 tx=0).stop 2023-07-21T22:21:01.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 shutdown_connections 2023-07-21T22:21:01.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f40980a1260 0x7f40980a3710 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f40b0101660 0x7f40b019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f40b00ffae0 0x7f40b0195880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 --2- 172.21.15.66:0/1586831176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f40b0100cd0 0x7f40b0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 >> 172.21.15.66:0/1586831176 conn(0x7f40b00fb0e0 msgr2=0x7f40b00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:01.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 shutdown_connections 2023-07-21T22:21:01.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.364+0000 7f40b53ff700 1 -- 172.21.15.66:0/1586831176 wait complete. 2023-07-21T22:21:01.429 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:21:01.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.590+0000 7f669f1ff700 1 -- 172.21.15.66:0/4172398455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698101680 msgr2=0x7f6698107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.590+0000 7f669f1ff700 1 --2- 172.21.15.66:0/4172398455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698101680 0x7f6698107f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f669000b0a0 tx=0x7f669000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 -- 172.21.15.66:0/4172398455 shutdown_connections 2023-07-21T22:21:01.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 --2- 172.21.15.66:0/4172398455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698101680 0x7f6698107f40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 --2- 172.21.15.66:0/4172398455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f66980fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 --2- 172.21.15.66:0/4172398455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698100cf0 0x7f6698101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 -- 172.21.15.66:0/4172398455 >> 172.21.15.66:0/4172398455 conn(0x7f66980fb0e0 msgr2=0x7f66980fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:01.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 -- 172.21.15.66:0/4172398455 shutdown_connections 2023-07-21T22:21:01.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.591+0000 7f669f1ff700 1 -- 172.21.15.66:0/4172398455 wait complete. 2023-07-21T22:21:01.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 Processor -- start 2023-07-21T22:21:01.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 -- start start 2023-07-21T22:21:01.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f6698195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698100cf0 0x7f6698195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101680 0x7f669819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819d410 con 0x7f66980ffb00 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819d550 con 0x7f6698100cf0 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.592+0000 7f669f1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669819d6e0 con 0x7f6698101680 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f6698195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f6698195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48776/0 (socket says 172.21.15.66:48776) 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 -- 172.21.15.66:0/1775214556 learned_addr learned my addr 172.21.15.66:0/1775214556 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f6697fff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698100cf0 0x7f6698195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669d79c700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101680 0x7f669819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 -- 172.21.15.66:0/1775214556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101680 msgr2=0x7f669819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101680 0x7f669819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 -- 172.21.15.66:0/1775214556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698100cf0 msgr2=0x7f6698195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698100cf0 0x7f6698195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 -- 172.21.15.66:0/1775214556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c009520 con 0x7f66980ffb00 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f669cf9b700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f6698195890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f668c00ea50 tx=0x7f668c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.593+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c00cbd0 con 0x7f66980ffb00 2023-07-21T22:21:01.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.594+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668c00cd30 con 0x7f66980ffb00 2023-07-21T22:21:01.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.594+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6690009d00 con 0x7f66980ffb00 2023-07-21T22:21:01.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.594+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c018680 con 0x7f66980ffb00 2023-07-21T22:21:01.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.594+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669819dd90 con 0x7f66980ffb00 2023-07-21T22:21:01.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.595+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669810b2e0 con 0x7f66980ffb00 2023-07-21T22:21:01.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.597+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f668c010430 con 0x7f66980ffb00 2023-07-21T22:21:01.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.598+0000 7f6695ffb700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f66880a12a0 0x7f66880a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:01.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.598+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f668c014070 con 0x7f66980ffb00 2023-07-21T22:21:01.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.598+0000 7f6697fff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f66880a12a0 0x7f66880a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:01.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.599+0000 7f6697fff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f66880a12a0 0x7f66880a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6684009c60 tx=0x7f6684009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:01.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.602+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f668c0964e0 con 0x7f66980ffb00 2023-07-21T22:21:01.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:01 smithi066 ceph-mon[126379]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:21:01.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/45985426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:01.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/454756221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:01.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:01 smithi149 ceph-mon[129268]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:21:01.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:01 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/45985426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:01.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:01 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/454756221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:01.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:01 smithi104 ceph-mon[129021]: pgmap v335: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T22:21:01.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:01 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/45985426' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:01.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:01 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/454756221' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:01.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.858+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66981068c0 con 0x7f66980ffb00 2023-07-21T22:21:01.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.859+0000 7f6695ffb700 1 -- 172.21.15.66:0/1775214556 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f668c099b00 con 0x7f66980ffb00 2023-07-21T22:21:01.904 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:01.904 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:21:01.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f66880a12a0 msgr2=0x7f66880a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f66880a12a0 0x7f66880a3750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6684009c60 tx=0x7f6684009260 comp rx=0 tx=0).stop 2023-07-21T22:21:01.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 msgr2=0x7f6698195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:01.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f6698195890 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f668c00ea50 tx=0x7f668c00ed60 comp rx=0 tx=0).stop 2023-07-21T22:21:01.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 shutdown_connections 2023-07-21T22:21:01.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f66880a12a0 0x7f66880a3750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6698101680 0x7f669819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66980ffb00 0x7f6698195890 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 --2- 172.21.15.66:0/1775214556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6698100cf0 0x7f6698195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:01.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 >> 172.21.15.66:0/1775214556 conn(0x7f66980fb0e0 msgr2=0x7f66980fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:01.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 shutdown_connections 2023-07-21T22:21:01.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:01.861+0000 7f669f1ff700 1 -- 172.21.15.66:0/1775214556 wait complete. 2023-07-21T22:21:01.908 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:21:01.926 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:21:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 -- 172.21.15.66:0/674524769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc0ffb00 msgr2=0x7f45cc0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 --2- 172.21.15.66:0/674524769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc0ffb00 0x7f45cc0fff10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009990 tx=0x7f45bc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 -- 172.21.15.66:0/674524769 shutdown_connections 2023-07-21T22:21:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 --2- 172.21.15.66:0/674524769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc0ffb00 0x7f45cc0fff10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 --2- 172.21.15.66:0/674524769 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc101680 0x7f45cc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 --2- 172.21.15.66:0/674524769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc100cf0 0x7f45cc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 -- 172.21.15.66:0/674524769 >> 172.21.15.66:0/674524769 conn(0x7f45cc0fb0e0 msgr2=0x7f45cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 -- 172.21.15.66:0/674524769 shutdown_connections 2023-07-21T22:21:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.088+0000 7f45d1bff700 1 -- 172.21.15.66:0/674524769 wait complete. 2023-07-21T22:21:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 Processor -- start 2023-07-21T22:21:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 -- start start 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 0x7f45cc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 0x7f45cc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc101680 0x7f45cc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19f580 con 0x7f45cc0ffb00 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19f6c0 con 0x7f45cc100cf0 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.089+0000 7f45d1bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19f850 con 0x7f45cc101680 2023-07-21T22:21:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 0x7f45cc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 0x7f45cc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc101680 0x7f45cc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 0x7f45cc197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:59920/0 (socket says 172.21.15.66:59920) 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 -- 172.21.15.66:0/4026839850 learned_addr learned my addr 172.21.15.66:0/4026839850 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45cb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 0x7f45cc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42240/0 (socket says 172.21.15.66:42240) 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 -- 172.21.15.66:0/4026839850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc101680 msgr2=0x7f45cc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc101680 0x7f45cc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 -- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 msgr2=0x7f45cc197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 0x7f45cc197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 -- 172.21.15.66:0/4026839850 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0095f0 con 0x7f45cc100cf0 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45cbfff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc101680 0x7f45cc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45cb7fe700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 0x7f45cc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.090+0000 7f45caffd700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 0x7f45cc197f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9a0 tx=0x7f45c000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.091+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0009790 con 0x7f45cc100cf0 2023-07-21T22:21:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.091+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c0010430 con 0x7f45cc100cf0 2023-07-21T22:21:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.091+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc19fb30 con 0x7f45cc100cf0 2023-07-21T22:21:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.091+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0018680 con 0x7f45cc100cf0 2023-07-21T22:21:02.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.091+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc1a0090 con 0x7f45cc100cf0 2023-07-21T22:21:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.092+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45cc192190 con 0x7f45cc100cf0 2023-07-21T22:21:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.094+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f45c00105a0 con 0x7f45cc100cf0 2023-07-21T22:21:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.096+0000 7f45c8ff9700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f45b40a12a0 0x7f45b40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.096+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f45c0014070 con 0x7f45cc100cf0 2023-07-21T22:21:02.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.096+0000 7f45cb7fe700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f45b40a12a0 0x7f45b40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.097+0000 7f45cb7fe700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f45b40a12a0 0x7f45b40a3750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006010 tx=0x7f45bc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.098+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f45c00963f0 con 0x7f45cc100cf0 2023-07-21T22:21:02.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.310+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45cc02caa0 con 0x7f45cc100cf0 2023-07-21T22:21:02.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.312+0000 7f45c8ff9700 1 -- 172.21.15.66:0/4026839850 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7f45c0026020 con 0x7f45cc100cf0 2023-07-21T22:21:02.358 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:02.358 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:21:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f45b40a12a0 msgr2=0x7f45b40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f45b40a12a0 0x7f45b40a3750 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006010 tx=0x7f45bc005960 comp rx=0 tx=0).stop 2023-07-21T22:21:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 msgr2=0x7f45cc197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 0x7f45cc197f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45c000e9a0 tx=0x7f45c000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 shutdown_connections 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f45b40a12a0 0x7f45b40a3750 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45cc101680 0x7f45cc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f45cc0ffb00 0x7f45cc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 --2- 172.21.15.66:0/4026839850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45cc100cf0 0x7f45cc197f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 >> 172.21.15.66:0/4026839850 conn(0x7f45cc0fb0e0 msgr2=0x7f45cc1065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 shutdown_connections 2023-07-21T22:21:02.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.315+0000 7f45d1bff700 1 -- 172.21.15.66:0/4026839850 wait complete. 2023-07-21T22:21:02.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T22:19:06.899289+0000', 'modified': '2023-07-21T22:19:34.299634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24530, 'mds_2': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6835/1642617037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 1642617037}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 1642617037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/1273148910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 1273148910}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 1273148910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.149:6837/1037928792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 1037928792}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 1037928792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.149:6839/2751755690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 2751755690}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 2751755690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24530': {'gid': 24530, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.104:6837/1237242254', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 1237242254}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 1237242254}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.104:6839/1299576617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 1299576617}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 1299576617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T22:21:02.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-21T22:21:02.381 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T22:21:02.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 -- 172.21.15.66:0/3382062282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 msgr2=0x7f4764101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 --2- 172.21.15.66:0/3382062282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 0x7f4764101120 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4754009990 tx=0x7f4754009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 -- 172.21.15.66:0/3382062282 shutdown_connections 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 --2- 172.21.15.66:0/3382062282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 --2- 172.21.15.66:0/3382062282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47640ffae0 0x7f47640ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 --2- 172.21.15.66:0/3382062282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 0x7f4764101120 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 -- 172.21.15.66:0/3382062282 >> 172.21.15.66:0/3382062282 conn(0x7f47640fb0e0 msgr2=0x7f47640fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 -- 172.21.15.66:0/3382062282 shutdown_connections 2023-07-21T22:21:02.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.544+0000 7f476c3ff700 1 -- 172.21.15.66:0/3382062282 wait complete. 2023-07-21T22:21:02.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.545+0000 7f476c3ff700 1 Processor -- start 2023-07-21T22:21:02.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.545+0000 7f476c3ff700 1 -- start start 2023-07-21T22:21:02.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.545+0000 7f476c3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47640ffae0 0x7f47640742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476c3ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 0x7f4764072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476c3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476c3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640735e0 con 0x7f4764101660 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476c3ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764073750 con 0x7f4764100cd0 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476c3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476419f8d0 con 0x7f47640ffae0 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42250/0 (socket says 172.21.15.66:42250) 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 -- 172.21.15.66:0/2019063046 learned_addr learned my addr 172.21.15.66:0/2019063046 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a19b700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47640ffae0 0x7f47640742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476999a700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 0x7f4764072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 -- 172.21.15.66:0/2019063046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47640ffae0 msgr2=0x7f47640742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47640ffae0 0x7f47640742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 -- 172.21.15.66:0/2019063046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 msgr2=0x7f4764072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 0x7f4764072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.546+0000 7f476a99c700 1 -- 172.21.15.66:0/2019063046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540095f0 con 0x7f4764101660 2023-07-21T22:21:02.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.547+0000 7f476a99c700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764072e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f475c00ef30 tx=0x7f475c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.547+0000 7f475b7fe700 1 -- 172.21.15.66:0/2019063046 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c018070 con 0x7f4764101660 2023-07-21T22:21:02.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.547+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f476419fb50 con 0x7f4764101660 2023-07-21T22:21:02.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.547+0000 7f475b7fe700 1 -- 172.21.15.66:0/2019063046 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475c00f040 con 0x7f4764101660 2023-07-21T22:21:02.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.547+0000 7f475b7fe700 1 -- 172.21.15.66:0/2019063046 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c00cd00 con 0x7f4764101660 2023-07-21T22:21:02.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.547+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47641a00b0 con 0x7f4764101660 2023-07-21T22:21:02.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.550+0000 7f475b7fe700 1 -- 172.21.15.66:0/2019063046 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f475c003a00 con 0x7f4764101660 2023-07-21T22:21:02.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.551+0000 7f477fd9fb80 1 Processor -- start 2023-07-21T22:21:02.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.551+0000 7f477fd9fb80 1 -- start start 2023-07-21T22:21:02.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f475b7fe700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f47500a11e0 0x7f47500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623019bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622f1dc60 con 0x5626230197e0 2023-07-21T22:21:02.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622f709f0 con 0x56262301a150 2023-07-21T22:21:02.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622f69e80 con 0x5626230189c0 2023-07-21T22:21:02.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f475b7fe700 1 -- 172.21.15.66:0/2019063046 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f475c0cc950 con 0x7f4764101660 2023-07-21T22:21:02.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f476a19b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623019bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f476999a700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f47500a11e0 0x7f47500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f476a19b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623019bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42252/0 (socket says 172.21.15.66:42252) 2023-07-21T22:21:02.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f476a19b700 1 -- 172.21.15.66:0/740977871 learned_addr learned my addr 172.21.15.66:0/740977871 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f476a99c700 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.552+0000 7f476999a700 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476a19b700 1 -- 172.21.15.66:0/740977871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 msgr2=0x562623018e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476a19b700 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476a19b700 1 -- 172.21.15.66:0/740977871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 msgr2=0x5626230183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476a19b700 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230183e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476a19b700 1 -- 172.21.15.66:0/740977871 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562622d97d90 con 0x5626230197e0 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476999a700 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476a19b700 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623019bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4760009d30 tx=0x7f4760011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea8ecefee150721 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f476999a700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f47500a11e0 0x7f47500a3690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f475400b2e0 tx=0x7f475401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f4758bfe700 1 -- 172.21.15.66:0/740977871 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760004220 con 0x5626230197e0 2023-07-21T22:21:02.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f4758bfe700 1 -- 172.21.15.66:0/740977871 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4760004380 con 0x5626230197e0 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.553+0000 7f4758bfe700 1 -- 172.21.15.66:0/740977871 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47600198d0 con 0x5626230197e0 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 -- 172.21.15.66:0/740977871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 msgr2=0x562623019bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623019bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4760009d30 tx=0x7f4760011720 comp rx=0 tx=0).stop 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 -- 172.21.15.66:0/740977871 shutdown_connections 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623019bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/740977871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230183e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 -- 172.21.15.66:0/740977871 >> 172.21.15.66:0/740977871 conn(0x56262300dea0 msgr2=0x562623013310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 -- 172.21.15.66:0/740977871 shutdown_connections 2023-07-21T22:21:02.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.554+0000 7f477fd9fb80 1 -- 172.21.15.66:0/740977871 wait complete. 2023-07-21T22:21:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 -- start start 2023-07-21T22:21:02.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 Processor -- start 2023-07-21T22:21:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623022050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230197e0 0x562623022590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56262301a150 0x5626230295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622f09a70 con 0x56262301a150 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622f09770 con 0x5626230189c0 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622ded8d0 con 0x5626230197e0 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.555+0000 7f477fd9fb80 10 client.0 _ll_register_callbacks cb 0x562622f07710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623022050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623022050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:59950/0 (socket says 172.21.15.66:59950) 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 -- 172.21.15.66:0/3103401907 learned_addr learned my addr 172.21.15.66:0/3103401907 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a99c700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56262301a150 0x5626230295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476999a700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230197e0 0x562623022590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 -- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230197e0 msgr2=0x562623022590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230197e0 0x562623022590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 -- 172.21.15.66:0/3103401907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56262301a150 msgr2=0x5626230295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56262301a150 0x5626230295d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 -- 172.21.15.66:0/3103401907 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4760009960 con 0x5626230189c0 2023-07-21T22:21:02.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f476a19b700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623022050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4760006010 tx=0x7f4760005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d990047995eee746 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.556+0000 7f474dffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:21:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.557+0000 7f474dffb700 1 -- 172.21.15.66:0/3103401907 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760005040 con 0x5626230189c0 2023-07-21T22:21:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.557+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562622ec34a0 con 0x5626230189c0 2023-07-21T22:21:02.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.557+0000 7f474dffb700 1 -- 172.21.15.66:0/3103401907 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f47600051a0 con 0x5626230189c0 2023-07-21T22:21:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.557+0000 7f474dffb700 1 -- 172.21.15.66:0/3103401907 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4760005360 con 0x5626230189c0 2023-07-21T22:21:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.557+0000 7f477fd9fb80 10 client.25310 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:02.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.557+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562622d80c10 con 0x5626230189c0 2023-07-21T22:21:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f474dffb700 1 -- 172.21.15.66:0/3103401907 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562622ec34a0 con 0x5626230189c0 2023-07-21T22:21:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f474dffb700 1 -- 172.21.15.66:0/3103401907 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4760021040 con 0x5626230189c0 2023-07-21T22:21:02.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f477fd9fb80 10 client.25310 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f477fd9fb80 10 client.25310 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:21:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f477fd9fb80 20 client.25310 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:02.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56262302d270 0x56262302c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f477fd9fb80 4 client.25310 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:02.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562622f62e60 con 0x56262302d270 2023-07-21T22:21:02.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.558+0000 7f476999a700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56262302d270 0x56262302c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.559+0000 7f476999a700 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56262302d270 0x56262302c680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=257c740a855b9449 server_cookie=2c963bd4c40bb9f5 in_seq=0 out_seq=0 2023-07-21T22:21:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.559+0000 7f474dffb700 10 client.25310 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f474dffb700 1 -- 172.21.15.66:0/3103401907 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562622f62e60 con 0x56262302d270 2023-07-21T22:21:02.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f474dffb700 10 client.25310 handle_command_reply: tid=0 2023-07-21T22:21:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 client.25310 shutdown 2023-07-21T22:21:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 msgr2=0x562623022050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623022050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4760006010 tx=0x7f4760005d60 comp rx=0 tx=0).stop 2023-07-21T22:21:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 shutdown_connections 2023-07-21T22:21:02.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56262302d270 0x56262302c680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623022050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f474dffb700 0 client.25310 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:02.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56262301a150 0x5626230295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/3103401907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230197e0 0x562623022590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 >> 172.21.15.66:0/3103401907 conn(0x56262300dea0 msgr2=0x5626230129b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 shutdown_connections 2023-07-21T22:21:02.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 1 -- 172.21.15.66:0/3103401907 wait complete. 2023-07-21T22:21:02.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.560+0000 7f477fd9fb80 20 client.25310 trim_cache size 0 max 16384 2023-07-21T22:21:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.578+0000 7f477fd9fb80 1 Processor -- start 2023-07-21T22:21:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.578+0000 7f477fd9fb80 1 -- start start 2023-07-21T22:21:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623029560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622ec34d0 con 0x5626230197e0 2023-07-21T22:21:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622f134a0 con 0x56262301a150 2023-07-21T22:21:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562622df9de0 con 0x5626230189c0 2023-07-21T22:21:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623029560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623029560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42274/0 (socket says 172.21.15.66:42274) 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 -- 172.21.15.66:0/608515960 learned_addr learned my addr 172.21.15.66:0/608515960 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476a99c700 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476a19b700 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 -- 172.21.15.66:0/608515960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 msgr2=0x562623018d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 -- 172.21.15.66:0/608515960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 msgr2=0x5626230129b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230129b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476999a700 1 -- 172.21.15.66:0/608515960 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562622dbe200 con 0x5626230197e0 2023-07-21T22:21:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.579+0000 7f476a19b700 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.580+0000 7f476999a700 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623029560 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f47540098e0 tx=0x7f4754005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26d20f16b6b26d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.580+0000 7f47593ff700 1 -- 172.21.15.66:0/608515960 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475402d040 con 0x5626230197e0 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.580+0000 7f47593ff700 1 -- 172.21.15.66:0/608515960 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f475402e040 con 0x5626230197e0 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.580+0000 7f47593ff700 1 -- 172.21.15.66:0/608515960 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400f6c0 con 0x5626230197e0 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.580+0000 7f477fd9fb80 1 -- 172.21.15.66:0/608515960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 msgr2=0x562623029560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.580+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623029560 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f47540098e0 tx=0x7f4754005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 -- 172.21.15.66:0/608515960 shutdown_connections 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56262301a150 0x5626230129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626230189c0 0x562623018d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/608515960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623029560 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 -- 172.21.15.66:0/608515960 >> 172.21.15.66:0/608515960 conn(0x56262300dea0 msgr2=0x562623036000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 -- 172.21.15.66:0/608515960 shutdown_connections 2023-07-21T22:21:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 -- 172.21.15.66:0/608515960 wait complete. 2023-07-21T22:21:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 -- start start 2023-07-21T22:21:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.581+0000 7f477fd9fb80 1 Processor -- start 2023-07-21T22:21:02.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623024760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623024d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 0x562623081130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626230253d0 con 0x5626230197e0 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562623081670 con 0x5626230189c0 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626230817b0 con 0x56262301a150 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f477fd9fb80 10 client.0 _ll_register_callbacks cb 0x7f475c009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:02.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476999a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623024d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 0x562623081130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 0x562623081130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43236/0 (socket says 172.21.15.66:43236) 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 -- 172.21.15.66:0/2438486870 learned_addr learned my addr 172.21.15.66:0/2438486870 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a19b700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623024760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 -- 172.21.15.66:0/2438486870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 msgr2=0x562623024760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623024760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 -- 172.21.15.66:0/2438486870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 msgr2=0x562623024d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623024d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476a99c700 1 -- 172.21.15.66:0/2438486870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4754020040 con 0x56262301a150 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.582+0000 7f476999a700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623024d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f476a99c700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 0x562623081130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f475c0995c0 tx=0x7f475c0954e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d53bbf533630324 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f474dffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f474dffb700 1 -- 172.21.15.66:0/2438486870 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c0998c0 con 0x56262301a150 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4760004380 con 0x56262301a150 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f474dffb700 1 -- 172.21.15.66:0/2438486870 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f475c09d8f0 con 0x56262301a150 2023-07-21T22:21:02.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f474dffb700 1 -- 172.21.15.66:0/2438486870 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f475c0a1c80 con 0x56262301a150 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f477fd9fb80 10 client.25297 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562622d96260 con 0x56262301a150 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f474dffb700 1 -- 172.21.15.66:0/2438486870 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4760004380 con 0x56262301a150 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f474dffb700 1 -- 172.21.15.66:0/2438486870 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f475c0a17f0 con 0x56262301a150 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.583+0000 7f477fd9fb80 10 client.25297 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f477fd9fb80 10 client.25297 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T22:21:02.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f477fd9fb80 20 client.25297 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5626230863f0 0x562623082d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f477fd9fb80 4 client.25297 mds_command: new command op to 24520 tid=0[{"prefix": "status"}] 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562622f25000 con 0x5626230863f0 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f476a19b700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5626230863f0 0x562623082d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f476a19b700 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5626230863f0 0x562623082d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e777acdd26240b server_cookie=b2db51d36e47a64a in_seq=0 out_seq=0 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.584+0000 7f474dffb700 10 client.25297 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f474dffb700 1 -- 172.21.15.66:0/2438486870 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562622f25000 con 0x5626230863f0 2023-07-21T22:21:02.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f474dffb700 10 client.25297 handle_command_reply: tid=0 2023-07-21T22:21:02.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 client.25297 shutdown 2023-07-21T22:21:02.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 msgr2=0x562623081130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 0x562623081130 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f475c0995c0 tx=0x7f475c0954e0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 shutdown_connections 2023-07-21T22:21:02.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5626230863f0 0x562623082d40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626230189c0 0x562623024760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56262301a150 0x562623081130 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f474dffb700 0 client.25297 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 --2- 172.21.15.66:0/2438486870 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626230197e0 0x562623024d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 >> 172.21.15.66:0/2438486870 conn(0x56262300dea0 msgr2=0x562623018d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 shutdown_connections 2023-07-21T22:21:02.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 1 -- 172.21.15.66:0/2438486870 wait complete. 2023-07-21T22:21:02.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.585+0000 7f477fd9fb80 20 client.25297 trim_cache size 0 max 16384 2023-07-21T22:21:02.644 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:02.644 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:21:02.645 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 2, 2023-07-21T22:21:02.645 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24520, 2023-07-21T22:21:02.645 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:21:02.645 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:21:02.645 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:21:02.646 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 89.294355383999999, 2023-07-21T22:21:02.646 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 23, 2023-07-21T22:21:02.646 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 93, 2023-07-21T22:21:02.646 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:21:02.646 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 130.12616967100001 2023-07-21T22:21:02.647 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:02.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f47500a11e0 msgr2=0x7f47500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f47500a11e0 0x7f47500a3690 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f475400b2e0 tx=0x7f475401a040 comp rx=0 tx=0).stop 2023-07-21T22:21:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 msgr2=0x7f4764072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764072e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f475c00ef30 tx=0x7f475c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 shutdown_connections 2023-07-21T22:21:02.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f47500a11e0 0x7f47500a3690 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4764101660 0x7f4764072e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47640ffae0 0x7f47640742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 --2- 172.21.15.66:0/2019063046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4764100cd0 0x7f4764072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 >> 172.21.15.66:0/2019063046 conn(0x7f47640fb0e0 msgr2=0x7f47640fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 shutdown_connections 2023-07-21T22:21:02.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.586+0000 7f476c3ff700 1 -- 172.21.15.66:0/2019063046 wait complete. 2023-07-21T22:21:02.650 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1586831176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:02.650 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1775214556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:02.650 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4026839850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:02.651 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-21T22:21:02.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:02 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1586831176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:02.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:02 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1775214556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:02.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:02 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4026839850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:02.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 -- 172.21.15.66:0/3288275939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 msgr2=0x7f0fbc0fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/3288275939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc0fff00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009990 tx=0x7f0fac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 -- 172.21.15.66:0/3288275939 shutdown_connections 2023-07-21T22:21:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/3288275939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc101670 0x7f0fbc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/3288275939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc0fff00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/3288275939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc100ce0 0x7f0fbc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 -- 172.21.15.66:0/3288275939 >> 172.21.15.66:0/3288275939 conn(0x7f0fbc0fb130 msgr2=0x7f0fbc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 -- 172.21.15.66:0/3288275939 shutdown_connections 2023-07-21T22:21:02.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.814+0000 7f0fc0fff700 1 -- 172.21.15.66:0/3288275939 wait complete. 2023-07-21T22:21:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.815+0000 7f0fc0fff700 1 Processor -- start 2023-07-21T22:21:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.815+0000 7f0fc0fff700 1 -- start start 2023-07-21T22:21:02.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fc0fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fc0fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc100ce0 0x7f0fbc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fc0fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc101670 0x7f0fbc19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fc0fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc19f610 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fc0fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc19f750 con 0x7f0fbc100ce0 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fc0fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc19f8e0 con 0x7f0fbc101670 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42302/0 (socket says 172.21.15.66:42302) 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 -- 172.21.15.66:0/568192544 learned_addr learned my addr 172.21.15.66:0/568192544 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc101670 0x7f0fbc19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc100ce0 0x7f0fbc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 -- 172.21.15.66:0/568192544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc101670 msgr2=0x7f0fbc19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc101670 0x7f0fbc19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 -- 172.21.15.66:0/568192544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc100ce0 msgr2=0x7f0fbc197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc100ce0 0x7f0fbc197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.816+0000 7f0fba59c700 1 -- 172.21.15.66:0/568192544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0095f0 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.817+0000 7f0fba59c700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc197a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005740 tx=0x7f0fac009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.817+0000 7f0fb37fe700 1 -- 172.21.15.66:0/568192544 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac01d070 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.817+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc19fb60 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.817+0000 7f0fb37fe700 1 -- 172.21.15.66:0/568192544 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fac021410 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.817+0000 7f0fb37fe700 1 -- 172.21.15.66:0/568192544 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac017650 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.817+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc19ffe0 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.822+0000 7f0fb37fe700 1 -- 172.21.15.66:0/568192544 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0fac021810 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 Processor -- start 2023-07-21T22:21:02.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 -- start start 2023-07-21T22:21:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ef7e0 0x55fc437efbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 0x55fc437ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ee9c0 0x55fc437eee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc436f3c60 con 0x55fc437f0150 2023-07-21T22:21:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc437469f0 con 0x55fc437ee9c0 2023-07-21T22:21:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fd483db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4373fe80 con 0x55fc437ef7e0 2023-07-21T22:21:02.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.823+0000 7f0fb37fe700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0fa80a1260 0x7f0fa80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 0x55fc437ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 0x55fc437ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42314/0 (socket says 172.21.15.66:42314) 2023-07-21T22:21:02.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 -- 172.21.15.66:0/4013897890 learned_addr learned my addr 172.21.15.66:0/4013897890 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ef7e0 0x55fc437efbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ee9c0 0x55fc437eee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fb37fe700 1 -- 172.21.15.66:0/568192544 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0fac0ce2e0 con 0x7f0fbc0ffaf0 2023-07-21T22:21:02.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0fa80a1260 0x7f0fa80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 -- 172.21.15.66:0/4013897890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ef7e0 msgr2=0x55fc437efbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ef7e0 0x55fc437efbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 -- 172.21.15.66:0/4013897890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ee9c0 msgr2=0x55fc437eee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ee9c0 0x55fc437eee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 -- 172.21.15.66:0/4013897890 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4356dd90 con 0x55fc437f0150 2023-07-21T22:21:02.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.824+0000 7f0fba59c700 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 0x55fc437ee3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0fac013ff0 tx=0x7f0fac009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489773ed7c1b5c36 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0fa80a1260 0x7f0fa80a3710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55fc437ee370 tx=0x7f0fb400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fb0ff9700 1 -- 172.21.15.66:0/4013897890 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac09f280 con 0x55fc437f0150 2023-07-21T22:21:02.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fb0ff9700 1 -- 172.21.15.66:0/4013897890 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fac0a3610 con 0x55fc437f0150 2023-07-21T22:21:02.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fb0ff9700 1 -- 172.21.15.66:0/4013897890 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac08a2e0 con 0x55fc437f0150 2023-07-21T22:21:02.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fd483db80 1 -- 172.21.15.66:0/4013897890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 msgr2=0x55fc437ee3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fd483db80 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 0x55fc437ee3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0fac013ff0 tx=0x7f0fac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fd483db80 1 -- 172.21.15.66:0/4013897890 shutdown_connections 2023-07-21T22:21:02.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fd483db80 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ee9c0 0x55fc437eee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fd483db80 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ef7e0 0x55fc437efbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.825+0000 7f0fd483db80 1 --2- 172.21.15.66:0/4013897890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437f0150 0x55fc437ee3e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.826+0000 7f0fd483db80 1 -- 172.21.15.66:0/4013897890 >> 172.21.15.66:0/4013897890 conn(0x55fc437e3ea0 msgr2=0x55fc437e9310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.826+0000 7f0fd483db80 1 -- 172.21.15.66:0/4013897890 shutdown_connections 2023-07-21T22:21:02.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.826+0000 7f0fd483db80 1 -- 172.21.15.66:0/4013897890 wait complete. 2023-07-21T22:21:02.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.826+0000 7f0fd483db80 1 -- start start 2023-07-21T22:21:02.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.826+0000 7f0fd483db80 1 Processor -- start 2023-07-21T22:21:02.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:02 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1586831176' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:02.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:02 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1775214556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:02.882 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:02 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4026839850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:02.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc436dfa70 con 0x55fc437ee9c0 2023-07-21T22:21:02.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc436df770 con 0x55fc437ef7e0 2023-07-21T22:21:02.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc435c38d0 con 0x55fc437f0150 2023-07-21T22:21:02.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fd483db80 10 client.0 _ll_register_callbacks cb 0x55fc436dd710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:02.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437f7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42330/0 (socket says 172.21.15.66:42330) 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 -- 172.21.15.66:0/353471671 learned_addr learned my addr 172.21.15.66:0/353471671 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 -- 172.21.15.66:0/353471671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 msgr2=0x55fc437ff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437ff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 -- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 msgr2=0x55fc437f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.827+0000 7f0fba59c700 1 -- 172.21.15.66:0/353471671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc19ffe0 con 0x55fc437ee9c0 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fba59c700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437f7d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0a30e0 tx=0x7f0fac005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688e8aa22a26ac52 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/353471671 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0a3380 con 0x55fc437ee9c0 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/353471671 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0fac099390 con 0x55fc437ee9c0 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc436994a0 con 0x55fc437ee9c0 2023-07-21T22:21:02.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/353471671 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fac09f280 con 0x55fc437ee9c0 2023-07-21T22:21:02.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fd483db80 10 client.15267 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:02.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fc43556c10 con 0x55fc437ee9c0 2023-07-21T22:21:02.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.828+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/353471671 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc436994a0 con 0x55fc437ee9c0 2023-07-21T22:21:02.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/353471671 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fac09a1b0 con 0x55fc437ee9c0 2023-07-21T22:21:02.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fd483db80 10 client.15267 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:02.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fd483db80 10 client.15267 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T22:21:02.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fd483db80 20 client.15267 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:02.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fd483db80 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc43802a20 0x55fc438021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fd483db80 4 client.15267 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:02.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc43738e60 con 0x55fc43802a20 2023-07-21T22:21:02.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.829+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc43802a20 0x55fc438021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.830+0000 7f0fa1ffb700 10 client.15267 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.830+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc43802a20 0x55fc438021e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43530aa4f474ce5a server_cookie=2bf7382b3734edd3 in_seq=0 out_seq=0 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.830+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/353471671 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc43738e60 con 0x55fc43802a20 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.830+0000 7f0fa1ffb700 10 client.15267 handle_command_reply: tid=0 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.830+0000 7f0fd483db80 1 client.15267 shutdown 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 msgr2=0x55fc437f7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437f7d30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0a30e0 tx=0x7f0fac005c70 comp rx=0 tx=0).stop 2023-07-21T22:21:02.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 shutdown_connections 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437f7d30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc43802a20 0x55fc438021e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 --2- 172.21.15.66:0/353471671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 >> 172.21.15.66:0/353471671 conn(0x55fc437e3ea0 msgr2=0x55fc437e89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fa1ffb700 0 client.15267 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:02.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 shutdown_connections 2023-07-21T22:21:02.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 1 -- 172.21.15.66:0/353471671 wait complete. 2023-07-21T22:21:02.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.831+0000 7f0fd483db80 20 client.15267 trim_cache size 0 max 16384 2023-07-21T22:21:02.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 Processor -- start 2023-07-21T22:21:02.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 -- start start 2023-07-21T22:21:02.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ee9c0 0x55fc437e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 0x55fc437ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437f0150 0x55fc4380b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc43803d00 con 0x55fc437ef7e0 2023-07-21T22:21:02.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc43738e90 con 0x55fc437f0150 2023-07-21T22:21:02.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fd483db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc436e9470 con 0x55fc437ee9c0 2023-07-21T22:21:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fb3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 0x55fc437ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fb3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 0x55fc437ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42342/0 (socket says 172.21.15.66:42342) 2023-07-21T22:21:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fb3fff700 1 -- 172.21.15.66:0/733496964 learned_addr learned my addr 172.21.15.66:0/733496964 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437f0150 0x55fc4380b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.850+0000 7f0fba59c700 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ee9c0 0x55fc437e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb3fff700 1 -- 172.21.15.66:0/733496964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ee9c0 msgr2=0x55fc437e92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ee9c0 0x55fc437e92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb3fff700 1 -- 172.21.15.66:0/733496964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437f0150 msgr2=0x55fc4380b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437f0150 0x55fc4380b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb3fff700 1 -- 172.21.15.66:0/733496964 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc438024b0 con 0x55fc437ef7e0 2023-07-21T22:21:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 0x55fc437ff270 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4004790 tx=0x7f0fa4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629a4dc07665a888 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb17fa700 1 -- 172.21.15.66:0/733496964 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4017040 con 0x55fc437ef7e0 2023-07-21T22:21:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb17fa700 1 -- 172.21.15.66:0/733496964 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa4018040 con 0x55fc437ef7e0 2023-07-21T22:21:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.851+0000 7f0fb17fa700 1 -- 172.21.15.66:0/733496964 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4003cb0 con 0x55fc437ef7e0 2023-07-21T22:21:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 -- 172.21.15.66:0/733496964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 msgr2=0x55fc437ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 0x55fc437ff270 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4004790 tx=0x7f0fa4009260 comp rx=0 tx=0).stop 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 -- 172.21.15.66:0/733496964 shutdown_connections 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437f0150 0x55fc4380b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437ee9c0 0x55fc437e92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 --2- 172.21.15.66:0/733496964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ef7e0 0x55fc437ff270 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 -- 172.21.15.66:0/733496964 >> 172.21.15.66:0/733496964 conn(0x55fc437e3ea0 msgr2=0x55fc437ea010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 -- 172.21.15.66:0/733496964 shutdown_connections 2023-07-21T22:21:02.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 -- 172.21.15.66:0/733496964 wait complete. 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.852+0000 7f0fd483db80 1 -- start start 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 Processor -- start 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc437fe060 con 0x55fc437ee9c0 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc437fe1d0 con 0x55fc437ef7e0 2023-07-21T22:21:02.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc437fe390 con 0x55fc437f0150 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fd483db80 10 client.0 _ll_register_callbacks cb 0x7f0fa4009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437fa5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42354/0 (socket says 172.21.15.66:42354) 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fba59c700 1 -- 172.21.15.66:0/73598730 learned_addr learned my addr 172.21.15.66:0/73598730 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.853+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fba59c700 1 -- 172.21.15.66:0/73598730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 msgr2=0x55fc437fdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fba59c700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437fdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fba59c700 1 -- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 msgr2=0x55fc437fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fba59c700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437fab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fba59c700 1 -- 172.21.15.66:0/73598730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc43594200 con 0x55fc437ee9c0 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fbad9d700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437fdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fba59c700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437fa5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0fac09a540 tx=0x7f0fac009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b003e236df1368 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fa1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:02.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/73598730 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac09abb0 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/73598730 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0fac021580 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fac0a3610 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.854+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/73598730 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fac09f280 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 10 client.15273 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fc4356c260 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/73598730 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fac0a3610 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/73598730 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fac097040 con 0x55fc437ee9c0 2023-07-21T22:21:02.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 10 client.15273 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 20 client.15273 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc4385bb30 0x55fc438588b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 4 client.15273 mds_command: new command op to 24530 tid=0[{"prefix": "status"}] 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fc436fb000 con 0x55fc4385bb30 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.855+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc4385bb30 0x55fc438588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fb3fff700 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc4385bb30 0x55fc438588b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2aedd71e1dd53801 server_cookie=a6f8ae39050c3ff0 in_seq=0 out_seq=0 2023-07-21T22:21:02.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fa1ffb700 10 client.15273 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:02.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fa1ffb700 1 -- 172.21.15.66:0/73598730 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fc436fb000 con 0x55fc4385bb30 2023-07-21T22:21:02.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fa1ffb700 10 client.15273 handle_command_reply: tid=0 2023-07-21T22:21:02.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 client.15273 shutdown 2023-07-21T22:21:02.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 msgr2=0x55fc437fa5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437fa5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0fac09a540 tx=0x7f0fac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 shutdown_connections 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fc437f0150 0x55fc437fdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fc437ee9c0 0x55fc437fa5c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55fc4385bb30 0x55fc438588b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 --2- 172.21.15.66:0/73598730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fc437ef7e0 0x55fc437fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fa1ffb700 0 client.15273 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 >> 172.21.15.66:0/73598730 conn(0x55fc437e3ea0 msgr2=0x55fc4380b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.856+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 shutdown_connections 2023-07-21T22:21:02.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fd483db80 1 -- 172.21.15.66:0/73598730 wait complete. 2023-07-21T22:21:02.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fd483db80 20 client.15273 trim_cache size 0 max 16384 2023-07-21T22:21:02.911 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:02.911 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:21:02.911 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 1, 2023-07-21T22:21:02.911 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24530, 2023-07-21T22:21:02.911 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 90.575554925000006, 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 23, 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 93, 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:21:02.912 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 132.74722050400001 2023-07-21T22:21:02.913 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:02.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0fa80a1260 msgr2=0x7f0fa80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0fa80a1260 0x7f0fa80a3710 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55fc437ee370 tx=0x7f0fb400b040 comp rx=0 tx=0).stop 2023-07-21T22:21:02.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 msgr2=0x7f0fbc197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc197a90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0fac005740 tx=0x7f0fac009510 comp rx=0 tx=0).stop 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 shutdown_connections 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f0fa80a1260 0x7f0fa80a3710 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0fbc101670 0x7f0fbc19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0fbc0ffaf0 0x7f0fbc197a90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 --2- 172.21.15.66:0/568192544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fbc100ce0 0x7f0fbc197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:02.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.857+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 >> 172.21.15.66:0/568192544 conn(0x7f0fbc0fb130 msgr2=0x7f0fbc0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:02.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.858+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 shutdown_connections 2023-07-21T22:21:02.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:02.858+0000 7f0fc0fff700 1 -- 172.21.15.66:0/568192544 wait complete. 2023-07-21T22:21:02.921 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T22:21:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 -- 172.21.15.66:0/1332827521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 msgr2=0x7f88c00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1332827521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 0x7f88c00ffef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0098f0 tx=0x7f88bc009c00 comp rx=0 tx=0).stop 2023-07-21T22:21:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 -- 172.21.15.66:0/1332827521 shutdown_connections 2023-07-21T22:21:03.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1332827521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0101660 0x7f88c0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1332827521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 0x7f88c00ffef0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1332827521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0100cd0 0x7f88c0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 -- 172.21.15.66:0/1332827521 >> 172.21.15.66:0/1332827521 conn(0x7f88c00fb0e0 msgr2=0x7f88c00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 -- 172.21.15.66:0/1332827521 shutdown_connections 2023-07-21T22:21:03.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.084+0000 7f88c83ff700 1 -- 172.21.15.66:0/1332827521 wait complete. 2023-07-21T22:21:03.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.085+0000 7f88c83ff700 1 Processor -- start 2023-07-21T22:21:03.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 -- start start 2023-07-21T22:21:03.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 0x7f88c0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0100cd0 0x7f88c0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 0x7f88c019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019f5c0 con 0x7f88c0101660 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019f700 con 0x7f88c0100cd0 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c83ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c019f890 con 0x7f88c00ffae0 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 0x7f88c019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c599a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0100cd0 0x7f88c0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c619b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 0x7f88c0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 0x7f88c019eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42368/0 (socket says 172.21.15.66:42368) 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c599a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0100cd0 0x7f88c0197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:60044/0 (socket says 172.21.15.66:60044) 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.086+0000 7f88c699c700 1 -- 172.21.15.66:0/1888838996 learned_addr learned my addr 172.21.15.66:0/1888838996 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c699c700 1 -- 172.21.15.66:0/1888838996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 msgr2=0x7f88c0197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c699c700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 0x7f88c0197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c699c700 1 -- 172.21.15.66:0/1888838996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0100cd0 msgr2=0x7f88c0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c699c700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0100cd0 0x7f88c0197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c699c700 1 -- 172.21.15.66:0/1888838996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc009520 con 0x7f88c0101660 2023-07-21T22:21:03.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c699c700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 0x7f88c019eff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f88b800ef30 tx=0x7f88b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88b77fe700 1 -- 172.21.15.66:0/1888838996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8018070 con 0x7f88c0101660 2023-07-21T22:21:03.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c019fb10 con 0x7f88c0101660 2023-07-21T22:21:03.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88b77fe700 1 -- 172.21.15.66:0/1888838996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88b800f040 con 0x7f88c0101660 2023-07-21T22:21:03.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88b77fe700 1 -- 172.21.15.66:0/1888838996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b800ccf0 con 0x7f88c0101660 2023-07-21T22:21:03.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.087+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c01a0070 con 0x7f88c0101660 2023-07-21T22:21:03.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.090+0000 7f88b77fe700 1 -- 172.21.15.66:0/1888838996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f88b8003a00 con 0x7f88c0101660 2023-07-21T22:21:03.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 Processor -- start 2023-07-21T22:21:03.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 -- start start 2023-07-21T22:21:03.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88b77fe700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f88ac0a1260 0x7f88ac0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88b77fe700 1 -- 172.21.15.66:0/1888838996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f88b80cc8e0 con 0x7f88c0101660 2023-07-21T22:21:03.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf51150 0x55ac7bf4f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88c599a700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f88ac0a1260 0x7f88ac0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7be54c60 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bea79f0 con 0x55ac7bf507e0 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.092+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bea0e80 con 0x55ac7bf51150 2023-07-21T22:21:03.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf50bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf50bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:60050/0 (socket says 172.21.15.66:60050) 2023-07-21T22:21:03.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 -- 172.21.15.66:0/2179624462 learned_addr learned my addr 172.21.15.66:0/2179624462 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:03.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c599a700 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf51150 0x55ac7bf4f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 -- 172.21.15.66:0/2179624462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf51150 msgr2=0x55ac7bf4f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf51150 0x55ac7bf4f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 -- 172.21.15.66:0/2179624462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 msgr2=0x55ac7bf4fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 -- 172.21.15.66:0/2179624462 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac7bcced90 con 0x55ac7bf507e0 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c699c700 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c599a700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f88ac0a1260 0x7f88ac0a3710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005fd0 tx=0x7f88b0011720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88c619b700 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf50bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0098c0 tx=0x7f88bc005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ac9d100e475d24 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.093+0000 7f88b4bfe700 1 -- 172.21.15.66:0/2179624462 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc00f720 con 0x55ac7bf507e0 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88b4bfe700 1 -- 172.21.15.66:0/2179624462 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88bc01d040 con 0x55ac7bf507e0 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88b4bfe700 1 -- 172.21.15.66:0/2179624462 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc005680 con 0x55ac7bf507e0 2023-07-21T22:21:03.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2179624462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 msgr2=0x55ac7bf50bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf50bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0098c0 tx=0x7f88bc005dc0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2179624462 shutdown_connections 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf50bf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2179624462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf51150 0x55ac7bf4f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2179624462 >> 172.21.15.66:0/2179624462 conn(0x55ac7bf44ea0 msgr2=0x55ac7bf4a310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2179624462 shutdown_connections 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.094+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2179624462 wait complete. 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.095+0000 7f88dbe04b80 1 -- start start 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.095+0000 7f88dbe04b80 1 Processor -- start 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf59050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf59590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7be40a70 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7be40770 con 0x55ac7bf51150 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bd248d0 con 0x55ac7bf507e0 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88dbe04b80 10 client.0 _ll_register_callbacks cb 0x55ac7be3e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c619b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf59050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf59590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf59590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43344/0 (socket says 172.21.15.66:43344) 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c599a700 1 -- 172.21.15.66:0/3354428333 learned_addr learned my addr 172.21.15.66:0/3354428333 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:03.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c619b700 1 -- 172.21.15.66:0/3354428333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 msgr2=0x55ac7bf59590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c619b700 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf59590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c619b700 1 -- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 msgr2=0x55ac7bf605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c619b700 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.096+0000 7f88c619b700 1 -- 172.21.15.66:0/3354428333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c01a0070 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88c619b700 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf59050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00fa40 tx=0x7f88bc00f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa52ededabe65b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88a9ffb700 1 -- 172.21.15.66:0/3354428333 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc00b970 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac7bdfa4a0 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88a9ffb700 1 -- 172.21.15.66:0/3354428333 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f88bc00bad0 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88a9ffb700 1 -- 172.21.15.66:0/3354428333 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88bc005920 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88dbe04b80 10 client.15285 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:03.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f88c0108850 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.097+0000 7f88a9ffb700 1 -- 172.21.15.66:0/3354428333 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac7bdfa4a0 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88a9ffb700 1 -- 172.21.15.66:0/3354428333 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88bc028070 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88dbe04b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:03.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88dbe04b80 10 client.15285 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:21:03.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88dbe04b80 20 client.15285 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:03.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bf63de0 0x55ac7bf63500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88dbe04b80 4 client.15285 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:03.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac7be99e60 con 0x55ac7bf63de0 2023-07-21T22:21:03.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.098+0000 7f88c599a700 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bf63de0 0x55ac7bf63500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.099+0000 7f88a9ffb700 10 client.15285 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:03.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.099+0000 7f88c599a700 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bf63de0 0x55ac7bf63500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd1030922f8c7518 server_cookie=bdd957b418b7879b in_seq=0 out_seq=0 2023-07-21T22:21:03.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.099+0000 7f88a9ffb700 1 -- 172.21.15.66:0/3354428333 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ac7be99e60 con 0x55ac7bf63de0 2023-07-21T22:21:03.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.099+0000 7f88a9ffb700 10 client.15285 handle_command_reply: tid=0 2023-07-21T22:21:03.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 client.15285 shutdown 2023-07-21T22:21:03.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 msgr2=0x55ac7bf59050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf59050 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00fa40 tx=0x7f88bc00f960 comp rx=0 tx=0).stop 2023-07-21T22:21:03.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 shutdown_connections 2023-07-21T22:21:03.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bf63de0 0x55ac7bf63500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf59050 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/3354428333 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf59590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88a9ffb700 0 client.15285 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 >> 172.21.15.66:0/3354428333 conn(0x55ac7bf44ea0 msgr2=0x55ac7bf499b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 shutdown_connections 2023-07-21T22:21:03.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 1 -- 172.21.15.66:0/3354428333 wait complete. 2023-07-21T22:21:03.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.100+0000 7f88dbe04b80 20 client.15285 trim_cache size 0 max 16384 2023-07-21T22:21:03.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 Processor -- start 2023-07-21T22:21:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 -- start start 2023-07-21T22:21:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf60590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bf47010 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bf59f50 con 0x55ac7bf51150 2023-07-21T22:21:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7be99e60 con 0x55ac7bf507e0 2023-07-21T22:21:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88c619b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88c619b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42392/0 (socket says 172.21.15.66:42392) 2023-07-21T22:21:03.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88c699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88c619b700 1 -- 172.21.15.66:0/2774056323 learned_addr learned my addr 172.21.15.66:0/2774056323 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:03.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.119+0000 7f88c599a700 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf60590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c619b700 1 -- 172.21.15.66:0/2774056323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 msgr2=0x55ac7bf60590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c619b700 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf60590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c619b700 1 -- 172.21.15.66:0/2774056323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 msgr2=0x55ac7bf4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c619b700 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c619b700 1 -- 172.21.15.66:0/2774056323 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac7bcb7c10 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c699c700 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:03.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88c619b700 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4f300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005850 tx=0x7f88bc005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da934ff890a6d35 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88b53ff700 1 -- 172.21.15.66:0/2774056323 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc005920 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88b53ff700 1 -- 172.21.15.66:0/2774056323 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88bc0055d0 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.120+0000 7f88b53ff700 1 -- 172.21.15.66:0/2774056323 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0038e0 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2774056323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 msgr2=0x55ac7bf4f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4f300 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f88bc005850 tx=0x7f88bc005e50 comp rx=0 tx=0).stop 2023-07-21T22:21:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2774056323 shutdown_connections 2023-07-21T22:21:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf51150 0x55ac7bf4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf4f300 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/2774056323 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf507e0 0x55ac7bf60590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2774056323 >> 172.21.15.66:0/2774056323 conn(0x55ac7bf44ea0 msgr2=0x55ac7bf57d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2774056323 shutdown_connections 2023-07-21T22:21:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 -- 172.21.15.66:0/2774056323 wait complete. 2023-07-21T22:21:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.121+0000 7f88dbe04b80 1 -- start start 2023-07-21T22:21:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 Processor -- start 2023-07-21T22:21:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf5bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 0x55ac7bfb7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bf60710 con 0x55ac7bf51150 2023-07-21T22:21:03.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bf60880 con 0x55ac7bf507e0 2023-07-21T22:21:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac7bfb81c0 con 0x55ac7bf4f9c0 2023-07-21T22:21:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88dbe04b80 10 client.0 _ll_register_callbacks cb 0x7f88b8009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88c699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 0x55ac7bfb7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88c699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 0x55ac7bfb7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42398/0 (socket says 172.21.15.66:42398) 2023-07-21T22:21:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88c699c700 1 -- 172.21.15.66:0/228050706 learned_addr learned my addr 172.21.15.66:0/228050706 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88c619b700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.122+0000 7f88c599a700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf5bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88c699c700 1 -- 172.21.15.66:0/228050706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf4f9c0 msgr2=0x55ac7bf5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88c699c700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88c699c700 1 -- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 msgr2=0x55ac7bf5bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88c699c700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf5bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88c699c700 1 -- 172.21.15.66:0/228050706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c019fb10 con 0x55ac7bf51150 2023-07-21T22:21:03.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88c699c700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 0x55ac7bfb7c80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f88b80090a0 tx=0x7f88b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8748cbfe5b7047bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88a9ffb700 1 -- 172.21.15.66:0/228050706 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b1070 con 0x55ac7bf51150 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88a9ffb700 1 -- 172.21.15.66:0/228050706 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f88b809d880 con 0x55ac7bf51150 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88bc01d040 con 0x55ac7bf51150 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88a9ffb700 1 -- 172.21.15.66:0/228050706 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88b80b92b0 con 0x55ac7bf51150 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88dbe04b80 10 client.15291 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:03.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.123+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f88c0108850 con 0x55ac7bf51150 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88a9ffb700 1 -- 172.21.15.66:0/228050706 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88bc01d040 con 0x55ac7bf51150 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88a9ffb700 1 -- 172.21.15.66:0/228050706 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f88b8098d40 con 0x55ac7bf51150 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88dbe04b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88dbe04b80 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88dbe04b80 20 client.15291 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bfbcf20 0x55ac7bfb9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:03.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88dbe04b80 4 client.15291 mds_command: new command op to 24572 tid=0[{"prefix": "status"}] 2023-07-21T22:21:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ac7be5c000 con 0x55ac7bfbcf20 2023-07-21T22:21:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.124+0000 7f88c619b700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bfbcf20 0x55ac7bfb9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88a9ffb700 10 client.15291 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88c619b700 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bfbcf20 0x55ac7bfb9890 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46c91fb3618ab76 server_cookie=e88240425c8c19b5 in_seq=0 out_seq=0 2023-07-21T22:21:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88a9ffb700 1 -- 172.21.15.66:0/228050706 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55ac7be5c000 con 0x55ac7bfbcf20 2023-07-21T22:21:03.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88a9ffb700 10 client.15291 handle_command_reply: tid=0 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88dbe04b80 1 client.15291 shutdown 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 msgr2=0x55ac7bfb7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 0x55ac7bfb7c80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f88b80090a0 tx=0x7f88b8004ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 shutdown_connections 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ac7bfbcf20 0x55ac7bfb9890 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.125+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ac7bf51150 0x55ac7bfb7c80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ac7bf4f9c0 0x55ac7bf5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88dbe04b80 1 --2- 172.21.15.66:0/228050706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac7bf507e0 0x55ac7bf5bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88a9ffb700 0 client.15291 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:03.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 >> 172.21.15.66:0/228050706 conn(0x55ac7bf44ea0 msgr2=0x55ac7bf4fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 shutdown_connections 2023-07-21T22:21:03.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88dbe04b80 1 -- 172.21.15.66:0/228050706 wait complete. 2023-07-21T22:21:03.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88dbe04b80 20 client.15291 trim_cache size 0 max 16384 2023-07-21T22:21:03.187 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:03.187 INFO:teuthology.orchestra.run.smithi066.stdout: "cluster_fsid": "6bcb348c-2813-11ee-9b34-001a4aab830c", 2023-07-21T22:21:03.187 INFO:teuthology.orchestra.run.smithi066.stdout: "whoami": 0, 2023-07-21T22:21:03.187 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 24572, 2023-07-21T22:21:03.187 INFO:teuthology.orchestra.run.smithi066.stdout: "want_state": "up:active", 2023-07-21T22:21:03.187 INFO:teuthology.orchestra.run.smithi066.stdout: "state": "up:active", 2023-07-21T22:21:03.188 INFO:teuthology.orchestra.run.smithi066.stdout: "fs_name": "cephfs", 2023-07-21T22:21:03.188 INFO:teuthology.orchestra.run.smithi066.stdout: "rank_uptime": 115.725139549, 2023-07-21T22:21:03.188 INFO:teuthology.orchestra.run.smithi066.stdout: "mdsmap_epoch": 23, 2023-07-21T22:21:03.189 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch": 93, 2023-07-21T22:21:03.189 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_epoch_barrier": 89, 2023-07-21T22:21:03.189 INFO:teuthology.orchestra.run.smithi066.stdout: "uptime": 122.351237103 2023-07-21T22:21:03.190 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:03.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f88ac0a1260 msgr2=0x7f88ac0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f88ac0a1260 0x7f88ac0a3710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005fd0 tx=0x7f88b0011720 comp rx=0 tx=0).stop 2023-07-21T22:21:03.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 msgr2=0x7f88c019eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:03.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.126+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 0x7f88c019eff0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f88b800ef30 tx=0x7f88b800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 shutdown_connections 2023-07-21T22:21:03.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f88ac0a1260 0x7f88ac0a3710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88c0101660 0x7f88c019eff0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88c00ffae0 0x7f88c0197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 --2- 172.21.15.66:0/1888838996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88c0100cd0 0x7f88c0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:03.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 >> 172.21.15.66:0/1888838996 conn(0x7f88c00fb0e0 msgr2=0x7f88c0106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:03.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 shutdown_connections 2023-07-21T22:21:03.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:03.127+0000 7f88c83ff700 1 -- 172.21.15.66:0/1888838996 wait complete. 2023-07-21T22:21:03.195 INFO:teuthology.run_tasks:Running task kclient... 2023-07-21T22:21:03.209 INFO:tasks.kclient:Mounting kernel clients... 2023-07-21T22:21:03.210 INFO:tasks.kclient:config is {'client.0': {}} 2023-07-21T22:21:03.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:03.210 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-07-21T22:21:03.255 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-07-21T22:21:03.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:03.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-07-21T22:21:03.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:03.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2023-07-21T22:21:03.441 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:03 smithi066 ceph-mon[126379]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:21:03.486 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-07-21T22:21:03.487 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-21T22:21:03.487 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:03.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-21T22:21:03.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:03 smithi149 ceph-mon[129268]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:21:03.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:03 smithi104 ceph-mon[129021]: pgmap v336: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:21:04.235 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:21:05.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:05 smithi066 ceph-mon[126379]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:21:05.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:05 smithi149 ceph-mon[129268]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:21:05.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 -- 172.21.15.66:0/694508144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56181007a0 msgr2=0x7f5618104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 --2- 172.21.15.66:0/694508144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56181007a0 0x7f5618104770 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f56080099c0 tx=0x7f5608009cd0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 -- 172.21.15.66:0/694508144 shutdown_connections 2023-07-21T22:21:05.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 --2- 172.21.15.66:0/694508144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56180ffe00 0x7f56181001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 --2- 172.21.15.66:0/694508144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5618104ea0 0x7f561810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 --2- 172.21.15.66:0/694508144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56181007a0 0x7f5618104770 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 -- 172.21.15.66:0/694508144 >> 172.21.15.66:0/694508144 conn(0x7f56180fb660 msgr2=0x7f56180fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:05.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 -- 172.21.15.66:0/694508144 shutdown_connections 2023-07-21T22:21:05.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.716+0000 7f561fe12700 1 -- 172.21.15.66:0/694508144 wait complete. 2023-07-21T22:21:05.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.717+0000 7f561fe12700 1 Processor -- start 2023-07-21T22:21:05.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.717+0000 7f561fe12700 1 -- start start 2023-07-21T22:21:05.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.717+0000 7f561fe12700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 0x7f5618197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.717+0000 7f561fe12700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56181007a0 0x7f56181983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.717+0000 7f561fe12700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618104ea0 0x7f561819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561fe12700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819ce20 con 0x7f56180ffe00 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561fe12700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819cf90 con 0x7f5618104ea0 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561fe12700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561819d150 con 0x7f56181007a0 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 0x7f5618197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 0x7f5618197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42414/0 (socket says 172.21.15.66:42414) 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 -- 172.21.15.66:0/340567745 learned_addr learned my addr 172.21.15.66:0/340567745 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561d3ad700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56181007a0 0x7f56181983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561e3af700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618104ea0 0x7f561819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 -- 172.21.15.66:0/340567745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56181007a0 msgr2=0x7f56181983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56181007a0 0x7f56181983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 -- 172.21.15.66:0/340567745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618104ea0 msgr2=0x7f561819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:05.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618104ea0 0x7f561819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:05.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.718+0000 7f561dbae700 1 -- 172.21.15.66:0/340567745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080095f0 con 0x7f56180ffe00 2023-07-21T22:21:05.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.719+0000 7f561dbae700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 0x7f5618197e60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f561400e9a0 tx=0x7f561400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.719+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614009790 con 0x7f56180ffe00 2023-07-21T22:21:05.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.719+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5614010430 con 0x7f56180ffe00 2023-07-21T22:21:05.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.719+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561819d430 con 0x7f56180ffe00 2023-07-21T22:21:05.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.719+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614018680 con 0x7f56180ffe00 2023-07-21T22:21:05.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.719+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56181a4c90 con 0x7f56180ffe00 2023-07-21T22:21:05.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.720+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561804e210 con 0x7f56180ffe00 2023-07-21T22:21:05.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.722+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f56140105a0 con 0x7f56180ffe00 2023-07-21T22:21:05.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.723+0000 7f560effd700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56040a11e0 0x7f56040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:05.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.724+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5614014070 con 0x7f56180ffe00 2023-07-21T22:21:05.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.724+0000 7f561d3ad700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56040a11e0 0x7f56040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:05.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.725+0000 7f561d3ad700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56040a11e0 0x7f56040a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5608009fd0 tx=0x7f5608005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:05.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.728+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5614051770 con 0x7f56180ffe00 2023-07-21T22:21:05.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:05 smithi104 ceph-mon[129021]: pgmap v337: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:21:06.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.980+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f56181a4ec0 con 0x7f56180ffe00 2023-07-21T22:21:06.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.981+0000 7f560effd700 1 -- 172.21.15.66:0/340567745 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5614096250 con 0x7f56180ffe00 2023-07-21T22:21:06.027 INFO:teuthology.orchestra.run.smithi066.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-07-21T22:21:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56040a11e0 msgr2=0x7f56040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56040a11e0 0x7f56040a3690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5608009fd0 tx=0x7f5608005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 msgr2=0x7f5618197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 0x7f5618197e60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f561400e9a0 tx=0x7f561400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 shutdown_connections 2023-07-21T22:21:06.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f56040a11e0 0x7f56040a3690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5618104ea0 0x7f561819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f56180ffe00 0x7f5618197e60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.983+0000 7f561fe12700 1 --2- 172.21.15.66:0/340567745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56181007a0 0x7f56181983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.984+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 >> 172.21.15.66:0/340567745 conn(0x7f56180fb660 msgr2=0x7f56180fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.984+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 shutdown_connections 2023-07-21T22:21:06.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:05.984+0000 7f561fe12700 1 -- 172.21.15.66:0/340567745 wait complete. 2023-07-21T22:21:06.434 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-21T22:21:06.434 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-21T22:21:06.434 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi066.front.sepia.ceph.com 2023-07-21T22:21:06.434 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-21T22:21:06.435 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:06.435 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-21T22:21:06.435 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-21T22:21:06.435 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-21T22:21:06.435 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-21T22:21:06.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:06.436 DEBUG:teuthology.orchestra.run.smithi066:> ip addr 2023-07-21T22:21:06.482 INFO:teuthology.orchestra.run.smithi066.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-21T22:21:06.482 INFO:teuthology.orchestra.run.smithi066.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-21T22:21:06.482 INFO:teuthology.orchestra.run.smithi066.stdout: inet 127.0.0.1/8 scope host lo 2023-07-21T22:21:06.482 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 ::1/128 scope host 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:84 brd ff:ff:ff:ff:ff:ff 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f0 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:21:06.483 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:a8:b4:85 brd ff:ff:ff:ff:ff:ff 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout: altname enp7s0f1 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bc brd ff:ff:ff:ff:ff:ff 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout: link/ether 0c:c4:7a:b6:f8:bd brd ff:ff:ff:ff:ff:ff 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout: inet 172.21.15.66/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft 41234sec preferred_lft 41234sec 2023-07-21T22:21:06.484 INFO:teuthology.orchestra.run.smithi066.stdout: inet6 fe80::1007:d553:b784:5869/64 scope link noprefixroute 2023-07-21T22:21:06.485 INFO:teuthology.orchestra.run.smithi066.stdout: valid_lft forever preferred_lft forever 2023-07-21T22:21:06.485 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-21T22:21:06.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:06.486 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:21:06.487 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:21:06.487 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add name ceph-brx type bridge 2023-07-21T22:21:06.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr flush dev ceph-brx 2023-07-21T22:21:06.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set ceph-brx up 2023-07-21T22:21:06.488 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-21T22:21:06.489 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:21:06.668 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:06 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/340567745' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:21:06.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:06 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/340567745' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:21:06.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:06.743 DEBUG:teuthology.orchestra.run.smithi066:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-21T22:21:06.815 INFO:teuthology.orchestra.run.smithi066.stdout:1 2023-07-21T22:21:06.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:06.818 DEBUG:teuthology.orchestra.run.smithi066:> route 2023-07-21T22:21:06.865 INFO:teuthology.orchestra.run.smithi066.stdout:Kernel IP routing table 2023-07-21T22:21:06.866 INFO:teuthology.orchestra.run.smithi066.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-21T22:21:06.866 INFO:teuthology.orchestra.run.smithi066.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-21T22:21:06.866 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-21T22:21:06.866 INFO:teuthology.orchestra.run.smithi066.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-21T22:21:06.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:06.867 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:21:06.868 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:21:06.868 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-21T22:21:06.868 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-21T22:21:06.869 DEBUG:teuthology.orchestra.run.smithi066:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-21T22:21:06.869 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:21:06.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:06 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/340567745' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T22:21:07.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:07.049 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-07-21T22:21:07.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:07.095 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list-id 2023-07-21T22:21:07.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:07.179 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:21:07.179 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:21:07.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T22:21:07.180 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-21T22:21:07.180 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:21:07.368 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-21T22:21:07.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:07.369 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:21:07.369 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:21:07.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-21T22:21:07.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-21T22:21:07.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-21T22:21:07.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-21T22:21:07.370 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-21T22:21:07.370 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:21:07.584 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:07 smithi066 ceph-mon[126379]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:21:07.585 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:07 smithi066 ceph-mon[126379]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-21T22:21:07.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:07.632 DEBUG:teuthology.orchestra.run.smithi066:> (cd / && exec bash -c ' 2023-07-21T22:21:07.632 DEBUG:teuthology.orchestra.run.smithi066:> set -e 2023-07-21T22:21:07.632 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set brx.0 up 2023-07-21T22:21:07.632 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link set dev brx.0 master ceph-brx 2023-07-21T22:21:07.633 DEBUG:teuthology.orchestra.run.smithi066:> ') 2023-07-21T22:21:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:07 smithi149 ceph-mon[129268]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:21:07.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:07 smithi149 ceph-mon[129268]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-21T22:21:07.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:21:07.813 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:07.857 INFO:teuthology.orchestra.run.smithi066.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-21T22:21:07.858 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T22:21:07.858 DEBUG:teuthology.orchestra.run.smithi066:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:07.868 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:07 smithi104 ceph-mon[129021]: pgmap v338: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T22:21:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:07 smithi104 ceph-mon[129021]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-07-21T22:21:07.941 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-07-21T22:21:07.942 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-07-21T22:21:07.942 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca 2023-07-21T22:21:07.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:07.943 DEBUG:teuthology.orchestra.run.smithi066:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-21T22:21:08.090 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-21T22:21:08.091 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-07-21T22:21:08.091 INFO:teuthology.orchestra.run.smithi066.stdout:invalid new device string format 2023-07-21T22:21:08.549 INFO:teuthology.orchestra.run.smithi066.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2023-07-21T22:21:08.549 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2023-07-21T22:21:08.549 INFO:teuthology.orchestra.run.smithi066.stdout:invalid new device string format 2023-07-21T22:21:08.550 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: resolved to: "172.21.15.66:3300,172.21.15.104:3300,172.21.15.149:3300" 2023-07-21T22:21:08.550 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.66:3300,172.21.15.104:3300,172.21.15.149:3300:/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca 2023-07-21T22:21:08.550 INFO:teuthology.orchestra.run.smithi066.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=6bcb348c-2813-11ee-9b34-001a4aab830c" will pass to kernel 2023-07-21T22:21:08.556 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-07-21T22:21:08.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:08.557 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:08.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:08.635 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:21:08.724 INFO:teuthology.orchestra.run.smithi066.stdout:client25342 2023-07-21T22:21:08.724 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:21:08.724 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:21:08.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:08.769 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:21:08.897 INFO:teuthology.orchestra.run.smithi066.stdout:client25342 2023-07-21T22:21:08.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:08.898 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-07-21T22:21:08.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:08 smithi066 ceph-mon[126379]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T22:21:08.986 INFO:teuthology.orchestra.run.smithi066.stdout:client25342 2023-07-21T22:21:08.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:21:08.987 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c.client25342/status') 2023-07-21T22:21:09.097 INFO:teuthology.orchestra.run.smithi066.stdout:instance: client.25342 (3)172.21.15.66:0/2550694025 2023-07-21T22:21:09.098 INFO:teuthology.orchestra.run.smithi066.stdout:blocklisted: false 2023-07-21T22:21:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records in 2023-07-21T22:21:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:0+1 records out 2023-07-21T22:21:09.098 INFO:teuthology.orchestra.run.smithi066.stderr:71 bytes copied, 7.2773e-05 s, 976 kB/s 2023-07-21T22:21:09.101 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-21T22:21:09.111 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-21T22:21:09.124 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:21:09.125 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:08 smithi104 ceph-mon[129021]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T22:21:09.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:08 smithi149 ceph-mon[129268]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T22:21:09.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.288+0000 7fea09bff700 1 -- 172.21.15.66:0/1229721008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea04101680 msgr2=0x7fea04107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.288+0000 7fea09bff700 1 --2- 172.21.15.66:0/1229721008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea04101680 0x7fea04107f40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009f90 tx=0x7fe9fc00b330 comp rx=0 tx=0).stop 2023-07-21T22:21:09.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 -- 172.21.15.66:0/1229721008 shutdown_connections 2023-07-21T22:21:09.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 --2- 172.21.15.66:0/1229721008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea040ffb00 0x7fea040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 --2- 172.21.15.66:0/1229721008 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea04101680 0x7fea04107f40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 --2- 172.21.15.66:0/1229721008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea04100cf0 0x7fea04101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 -- 172.21.15.66:0/1229721008 >> 172.21.15.66:0/1229721008 conn(0x7fea040fb0e0 msgr2=0x7fea040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:09.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 -- 172.21.15.66:0/1229721008 shutdown_connections 2023-07-21T22:21:09.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.289+0000 7fea09bff700 1 -- 172.21.15.66:0/1229721008 wait complete. 2023-07-21T22:21:09.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 Processor -- start 2023-07-21T22:21:09.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 -- start start 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 0x7fea04197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea04100cf0 0x7fea04198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea04101680 0x7fea0419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0419f6a0 con 0x7fea040ffb00 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0419f7e0 con 0x7fea04100cf0 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.290+0000 7fea09bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0419f970 con 0x7fea04101680 2023-07-21T22:21:09.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 0x7fea04197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 0x7fea04197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42428/0 (socket says 172.21.15.66:42428) 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 -- 172.21.15.66:0/1892720199 learned_addr learned my addr 172.21.15.66:0/1892720199 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea02ffd700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea04100cf0 0x7fea04198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea03fff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea04101680 0x7fea0419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 -- 172.21.15.66:0/1892720199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea04101680 msgr2=0x7fea0419f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea04101680 0x7fea0419f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 -- 172.21.15.66:0/1892720199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea04100cf0 msgr2=0x7fea04198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea04100cf0 0x7fea04198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 -- 172.21.15.66:0/1892720199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40095f0 con 0x7fea040ffb00 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.291+0000 7fea037fe700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 0x7fea04197b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400ebf0 tx=0x7fe9f400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.292+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4009a10 con 0x7fea040ffb00 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.292+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f400cc90 con 0x7fea040ffb00 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.292+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc009c40 con 0x7fea040ffb00 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.292+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4018750 con 0x7fea040ffb00 2023-07-21T22:21:09.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.292+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0419ffd0 con 0x7fea040ffb00 2023-07-21T22:21:09.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.293+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0410b330 con 0x7fea040ffb00 2023-07-21T22:21:09.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.295+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe9f40188b0 con 0x7fea040ffb00 2023-07-21T22:21:09.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.297+0000 7fea00ff9700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe9ec0a11a0 0x7fe9ec0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.297+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9f4014070 con 0x7fea040ffb00 2023-07-21T22:21:09.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.298+0000 7fea02ffd700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe9ec0a11a0 0x7fe9ec0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.298+0000 7fea02ffd700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe9ec0a11a0 0x7fe9ec0a3650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009c60 tx=0x7fe9f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:09.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.302+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe9f4096670 con 0x7fea040ffb00 2023-07-21T22:21:09.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.516+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea0410b520 con 0x7fea040ffb00 2023-07-21T22:21:09.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.518+0000 7fea00ff9700 1 -- 172.21.15.66:0/1892720199 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7fe9f4099c90 con 0x7fea040ffb00 2023-07-21T22:21:09.563 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:09.564 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:21:09.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe9ec0a11a0 msgr2=0x7fe9ec0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe9ec0a11a0 0x7fe9ec0a3650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009c60 tx=0x7fe9f8009260 comp rx=0 tx=0).stop 2023-07-21T22:21:09.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 msgr2=0x7fea04197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 0x7fea04197b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400ebf0 tx=0x7fe9f400ef00 comp rx=0 tx=0).stop 2023-07-21T22:21:09.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 shutdown_connections 2023-07-21T22:21:09.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe9ec0a11a0 0x7fe9ec0a3650 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea04101680 0x7fea0419f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fea040ffb00 0x7fea04197b20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 --2- 172.21.15.66:0/1892720199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea04100cf0 0x7fea04198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 >> 172.21.15.66:0/1892720199 conn(0x7fea040fb0e0 msgr2=0x7fea040fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:09.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 shutdown_connections 2023-07-21T22:21:09.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.520+0000 7fea09bff700 1 -- 172.21.15.66:0/1892720199 wait complete. 2023-07-21T22:21:09.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:21:09.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:21:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 -- 172.21.15.66:0/3040199523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac0ffae0 msgr2=0x7f5fac0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/3040199523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac0ffae0 0x7f5fac0ffef0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009990 tx=0x7f5f9c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 -- 172.21.15.66:0/3040199523 shutdown_connections 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/3040199523 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac101660 0x7f5fac107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/3040199523 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac0ffae0 0x7f5fac0ffef0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/3040199523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac100cd0 0x7f5fac101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 -- 172.21.15.66:0/3040199523 >> 172.21.15.66:0/3040199523 conn(0x7f5fac0fb0e0 msgr2=0x7f5fac0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 -- 172.21.15.66:0/3040199523 shutdown_connections 2023-07-21T22:21:09.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.752+0000 7f5fb1fff700 1 -- 172.21.15.66:0/3040199523 wait complete. 2023-07-21T22:21:09.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 Processor -- start 2023-07-21T22:21:09.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 -- start start 2023-07-21T22:21:09.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 0x7f5fac197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac100cd0 0x7f5fac197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac101660 0x7f5fac19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac19f620 con 0x7f5fac0ffae0 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac19f760 con 0x7f5fac101660 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.753+0000 7f5fb1fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac19f8f0 con 0x7f5fac100cd0 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 0x7f5fac197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 0x7f5fac197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42448/0 (socket says 172.21.15.66:42448) 2023-07-21T22:21:09.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5faaffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac100cd0 0x7f5fac197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 -- 172.21.15.66:0/1245474807 learned_addr learned my addr 172.21.15.66:0/1245474807 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fabfff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac101660 0x7f5fac19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 -- 172.21.15.66:0/1245474807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac100cd0 msgr2=0x7f5fac197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac100cd0 0x7f5fac197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 -- 172.21.15.66:0/1245474807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac101660 msgr2=0x7f5fac19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac101660 0x7f5fac19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 -- 172.21.15.66:0/1245474807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0095f0 con 0x7f5fac0ffae0 2023-07-21T22:21:09.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.754+0000 7f5fab7fe700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 0x7f5fac197aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:09.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.755+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c01d070 con 0x7f5fac0ffae0 2023-07-21T22:21:09.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.755+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac19fb70 con 0x7f5fac0ffae0 2023-07-21T22:21:09.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.755+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f9c022470 con 0x7f5fac0ffae0 2023-07-21T22:21:09.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.755+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c017650 con 0x7f5fac0ffae0 2023-07-21T22:21:09.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.755+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac19fff0 con 0x7f5fac0ffae0 2023-07-21T22:21:09.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.756+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac10b250 con 0x7f5fac0ffae0 2023-07-21T22:21:09.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.758+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5f9c0225e0 con 0x7f5fac0ffae0 2023-07-21T22:21:09.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.759+0000 7f5fa8ff9700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5f940a1150 0x7f5f940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:09.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.759+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5f9c0cdfb0 con 0x7f5fac0ffae0 2023-07-21T22:21:09.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.760+0000 7f5faaffd700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5f940a1150 0x7f5f940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:09.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.760+0000 7f5faaffd700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5f940a1150 0x7f5f940a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009c60 tx=0x7f5fa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:09.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.763+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5f9c0973c0 con 0x7f5fac0ffae0 2023-07-21T22:21:09.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:09 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1892720199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:10.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.975+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5fac1a0600 con 0x7f5fac0ffae0 2023-07-21T22:21:10.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.976+0000 7f5fa8ff9700 1 -- 172.21.15.66:0/1245474807 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f9c09a9e0 con 0x7f5fac0ffae0 2023-07-21T22:21:10.021 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:21:10.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5f940a1150 msgr2=0x7f5f940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5f940a1150 0x7f5f940a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009c60 tx=0x7f5fa0009260 comp rx=0 tx=0).stop 2023-07-21T22:21:10.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 msgr2=0x7f5fac197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 0x7f5fac197aa0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c000c00 tx=0x7f5f9c004730 comp rx=0 tx=0).stop 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 shutdown_connections 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f5f940a1150 0x7f5f940a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fac101660 0x7f5fac19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5fac0ffae0 0x7f5fac197aa0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 --2- 172.21.15.66:0/1245474807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5fac100cd0 0x7f5fac197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 >> 172.21.15.66:0/1245474807 conn(0x7f5fac0fb0e0 msgr2=0x7f5fac0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:10.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 shutdown_connections 2023-07-21T22:21:10.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:09.978+0000 7f5fb1fff700 1 -- 172.21.15.66:0/1245474807 wait complete. 2023-07-21T22:21:10.042 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:21:10.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:09 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1892720199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:10.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:09 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1892720199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:10.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 -- 172.21.15.66:0/4343987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc34100cf0 msgr2=0x7efc34101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 --2- 172.21.15.66:0/4343987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc34100cf0 0x7efc34101140 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009990 tx=0x7efc1c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 -- 172.21.15.66:0/4343987 shutdown_connections 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 --2- 172.21.15.66:0/4343987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34101680 0x7efc34107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 --2- 172.21.15.66:0/4343987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc340ffb00 0x7efc340fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 --2- 172.21.15.66:0/4343987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc34100cf0 0x7efc34101140 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 -- 172.21.15.66:0/4343987 >> 172.21.15.66:0/4343987 conn(0x7efc340fb120 msgr2=0x7efc340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 -- 172.21.15.66:0/4343987 shutdown_connections 2023-07-21T22:21:10.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.207+0000 7efc393ff700 1 -- 172.21.15.66:0/4343987 wait complete. 2023-07-21T22:21:10.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.208+0000 7efc393ff700 1 Processor -- start 2023-07-21T22:21:10.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.208+0000 7efc393ff700 1 -- start start 2023-07-21T22:21:10.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc393ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 0x7efc341958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc393ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc34100cf0 0x7efc34195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc393ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34101680 0x7efc3419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc393ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3419d450 con 0x7efc340ffb00 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc393ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3419d590 con 0x7efc34101680 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc393ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3419d720 con 0x7efc34100cf0 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc32ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 0x7efc341958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc2bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc34100cf0 0x7efc34195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc32ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 0x7efc341958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42470/0 (socket says 172.21.15.66:42470) 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc32ffd700 1 -- 172.21.15.66:0/3772572483 learned_addr learned my addr 172.21.15.66:0/3772572483 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:10.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.209+0000 7efc337fe700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34101680 0x7efc3419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc32ffd700 1 -- 172.21.15.66:0/3772572483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc34100cf0 msgr2=0x7efc34195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc32ffd700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc34100cf0 0x7efc34195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc32ffd700 1 -- 172.21.15.66:0/3772572483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34101680 msgr2=0x7efc3419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc32ffd700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34101680 0x7efc3419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc32ffd700 1 -- 172.21.15.66:0/3772572483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c0095f0 con 0x7efc340ffb00 2023-07-21T22:21:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc32ffd700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 0x7efc341958d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efc2400e970 tx=0x7efc2400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24009790 con 0x7efc340ffb00 2023-07-21T22:21:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc3419da00 con 0x7efc340ffb00 2023-07-21T22:21:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc2400c7e0 con 0x7efc340ffb00 2023-07-21T22:21:10.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24003c70 con 0x7efc340ffb00 2023-07-21T22:21:10.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.210+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3419df60 con 0x7efc340ffb00 2023-07-21T22:21:10.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.212+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc341070a0 con 0x7efc340ffb00 2023-07-21T22:21:10.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.213+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efc2400cc50 con 0x7efc340ffb00 2023-07-21T22:21:10.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.215+0000 7efc30ff9700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc200a1150 0x7efc200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.215+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efc24014070 con 0x7efc340ffb00 2023-07-21T22:21:10.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.215+0000 7efc2bfff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc200a1150 0x7efc200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.216+0000 7efc2bfff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc200a1150 0x7efc200a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b2e0 tx=0x7efc1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:10.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.219+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efc240961f0 con 0x7efc340ffb00 2023-07-21T22:21:10.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.428+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc340611c0 con 0x7efc340ffb00 2023-07-21T22:21:10.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.428+0000 7efc30ff9700 1 -- 172.21.15.66:0/3772572483 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc24003dd0 con 0x7efc340ffb00 2023-07-21T22:21:10.475 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:10.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc200a1150 msgr2=0x7efc200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc200a1150 0x7efc200a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00b2e0 tx=0x7efc1c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 msgr2=0x7efc341958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 0x7efc341958d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efc2400e970 tx=0x7efc2400ec80 comp rx=0 tx=0).stop 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 shutdown_connections 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efc200a1150 0x7efc200a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34101680 0x7efc3419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc340ffb00 0x7efc341958d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 --2- 172.21.15.66:0/3772572483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc34100cf0 0x7efc34195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 >> 172.21.15.66:0/3772572483 conn(0x7efc340fb120 msgr2=0x7efc341041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 shutdown_connections 2023-07-21T22:21:10.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.431+0000 7efc393ff700 1 -- 172.21.15.66:0/3772572483 wait complete. 2023-07-21T22:21:10.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:21:10.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 -- 172.21.15.66:0/2305461471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1724101670 msgr2=0x7f1724107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 --2- 172.21.15.66:0/2305461471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1724101670 0x7f1724107f30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f171c00b0a0 tx=0x7f171c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 -- 172.21.15.66:0/2305461471 shutdown_connections 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 --2- 172.21.15.66:0/2305461471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17240ffaf0 0x7f17240fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 --2- 172.21.15.66:0/2305461471 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1724101670 0x7f1724107f30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 --2- 172.21.15.66:0/2305461471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1724100ce0 0x7f1724101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 -- 172.21.15.66:0/2305461471 >> 172.21.15.66:0/2305461471 conn(0x7f17240fb130 msgr2=0x7f17240fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 -- 172.21.15.66:0/2305461471 shutdown_connections 2023-07-21T22:21:10.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.661+0000 7f172a9ff700 1 -- 172.21.15.66:0/2305461471 wait complete. 2023-07-21T22:21:10.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.662+0000 7f172a9ff700 1 Processor -- start 2023-07-21T22:21:10.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 -- start start 2023-07-21T22:21:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 0x7f1724195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1724100ce0 0x7f1724195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1724101670 0x7f172419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419d4c0 con 0x7f17240ffaf0 2023-07-21T22:21:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419d600 con 0x7f1724101670 2023-07-21T22:21:10.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f172a9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172419d790 con 0x7f1724100ce0 2023-07-21T22:21:10.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.663+0000 7f1723fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 0x7f1724195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 0x7f1724195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42476/0 (socket says 172.21.15.66:42476) 2023-07-21T22:21:10.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 -- 172.21.15.66:0/53076666 learned_addr learned my addr 172.21.15.66:0/53076666 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:10.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f17237fe700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1724100ce0 0x7f1724195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1728f9c700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1724101670 0x7f172419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 -- 172.21.15.66:0/53076666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1724100ce0 msgr2=0x7f1724195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1724100ce0 0x7f1724195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 -- 172.21.15.66:0/53076666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1724101670 msgr2=0x7f172419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1724101670 0x7f172419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 -- 172.21.15.66:0/53076666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718009520 con 0x7f17240ffaf0 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f1723fff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 0x7f1724195940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f171400e970 tx=0x7f171400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:10.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714009790 con 0x7f17240ffaf0 2023-07-21T22:21:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f171400c7e0 con 0x7f17240ffaf0 2023-07-21T22:21:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c009d00 con 0x7f17240ffaf0 2023-07-21T22:21:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.664+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714003c70 con 0x7f17240ffaf0 2023-07-21T22:21:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.665+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172419de40 con 0x7f17240ffaf0 2023-07-21T22:21:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.666+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172410b320 con 0x7f17240ffaf0 2023-07-21T22:21:10.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.668+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f171400cc50 con 0x7f17240ffaf0 2023-07-21T22:21:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.669+0000 7f17217fa700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f170c0a11d0 0x7f170c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.669+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1714014070 con 0x7f17240ffaf0 2023-07-21T22:21:10.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.669+0000 7f17237fe700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f170c0a11d0 0x7f170c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:10.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.670+0000 7f17237fe700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f170c0a11d0 0x7f170c0a3680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1718009df0 tx=0x7f1718009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:10.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.673+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f17140961a0 con 0x7f17240ffaf0 2023-07-21T22:21:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.889+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1724106970 con 0x7f17240ffaf0 2023-07-21T22:21:10.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.889+0000 7f17217fa700 1 -- 172.21.15.66:0/53076666 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1714003dd0 con 0x7f17240ffaf0 2023-07-21T22:21:10.935 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:21:10.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f170c0a11d0 msgr2=0x7f170c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f170c0a11d0 0x7f170c0a3680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1718009df0 tx=0x7f1718009260 comp rx=0 tx=0).stop 2023-07-21T22:21:10.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 msgr2=0x7f1724195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:10.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 0x7f1724195940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f171400e970 tx=0x7f171400ec80 comp rx=0 tx=0).stop 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 shutdown_connections 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f170c0a11d0 0x7f170c0a3680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1724101670 0x7f172419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17240ffaf0 0x7f1724195940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 --2- 172.21.15.66:0/53076666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1724100ce0 0x7f1724195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 >> 172.21.15.66:0/53076666 conn(0x7f17240fb130 msgr2=0x7f17240fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:10.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 shutdown_connections 2023-07-21T22:21:10.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:10.892+0000 7f172a9ff700 1 -- 172.21.15.66:0/53076666 wait complete. 2023-07-21T22:21:10.956 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:21:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:10 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1245474807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:10 smithi104 ceph-mon[129021]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-21T22:21:11.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:10 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3772572483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 -- 172.21.15.66:0/1326100504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe041005c0 msgr2=0x7fbe041029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 --2- 172.21.15.66:0/1326100504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe041005c0 0x7fbe041029a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009990 tx=0x7fbdf4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 -- 172.21.15.66:0/1326100504 shutdown_connections 2023-07-21T22:21:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 --2- 172.21.15.66:0/1326100504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 0x7fbe04107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 --2- 172.21.15.66:0/1326100504 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe041005c0 0x7fbe041029a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 --2- 172.21.15.66:0/1326100504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe04102ee0 0x7fbe041052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 -- 172.21.15.66:0/1326100504 >> 172.21.15.66:0/1326100504 conn(0x7fbe040fa000 msgr2=0x7fbe040fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 -- 172.21.15.66:0/1326100504 shutdown_connections 2023-07-21T22:21:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.121+0000 7fbe09dff700 1 -- 172.21.15.66:0/1326100504 wait complete. 2023-07-21T22:21:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.122+0000 7fbe09dff700 1 Processor -- start 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.122+0000 7fbe09dff700 1 -- start start 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe09dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041005c0 0x7fbe04197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe09dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 0x7fbe04197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe09dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 0x7fbe0419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe09dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419f5e0 con 0x7fbe04102ee0 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe09dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419f720 con 0x7fbe04105800 2023-07-21T22:21:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe09dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0419f8b0 con 0x7fbe041005c0 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe02ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 0x7fbe04197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe02ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 0x7fbe04197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42506/0 (socket says 172.21.15.66:42506) 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe037fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041005c0 0x7fbe04197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe02ffd700 1 -- 172.21.15.66:0/2837040614 learned_addr learned my addr 172.21.15.66:0/2837040614 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe03fff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 0x7fbe0419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe02ffd700 1 -- 172.21.15.66:0/2837040614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041005c0 msgr2=0x7fbe04197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.123+0000 7fbe02ffd700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041005c0 0x7fbe04197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe02ffd700 1 -- 172.21.15.66:0/2837040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 msgr2=0x7fbe0419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe02ffd700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 0x7fbe0419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe02ffd700 1 -- 172.21.15.66:0/2837040614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf40095f0 con 0x7fbe04102ee0 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe037fe700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041005c0 0x7fbe04197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe03fff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 0x7fbe0419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe02ffd700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 0x7fbe04197fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e9a0 tx=0x7fbdf800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:11.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8009790 con 0x7fbe04102ee0 2023-07-21T22:21:11.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0419fb90 con 0x7fbe04102ee0 2023-07-21T22:21:11.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdf8010430 con 0x7fbe04102ee0 2023-07-21T22:21:11.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8018680 con 0x7fbe04102ee0 2023-07-21T22:21:11.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.124+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe041a00a0 con 0x7fbe04102ee0 2023-07-21T22:21:11.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.125+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe04107370 con 0x7fbe04102ee0 2023-07-21T22:21:11.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.127+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbdf80105a0 con 0x7fbe04102ee0 2023-07-21T22:21:11.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.130+0000 7fbe00ff9700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbdec0a1150 0x7fbdec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.130+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbdf8014070 con 0x7fbe04102ee0 2023-07-21T22:21:11.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.130+0000 7fbe037fe700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbdec0a1150 0x7fbdec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.131+0000 7fbe037fe700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbdec0a1150 0x7fbdec0a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400b2e0 tx=0x7fbdf4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.133+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbdf8051770 con 0x7fbe04102ee0 2023-07-21T22:21:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1245474807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:10 smithi066 ceph-mon[126379]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-21T22:21:11.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3772572483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:10 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1245474807' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:11.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:10 smithi149 ceph-mon[129268]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-21T22:21:11.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:10 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3772572483' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:11.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.345+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbe040611c0 con 0x7fbe04102ee0 2023-07-21T22:21:11.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.345+0000 7fbe00ff9700 1 -- 172.21.15.66:0/2837040614 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbdf8096210 con 0x7fbe04102ee0 2023-07-21T22:21:11.391 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:11.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbdec0a1150 msgr2=0x7fbdec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbdec0a1150 0x7fbdec0a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400b2e0 tx=0x7fbdf4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 msgr2=0x7fbe04197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 0x7fbe04197fa0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800e9a0 tx=0x7fbdf800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 shutdown_connections 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fbdec0a1150 0x7fbdec0a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe04105800 0x7fbe0419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbe041005c0 0x7fbe04197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 --2- 172.21.15.66:0/2837040614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbe04102ee0 0x7fbe04197fa0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 >> 172.21.15.66:0/2837040614 conn(0x7fbe040fa000 msgr2=0x7fbe040fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 shutdown_connections 2023-07-21T22:21:11.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.348+0000 7fbe09dff700 1 -- 172.21.15.66:0/2837040614 wait complete. 2023-07-21T22:21:11.413 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:21:11.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.576+0000 7f3aecfff700 1 -- 172.21.15.66:0/3085958120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae8100cf0 msgr2=0x7f3ae8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.576+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3085958120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8101140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3adc009990 tx=0x7f3adc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 -- 172.21.15.66:0/3085958120 shutdown_connections 2023-07-21T22:21:11.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3085958120 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8101680 0x7f3ae8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3085958120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae80ffb00 0x7f3ae80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3085958120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8101140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 -- 172.21.15.66:0/3085958120 >> 172.21.15.66:0/3085958120 conn(0x7f3ae80fb0e0 msgr2=0x7f3ae80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:11.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 -- 172.21.15.66:0/3085958120 shutdown_connections 2023-07-21T22:21:11.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.577+0000 7f3aecfff700 1 -- 172.21.15.66:0/3085958120 wait complete. 2023-07-21T22:21:11.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.578+0000 7f3aecfff700 1 Processor -- start 2023-07-21T22:21:11.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.578+0000 7f3aecfff700 1 -- start start 2023-07-21T22:21:11.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3aecfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae80ffb00 0x7f3ae8195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3aecfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3aecfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8101680 0x7f3ae819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3aecfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae819d3d0 con 0x7f3ae8100cf0 2023-07-21T22:21:11.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3aecfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae819d510 con 0x7f3ae80ffb00 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3aecfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae819d6a0 con 0x7f3ae8101680 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3ae5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3ae5d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42522/0 (socket says 172.21.15.66:42522) 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3ae5d9b700 1 -- 172.21.15.66:0/3265372850 learned_addr learned my addr 172.21.15.66:0/3265372850 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3ae659c700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae80ffb00 0x7f3ae8195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.579+0000 7f3ae6d9d700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8101680 0x7f3ae819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ae5d9b700 1 -- 172.21.15.66:0/3265372850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8101680 msgr2=0x7f3ae819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ae5d9b700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8101680 0x7f3ae819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ae5d9b700 1 -- 172.21.15.66:0/3265372850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae80ffb00 msgr2=0x7f3ae8195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ae5d9b700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae80ffb00 0x7f3ae8195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ae5d9b700 1 -- 172.21.15.66:0/3265372850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009c40 con 0x7f3ae8100cf0 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ae5d9b700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8195d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00b620 tx=0x7f3adc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc01d070 con 0x7f3ae8100cf0 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc0095f0 con 0x7f3ae8100cf0 2023-07-21T22:21:11.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3adc022470 con 0x7f3ae8100cf0 2023-07-21T22:21:11.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc017650 con 0x7f3ae8100cf0 2023-07-21T22:21:11.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.580+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae819dbb0 con 0x7f3ae8100cf0 2023-07-21T22:21:11.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.581+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae810b320 con 0x7f3ae8100cf0 2023-07-21T22:21:11.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.583+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3adc0225e0 con 0x7f3ae8100cf0 2023-07-21T22:21:11.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.585+0000 7f3ad77fe700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ad00a11a0 0x7f3ad00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:11.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.585+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3adc0ce860 con 0x7f3ae8100cf0 2023-07-21T22:21:11.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.585+0000 7f3ae659c700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ad00a11a0 0x7f3ad00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:11.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.586+0000 7f3ae659c700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ad00a11a0 0x7f3ad00a3650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8100b50 tx=0x7f3ad8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:11.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.587+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3adc097c70 con 0x7f3ae8100cf0 2023-07-21T22:21:11.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.801+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ae810b510 con 0x7f3ae8100cf0 2023-07-21T22:21:11.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.803+0000 7f3ad77fe700 1 -- 172.21.15.66:0/3265372850 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7f3adc09b290 con 0x7f3ae8100cf0 2023-07-21T22:21:11.848 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:11.848 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:21:11.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ad00a11a0 msgr2=0x7f3ad00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ad00a11a0 0x7f3ad00a3650 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8100b50 tx=0x7f3ad8009260 comp rx=0 tx=0).stop 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 msgr2=0x7f3ae8195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8195d90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00b620 tx=0x7f3adc005740 comp rx=0 tx=0).stop 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 shutdown_connections 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f3ad00a11a0 0x7f3ad00a3650 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ae80ffb00 0x7f3ae8195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3ae8101680 0x7f3ae819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 --2- 172.21.15.66:0/3265372850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3ae8100cf0 0x7f3ae8195d90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:11.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 >> 172.21.15.66:0/3265372850 conn(0x7f3ae80fb0e0 msgr2=0x7f3ae80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:11.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 shutdown_connections 2023-07-21T22:21:11.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:11.805+0000 7f3aecfff700 1 -- 172.21.15.66:0/3265372850 wait complete. 2023-07-21T22:21:11.870 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:21:11.870 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:21:12.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 -- 172.21.15.66:0/411099559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 msgr2=0x7eff3c109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 --2- 172.21.15.66:0/411099559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c109e30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7eff3400b0a0 tx=0x7eff3400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 -- 172.21.15.66:0/411099559 shutdown_connections 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 --2- 172.21.15.66:0/411099559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c074550 0x7eff3c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 --2- 172.21.15.66:0/411099559 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c109e30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 --2- 172.21.15.66:0/411099559 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff3c072f70 0x7eff3c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 -- 172.21.15.66:0/411099559 >> 172.21.15.66:0/411099559 conn(0x7eff3c0fb130 msgr2=0x7eff3c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:12.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 -- 172.21.15.66:0/411099559 shutdown_connections 2023-07-21T22:21:12.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.039+0000 7eff409ff700 1 -- 172.21.15.66:0/411099559 wait complete. 2023-07-21T22:21:12.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.041+0000 7eff409ff700 1 Processor -- start 2023-07-21T22:21:12.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.041+0000 7eff409ff700 1 -- start start 2023-07-21T22:21:12.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff409ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff3c072f70 0x7eff3c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff409ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c074550 0x7eff3c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff409ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff409ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19f700 con 0x7eff3c1079b0 2023-07-21T22:21:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff409ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19f840 con 0x7eff3c074550 2023-07-21T22:21:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff409ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c19f9d0 con 0x7eff3c072f70 2023-07-21T22:21:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56760/0 (socket says 172.21.15.66:56760) 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff3c072f70 0x7eff3c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 -- 172.21.15.66:0/1953269284 learned_addr learned my addr 172.21.15.66:0/1953269284 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff33fff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c074550 0x7eff3c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 -- 172.21.15.66:0/1953269284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff3c072f70 msgr2=0x7eff3c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff3c072f70 0x7eff3c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 -- 172.21.15.66:0/1953269284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c074550 msgr2=0x7eff3c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c074550 0x7eff3c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.042+0000 7eff3ad9d700 1 -- 172.21.15.66:0/1953269284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34009d00 con 0x7eff3c1079b0 2023-07-21T22:21:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.043+0000 7eff3ad9d700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c19f130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7eff34009fd0 tx=0x7eff3400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:12.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.043+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400e040 con 0x7eff3c1079b0 2023-07-21T22:21:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.043+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff34003e10 con 0x7eff3c1079b0 2023-07-21T22:21:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.043+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c19fc50 con 0x7eff3c1079b0 2023-07-21T22:21:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.043+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3401b6a0 con 0x7eff3c1079b0 2023-07-21T22:21:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.043+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c1a0070 con 0x7eff3c1079b0 2023-07-21T22:21:12.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.044+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3c106060 con 0x7eff3c1079b0 2023-07-21T22:21:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.046+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7eff3401d470 con 0x7eff3c1079b0 2023-07-21T22:21:12.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.047+0000 7eff337fe700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff280a11d0 0x7eff280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.048+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7eff340cdfd0 con 0x7eff3c1079b0 2023-07-21T22:21:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.048+0000 7eff3a59c700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff280a11d0 0x7eff280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.049+0000 7eff3a59c700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff280a11d0 0x7eff280a3680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff3c074920 tx=0x7eff2c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:12.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.051+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7eff340973e0 con 0x7eff3c1079b0 2023-07-21T22:21:12.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:11 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/53076666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:12.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:11 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2837040614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:12.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:11 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/53076666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:12.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:11 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2837040614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:12.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:11 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/53076666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:12.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:11 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2837040614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:12.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.257+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff3c10d540 con 0x7eff3c1079b0 2023-07-21T22:21:12.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.258+0000 7eff337fe700 1 -- 172.21.15.66:0/1953269284 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff3409aa00 con 0x7eff3c1079b0 2023-07-21T22:21:12.303 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff280a11d0 msgr2=0x7eff280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff280a11d0 0x7eff280a3680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7eff3c074920 tx=0x7eff2c0073d0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 msgr2=0x7eff3c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c19f130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7eff34009fd0 tx=0x7eff3400bce0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 shutdown_connections 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7eff280a11d0 0x7eff280a3680 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7eff3c1079b0 0x7eff3c19f130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff3c072f70 0x7eff3c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 --2- 172.21.15.66:0/1953269284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff3c074550 0x7eff3c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.260+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 >> 172.21.15.66:0/1953269284 conn(0x7eff3c0fb130 msgr2=0x7eff3c109c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:12.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.261+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 shutdown_connections 2023-07-21T22:21:12.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.261+0000 7eff409ff700 1 -- 172.21.15.66:0/1953269284 wait complete. 2023-07-21T22:21:12.325 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:21:12.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/662582484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88101660 msgr2=0x7f4b88107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/662582484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88101660 0x7f4b88107f20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000b0a0 tx=0x7f4b8000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/662582484 shutdown_connections 2023-07-21T22:21:12.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/662582484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b880ffae0 0x7f4b880ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/662582484 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88101660 0x7f4b88107f20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/662582484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88100cd0 0x7f4b88101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/662582484 >> 172.21.15.66:0/662582484 conn(0x7f4b880fb0e0 msgr2=0x7f4b880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:12.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/662582484 shutdown_connections 2023-07-21T22:21:12.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.494+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/662582484 wait complete. 2023-07-21T22:21:12.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.495+0000 7f4b8f7ff700 1 Processor -- start 2023-07-21T22:21:12.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 -- start start 2023-07-21T22:21:12.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b880ffae0 0x7f4b880742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 0x7f4b88072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88101660 0x7f4b88072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b880735e0 con 0x7f4b88100cd0 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b88073750 con 0x7f4b880ffae0 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8f7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8819f9f0 con 0x7f4b88101660 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8cd9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 0x7f4b88072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8cd9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 0x7f4b88072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56772/0 (socket says 172.21.15.66:56772) 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8cd9a700 1 -- 172.21.15.66:0/2924377502 learned_addr learned my addr 172.21.15.66:0/2924377502 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8dd9c700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88101660 0x7f4b88072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.496+0000 7f4b8d59b700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b880ffae0 0x7f4b880742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8cd9a700 1 -- 172.21.15.66:0/2924377502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88101660 msgr2=0x7f4b88072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8cd9a700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88101660 0x7f4b88072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8cd9a700 1 -- 172.21.15.66:0/2924377502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b880ffae0 msgr2=0x7f4b880742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8cd9a700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b880ffae0 0x7f4b880742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8cd9a700 1 -- 172.21.15.66:0/2924377502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b80009d00 con 0x7f4b88100cd0 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8dd9c700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88101660 0x7f4b88072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8cd9a700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 0x7f4b88072900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d980 tx=0x7f4b7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:12.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78004220 con 0x7f4b88100cd0 2023-07-21T22:21:12.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b78007640 con 0x7f4b88100cd0 2023-07-21T22:21:12.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b78005690 con 0x7f4b88100cd0 2023-07-21T22:21:12.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8819fcd0 con 0x7f4b88100cd0 2023-07-21T22:21:12.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.497+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b881a0160 con 0x7f4b88100cd0 2023-07-21T22:21:12.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.498+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b88107020 con 0x7f4b88100cd0 2023-07-21T22:21:12.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.500+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4b78004750 con 0x7f4b88100cd0 2023-07-21T22:21:12.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.502+0000 7f4b7e7fc700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b740a11d0 0x7f4b740a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.502+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4b780cca30 con 0x7f4b88100cd0 2023-07-21T22:21:12.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.502+0000 7f4b8d59b700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b740a11d0 0x7f4b740a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.503+0000 7f4b8d59b700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b740a11d0 0x7f4b740a3680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4b88100b30 tx=0x7f4b84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:12.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.506+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4b78095db0 con 0x7f4b88100cd0 2023-07-21T22:21:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.713+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b88061270 con 0x7f4b88100cd0 2023-07-21T22:21:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.713+0000 7f4b7e7fc700 1 -- 172.21.15.66:0/2924377502 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b780160a0 con 0x7f4b88100cd0 2023-07-21T22:21:12.759 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.715+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b740a11d0 msgr2=0x7f4b740a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.715+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b740a11d0 0x7f4b740a3680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4b88100b30 tx=0x7f4b84009260 comp rx=0 tx=0).stop 2023-07-21T22:21:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.715+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 msgr2=0x7f4b88072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.715+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 0x7f4b88072900 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b7800d980 tx=0x7f4b7800dc90 comp rx=0 tx=0).stop 2023-07-21T22:21:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.715+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 shutdown_connections 2023-07-21T22:21:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4b740a11d0 0x7f4b740a3680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b880ffae0 0x7f4b880742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b88101660 0x7f4b88072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 --2- 172.21.15.66:0/2924377502 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4b88100cd0 0x7f4b88072900 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 >> 172.21.15.66:0/2924377502 conn(0x7f4b880fb0e0 msgr2=0x7f4b88104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 shutdown_connections 2023-07-21T22:21:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.716+0000 7f4b8f7ff700 1 -- 172.21.15.66:0/2924377502 wait complete. 2023-07-21T22:21:12.780 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:21:12.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 -- 172.21.15.66:0/19555245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c101660 msgr2=0x7fe57c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 --2- 172.21.15.66:0/19555245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c101660 0x7fe57c107f20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe57400b0a0 tx=0x7fe57400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 -- 172.21.15.66:0/19555245 shutdown_connections 2023-07-21T22:21:12.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 --2- 172.21.15.66:0/19555245 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c101660 0x7fe57c107f20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 --2- 172.21.15.66:0/19555245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 --2- 172.21.15.66:0/19555245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 0x7fe57c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.945+0000 7fe5817ff700 1 -- 172.21.15.66:0/19555245 >> 172.21.15.66:0/19555245 conn(0x7fe57c0fb0e0 msgr2=0x7fe57c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:12.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.946+0000 7fe5817ff700 1 -- 172.21.15.66:0/19555245 shutdown_connections 2023-07-21T22:21:12.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.946+0000 7fe5817ff700 1 -- 172.21.15.66:0/19555245 wait complete. 2023-07-21T22:21:12.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.946+0000 7fe5817ff700 1 Processor -- start 2023-07-21T22:21:12.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 -- start start 2023-07-21T22:21:12.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 0x7fe57c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c101660 0x7fe57c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0735e0 con 0x7fe57c0ffae0 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c073750 con 0x7fe57c100cd0 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe5817ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c19f940 con 0x7fe57c101660 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe57affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe57affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56792/0 (socket says 172.21.15.66:56792) 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe57affd700 1 -- 172.21.15.66:0/3479682225 learned_addr learned my addr 172.21.15.66:0/3479682225 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe57a7fc700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 0x7fe57c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.947+0000 7fe57b7fe700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c101660 0x7fe57c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57affd700 1 -- 172.21.15.66:0/3479682225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c101660 msgr2=0x7fe57c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57affd700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c101660 0x7fe57c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57affd700 1 -- 172.21.15.66:0/3479682225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 msgr2=0x7fe57c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57affd700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 0x7fe57c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57affd700 1 -- 172.21.15.66:0/3479682225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574009d00 con 0x7fe57c0ffae0 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57b7fe700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c101660 0x7fe57c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57a7fc700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 0x7fe57c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:12.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe57affd700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0742c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00d980 tx=0x7fe56c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:12.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00b950 con 0x7fe57c0ffae0 2023-07-21T22:21:12.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56c007620 con 0x7fe57c0ffae0 2023-07-21T22:21:12.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00f3c0 con 0x7fe57c0ffae0 2023-07-21T22:21:12.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.948+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c19fc20 con 0x7fe57c0ffae0 2023-07-21T22:21:12.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.949+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c1a00e0 con 0x7fe57c0ffae0 2023-07-21T22:21:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.950+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c106ff0 con 0x7fe57c0ffae0 2023-07-21T22:21:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.952+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe56c010640 con 0x7fe57c0ffae0 2023-07-21T22:21:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.953+0000 7fe563fff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5640a11d0 0x7fe5640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:12.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.953+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe56c0ccb10 con 0x7fe57c0ffae0 2023-07-21T22:21:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.953+0000 7fe57a7fc700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5640a11d0 0x7fe5640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:12.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.954+0000 7fe57a7fc700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5640a11d0 0x7fe5640a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe570009fd0 tx=0x7fe570009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:12.957+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe56c095e90 con 0x7fe57c0ffae0 2023-07-21T22:21:13.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:12 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3265372850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:13.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:12 smithi104 ceph-mon[129021]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-21T22:21:13.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:12 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1953269284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:12 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2924377502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:12 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3265372850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:13.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:12 smithi066 ceph-mon[126379]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-21T22:21:13.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:12 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1953269284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:12 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2924377502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.168+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe57c061270 con 0x7fe57c0ffae0 2023-07-21T22:21:13.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.168+0000 7fe563fff700 1 -- 172.21.15.66:0/3479682225 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe56c0160a0 con 0x7fe57c0ffae0 2023-07-21T22:21:13.214 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:21:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.170+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5640a11d0 msgr2=0x7fe5640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.170+0000 7fe5817ff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5640a11d0 0x7fe5640a3680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe570009fd0 tx=0x7fe570009260 comp rx=0 tx=0).stop 2023-07-21T22:21:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.170+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 msgr2=0x7fe57c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.170+0000 7fe5817ff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0742c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00d980 tx=0x7fe56c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:21:13.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 shutdown_connections 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fe5640a11d0 0x7fe5640a3680 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe57c101660 0x7fe57c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe57c0ffae0 0x7fe57c0742c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 --2- 172.21.15.66:0/3479682225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe57c100cd0 0x7fe57c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 >> 172.21.15.66:0/3479682225 conn(0x7fe57c0fb0e0 msgr2=0x7fe57c104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 shutdown_connections 2023-07-21T22:21:13.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.171+0000 7fe5817ff700 1 -- 172.21.15.66:0/3479682225 wait complete. 2023-07-21T22:21:13.234 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:21:13.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:12 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3265372850' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:13.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:12 smithi149 ceph-mon[129268]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-21T22:21:13.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:12 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1953269284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:12 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2924377502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 -- 172.21.15.66:0/2633752253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c0ffae0 msgr2=0x7f001c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 --2- 172.21.15.66:0/2633752253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c0ffae0 0x7f001c0ffef0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0018009990 tx=0x7f0018009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 -- 172.21.15.66:0/2633752253 shutdown_connections 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 --2- 172.21.15.66:0/2633752253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c101660 0x7f001c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 --2- 172.21.15.66:0/2633752253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c0ffae0 0x7f001c0ffef0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 --2- 172.21.15.66:0/2633752253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f001c100cd0 0x7f001c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 -- 172.21.15.66:0/2633752253 >> 172.21.15.66:0/2633752253 conn(0x7f001c0fb0e0 msgr2=0x7f001c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 -- 172.21.15.66:0/2633752253 shutdown_connections 2023-07-21T22:21:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.403+0000 7f00247ff700 1 -- 172.21.15.66:0/2633752253 wait complete. 2023-07-21T22:21:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.404+0000 7f00247ff700 1 Processor -- start 2023-07-21T22:21:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.404+0000 7f00247ff700 1 -- start start 2023-07-21T22:21:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f00247ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 0x7f001c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f00247ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 0x7f001c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f00247ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f001c101660 0x7f001c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f00247ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19f650 con 0x7f001c100cd0 2023-07-21T22:21:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f00247ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19f790 con 0x7f001c0ffae0 2023-07-21T22:21:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f00247ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c19f920 con 0x7f001c101660 2023-07-21T22:21:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f0021d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 0x7f001c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f002259b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 0x7f001c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f0021d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 0x7f001c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56808/0 (socket says 172.21.15.66:56808) 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f002259b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 0x7f001c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48178/0 (socket says 172.21.15.66:48178) 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f0021d9a700 1 -- 172.21.15.66:0/855345154 learned_addr learned my addr 172.21.15.66:0/855345154 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.405+0000 7f0022d9c700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f001c101660 0x7f001c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f0021d9a700 1 -- 172.21.15.66:0/855345154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f001c101660 msgr2=0x7f001c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f0021d9a700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f001c101660 0x7f001c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f0021d9a700 1 -- 172.21.15.66:0/855345154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 msgr2=0x7f001c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f0021d9a700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 0x7f001c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f0021d9a700 1 -- 172.21.15.66:0/855345154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180095f0 con 0x7f001c100cd0 2023-07-21T22:21:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f002259b700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 0x7f001c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f0021d9a700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 0x7f001c198010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f000c00d950 tx=0x7f000c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00f890 con 0x7f001c100cd0 2023-07-21T22:21:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f000c010460 con 0x7f001c100cd0 2023-07-21T22:21:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c00e610 con 0x7f001c100cd0 2023-07-21T22:21:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c19fc00 con 0x7f001c100cd0 2023-07-21T22:21:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.406+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c1a00c0 con 0x7f001c100cd0 2023-07-21T22:21:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.408+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c192170 con 0x7f001c100cd0 2023-07-21T22:21:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.410+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f000c0108d0 con 0x7f001c100cd0 2023-07-21T22:21:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.411+0000 7f00137fe700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f00080a1150 0x7f00080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.411+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f000c0cca90 con 0x7f001c100cd0 2023-07-21T22:21:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.411+0000 7f002259b700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f00080a1150 0x7f00080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.412+0000 7f002259b700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f00080a1150 0x7f00080a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f001800b5c0 tx=0x7f0018005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.415+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f000c095e10 con 0x7f001c100cd0 2023-07-21T22:21:13.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.625+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f001c0611c0 con 0x7f001c100cd0 2023-07-21T22:21:13.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.626+0000 7f00137fe700 1 -- 172.21.15.66:0/855345154 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f000c0160a0 con 0x7f001c100cd0 2023-07-21T22:21:13.671 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:13.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f00080a1150 msgr2=0x7f00080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f00080a1150 0x7f00080a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f001800b5c0 tx=0x7f0018005960 comp rx=0 tx=0).stop 2023-07-21T22:21:13.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 msgr2=0x7f001c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 0x7f001c198010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f000c00d950 tx=0x7f000c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 shutdown_connections 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f00080a1150 0x7f00080a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f001c0ffae0 0x7f001c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f001c101660 0x7f001c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 --2- 172.21.15.66:0/855345154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f001c100cd0 0x7f001c198010 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.628+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 >> 172.21.15.66:0/855345154 conn(0x7f001c0fb0e0 msgr2=0x7f001c106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:13.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.629+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 shutdown_connections 2023-07-21T22:21:13.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.629+0000 7f00247ff700 1 -- 172.21.15.66:0/855345154 wait complete. 2023-07-21T22:21:13.693 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-21T22:21:13.693 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:21:13.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 -- 172.21.15.66:0/3613975637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc101680 msgr2=0x7f34fc107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 --2- 172.21.15.66:0/3613975637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc101680 0x7f34fc107f40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f34f400b0a0 tx=0x7f34f400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 -- 172.21.15.66:0/3613975637 shutdown_connections 2023-07-21T22:21:13.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 --2- 172.21.15.66:0/3613975637 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc101680 0x7f34fc107f40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 --2- 172.21.15.66:0/3613975637 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34fc0ffb00 0x7f34fc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 --2- 172.21.15.66:0/3613975637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34fc100cf0 0x7f34fc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 -- 172.21.15.66:0/3613975637 >> 172.21.15.66:0/3613975637 conn(0x7f34fc0fb0e0 msgr2=0x7f34fc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:13.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.855+0000 7f35011ff700 1 -- 172.21.15.66:0/3613975637 shutdown_connections 2023-07-21T22:21:13.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.856+0000 7f35011ff700 1 -- 172.21.15.66:0/3613975637 wait complete. 2023-07-21T22:21:13.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 Processor -- start 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 -- start start 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 0x7f34fc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34fc100cf0 0x7f34fc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34fc101680 0x7f34fc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc19f6c0 con 0x7f34fc0ffb00 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc19f800 con 0x7f34fc100cf0 2023-07-21T22:21:13.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f35011ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc19f990 con 0x7f34fc101680 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f34fad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 0x7f34fc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f34fad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 0x7f34fc197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56824/0 (socket says 172.21.15.66:56824) 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f34fad9d700 1 -- 172.21.15.66:0/716335015 learned_addr learned my addr 172.21.15.66:0/716335015 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.857+0000 7f34fa59c700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34fc100cf0 0x7f34fc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fb59e700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34fc101680 0x7f34fc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fad9d700 1 -- 172.21.15.66:0/716335015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34fc101680 msgr2=0x7f34fc19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fad9d700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34fc101680 0x7f34fc19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fad9d700 1 -- 172.21.15.66:0/716335015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34fc100cf0 msgr2=0x7f34fc198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fad9d700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34fc100cf0 0x7f34fc198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fad9d700 1 -- 172.21.15.66:0/716335015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009d00 con 0x7f34fc0ffb00 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34fad9d700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 0x7f34fc197b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00e970 tx=0x7f34ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec009790 con 0x7f34fc0ffb00 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ec00c7e0 con 0x7f34fc0ffb00 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc19fc70 con 0x7f34fc0ffb00 2023-07-21T22:21:13.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.858+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec003c70 con 0x7f34fc0ffb00 2023-07-21T22:21:13.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.859+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1a01d0 con 0x7f34fc0ffb00 2023-07-21T22:21:13.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.860+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc10b330 con 0x7f34fc0ffb00 2023-07-21T22:21:13.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.862+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f34ec00cc50 con 0x7f34fc0ffb00 2023-07-21T22:21:13.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.864+0000 7f34e3fff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34e40a1220 0x7f34e40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:13.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.864+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f34ec014070 con 0x7f34fc0ffb00 2023-07-21T22:21:13.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.865+0000 7f34fa59c700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34e40a1220 0x7f34e40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:13.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.865+0000 7f34fa59c700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34e40a1220 0x7f34e40a36d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009c60 tx=0x7f34f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:13.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:13.868+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f34ec096170 con 0x7f34fc0ffb00 2023-07-21T22:21:13.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3479682225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:13.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/855345154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:14.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3479682225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:14.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/855345154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.126+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34fc1a0780 con 0x7f34fc0ffb00 2023-07-21T22:21:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.127+0000 7f34e3fff700 1 -- 172.21.15.66:0/716335015 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f34ec003dd0 con 0x7f34fc0ffb00 2023-07-21T22:21:14.172 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:14.173 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:21:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.129+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34e40a1220 msgr2=0x7f34e40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.129+0000 7f35011ff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34e40a1220 0x7f34e40a36d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009c60 tx=0x7f34f0009260 comp rx=0 tx=0).stop 2023-07-21T22:21:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.129+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 msgr2=0x7f34fc197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.129+0000 7f35011ff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 0x7f34fc197b40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00e970 tx=0x7f34ec00ec80 comp rx=0 tx=0).stop 2023-07-21T22:21:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 shutdown_connections 2023-07-21T22:21:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f34e40a1220 0x7f34e40a36d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34fc101680 0x7f34fc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f34fc0ffb00 0x7f34fc197b40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 --2- 172.21.15.66:0/716335015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34fc100cf0 0x7f34fc198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 >> 172.21.15.66:0/716335015 conn(0x7f34fc0fb0e0 msgr2=0x7f34fc106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 shutdown_connections 2023-07-21T22:21:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.130+0000 7f35011ff700 1 -- 172.21.15.66:0/716335015 wait complete. 2023-07-21T22:21:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:21:14.194 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-21T22:21:14.194 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-21T22:21:14.195 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:21:14.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3479682225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:14.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/855345154' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:14.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 -- 172.21.15.66:0/4049487312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f4100cf0 msgr2=0x7f91f4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 --2- 172.21.15.66:0/4049487312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f4100cf0 0x7f91f4101140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f91e40098f0 tx=0x7f91e4009c00 comp rx=0 tx=0).stop 2023-07-21T22:21:14.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 -- 172.21.15.66:0/4049487312 shutdown_connections 2023-07-21T22:21:14.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 --2- 172.21.15.66:0/4049487312 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4101680 0x7f91f4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 --2- 172.21.15.66:0/4049487312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91f40ffb00 0x7f91f40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 --2- 172.21.15.66:0/4049487312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f4100cf0 0x7f91f4101140 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 -- 172.21.15.66:0/4049487312 >> 172.21.15.66:0/4049487312 conn(0x7f91f40fb0e0 msgr2=0x7f91f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:14.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 -- 172.21.15.66:0/4049487312 shutdown_connections 2023-07-21T22:21:14.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.357+0000 7f91fbdff700 1 -- 172.21.15.66:0/4049487312 wait complete. 2023-07-21T22:21:14.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.358+0000 7f91fbdff700 1 Processor -- start 2023-07-21T22:21:14.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 -- start start 2023-07-21T22:21:14.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f40ffb00 0x7f91f4195910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 0x7f91f4195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91f4101680 0x7f91f419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f419d490 con 0x7f91f4100cf0 2023-07-21T22:21:14.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f419d5d0 con 0x7f91f40ffb00 2023-07-21T22:21:14.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fbdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f419d760 con 0x7f91f4101680 2023-07-21T22:21:14.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91f939a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 0x7f91f4195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91f9b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f40ffb00 0x7f91f4195910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91f9b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f40ffb00 0x7f91f4195910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48220/0 (socket says 172.21.15.66:48220) 2023-07-21T22:21:14.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91f939a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 0x7f91f4195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56840/0 (socket says 172.21.15.66:56840) 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91f9b9b700 1 -- 172.21.15.66:0/88465973 learned_addr learned my addr 172.21.15.66:0/88465973 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.359+0000 7f91fa39c700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91f4101680 0x7f91f419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91f939a700 1 -- 172.21.15.66:0/88465973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91f4101680 msgr2=0x7f91f419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91f939a700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91f4101680 0x7f91f419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91f939a700 1 -- 172.21.15.66:0/88465973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f40ffb00 msgr2=0x7f91f4195910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91f939a700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f40ffb00 0x7f91f4195910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91f939a700 1 -- 172.21.15.66:0/88465973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e4009520 con 0x7f91f4100cf0 2023-07-21T22:21:14.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91f939a700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 0x7f91f4195e50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009230 tx=0x7f91e400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e401c070 con 0x7f91f4100cf0 2023-07-21T22:21:14.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e400f7f0 con 0x7f91f4100cf0 2023-07-21T22:21:14.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f419d9e0 con 0x7f91f4100cf0 2023-07-21T22:21:14.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4003ac0 con 0x7f91f4100cf0 2023-07-21T22:21:14.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.360+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f419de00 con 0x7f91f4100cf0 2023-07-21T22:21:14.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.362+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f41070e0 con 0x7f91f4100cf0 2023-07-21T22:21:14.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.364+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f91e4020030 con 0x7f91f4100cf0 2023-07-21T22:21:14.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.365+0000 7f91eaffd700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f91e00a11a0 0x7f91e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.365+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f91e40cdc60 con 0x7f91f4100cf0 2023-07-21T22:21:14.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.365+0000 7f91f9b9b700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f91e00a11a0 0x7f91e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.366+0000 7f91f9b9b700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f91e00a11a0 0x7f91e00a3650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009cf0 tx=0x7f91f0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.369+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f91e4097070 con 0x7f91f4100cf0 2023-07-21T22:21:14.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.574+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91f4060950 con 0x7f91e00a11a0 2023-07-21T22:21:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.588+0000 7f91eaffd700 1 -- 172.21.15.66:0/88465973 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427379 (secure 0 0 0) 0x7f91f4060950 con 0x7f91e00a11a0 2023-07-21T22:21:14.634 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f91e00a11a0 msgr2=0x7f91e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f91e00a11a0 0x7f91e00a3650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009cf0 tx=0x7f91f0009310 comp rx=0 tx=0).stop 2023-07-21T22:21:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 msgr2=0x7f91f4195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 0x7f91f4195e50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f91e4009230 tx=0x7f91e400b730 comp rx=0 tx=0).stop 2023-07-21T22:21:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 shutdown_connections 2023-07-21T22:21:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f91e00a11a0 0x7f91e00a3650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91f40ffb00 0x7f91f4195910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91f4101680 0x7f91f419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 --2- 172.21.15.66:0/88465973 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91f4100cf0 0x7f91f4195e50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 >> 172.21.15.66:0/88465973 conn(0x7f91f40fb0e0 msgr2=0x7f91f4104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:14.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 shutdown_connections 2023-07-21T22:21:14.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.593+0000 7f91fbdff700 1 -- 172.21.15.66:0/88465973 wait complete. 2023-07-21T22:21:14.649 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:21:14.658 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2023-07-21T22:21:14.210615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659093,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":760,"num_read_kb":506,"num_write":348,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335828,"kb_used_data":12404,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605996,"statfs":{"total":1151940427776,"available":1151596539904,"internally_reserved":0,"allocated":12701696,"data_stored":8143565,"data_compressed":389043,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5483,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":55,"num_write":10,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003385"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.239002+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:21:06.239002+0000","last_peered":"2023-07-21T22:21:06.239002+0000","last_clean":"2023-07-21T22:21:06.239002+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:21:06.239002+0000","last_undegraded":"2023-07-21T22:21:06.239002+0000","last_fullsized":"2023-07-21T22:21:06.239002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:18:11.706021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241220+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:21:06.241220+0000","last_peered":"2023-07-21T22:21:06.241220+0000","last_clean":"2023-07-21T22:21:06.241220+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:21:06.241220+0000","last_undegraded":"2023-07-21T22:21:06.241220+0000","last_fullsized":"2023-07-21T22:21:06.241220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:12.418124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244777+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:21:06.244777+0000","last_peered":"2023-07-21T22:21:06.244777+0000","last_clean":"2023-07-21T22:21:06.244777+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:21:06.244777+0000","last_undegraded":"2023-07-21T22:21:06.244777+0000","last_fullsized":"2023-07-21T22:21:06.244777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:49:28.752633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.231962+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:21:06.231962+0000","last_peered":"2023-07-21T22:21:06.231962+0000","last_clean":"2023-07-21T22:21:06.231962+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:21:06.231962+0000","last_undegraded":"2023-07-21T22:21:06.231962+0000","last_fullsized":"2023-07-21T22:21:06.231962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:10.307052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.238638+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:21:06.238638+0000","last_peered":"2023-07-21T22:21:06.238638+0000","last_clean":"2023-07-21T22:21:06.238638+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:21:06.238638+0000","last_undegraded":"2023-07-21T22:21:06.238638+0000","last_fullsized":"2023-07-21T22:21:06.238638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:34.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241211+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:21:06.241211+0000","last_peered":"2023-07-21T22:21:06.241211+0000","last_clean":"2023-07-21T22:21:06.241211+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:21:06.241211+0000","last_undegraded":"2023-07-21T22:21:06.241211+0000","last_fullsized":"2023-07-21T22:21:06.241211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:56:26.017768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233052+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:21:06.233052+0000","last_peered":"2023-07-21T22:21:06.233052+0000","last_clean":"2023-07-21T22:21:06.233052+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:21:06.233052+0000","last_undegraded":"2023-07-21T22:21:06.233052+0000","last_fullsized":"2023-07-21T22:21:06.233052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:39:12.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243697+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:21:06.243697+0000","last_peered":"2023-07-21T22:21:06.243697+0000","last_clean":"2023-07-21T22:21:06.243697+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:21:06.243697+0000","last_undegraded":"2023-07-21T22:21:06.243697+0000","last_fullsized":"2023-07-21T22:21:06.243697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:55.560292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236208+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:21:06.236208+0000","last_peered":"2023-07-21T22:21:06.236208+0000","last_clean":"2023-07-21T22:21:06.236208+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:21:06.236208+0000","last_undegraded":"2023-07-21T22:21:06.236208+0000","last_fullsized":"2023-07-21T22:21:06.236208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:25:01.311975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241324+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:21:06.241324+0000","last_peered":"2023-07-21T22:21:06.241324+0000","last_clean":"2023-07-21T22:21:06.241324+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:21:06.241324+0000","last_undegraded":"2023-07-21T22:21:06.241324+0000","last_fullsized":"2023-07-21T22:21:06.241324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:45:28.789239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229830+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:21:06.229830+0000","last_peered":"2023-07-21T22:21:06.229830+0000","last_clean":"2023-07-21T22:21:06.229830+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:21:06.229830+0000","last_undegraded":"2023-07-21T22:21:06.229830+0000","last_fullsized":"2023-07-21T22:21:06.229830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:42.902545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233939+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:21:06.233939+0000","last_peered":"2023-07-21T22:21:06.233939+0000","last_clean":"2023-07-21T22:21:06.233939+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:21:06.233939+0000","last_undegraded":"2023-07-21T22:21:06.233939+0000","last_fullsized":"2023-07-21T22:21:06.233939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:04:05.043773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240104+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:21:06.240104+0000","last_peered":"2023-07-21T22:21:06.240104+0000","last_clean":"2023-07-21T22:21:06.240104+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:21:06.240104+0000","last_undegraded":"2023-07-21T22:21:06.240104+0000","last_fullsized":"2023-07-21T22:21:06.240104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:42:32.982654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244272+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:21:06.244272+0000","last_peered":"2023-07-21T22:21:06.244272+0000","last_clean":"2023-07-21T22:21:06.244272+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:21:06.244272+0000","last_undegraded":"2023-07-21T22:21:06.244272+0000","last_fullsized":"2023-07-21T22:21:06.244272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:42.745730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241909+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:21:06.241909+0000","last_peered":"2023-07-21T22:21:06.241909+0000","last_clean":"2023-07-21T22:21:06.241909+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:21:06.241909+0000","last_undegraded":"2023-07-21T22:21:06.241909+0000","last_fullsized":"2023-07-21T22:21:06.241909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:19:27.316051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243456+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:21:06.243456+0000","last_peered":"2023-07-21T22:21:06.243456+0000","last_clean":"2023-07-21T22:21:06.243456+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:21:06.243456+0000","last_undegraded":"2023-07-21T22:21:06.243456+0000","last_fullsized":"2023-07-21T22:21:06.243456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:43.668214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.239421+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:21:06.239421+0000","last_peered":"2023-07-21T22:21:06.239421+0000","last_clean":"2023-07-21T22:21:06.239421+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:21:06.239421+0000","last_undegraded":"2023-07-21T22:21:06.239421+0000","last_fullsized":"2023-07-21T22:21:06.239421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:38:36.512107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232295+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:21:06.232295+0000","last_peered":"2023-07-21T22:21:06.232295+0000","last_clean":"2023-07-21T22:21:06.232295+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:21:06.232295+0000","last_undegraded":"2023-07-21T22:21:06.232295+0000","last_fullsized":"2023-07-21T22:21:06.232295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:01.043172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.210433+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:21:06.210433+0000","last_peered":"2023-07-21T22:21:06.210433+0000","last_clean":"2023-07-21T22:21:06.210433+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:21:06.210433+0000","last_undegraded":"2023-07-21T22:21:06.210433+0000","last_fullsized":"2023-07-21T22:21:06.210433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:10.034128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305676+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:20:38.305676+0000","last_peered":"2023-07-21T22:20:38.305676+0000","last_clean":"2023-07-21T22:20:38.305676+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:20:38.305676+0000","last_undegraded":"2023-07-21T22:20:38.305676+0000","last_fullsized":"2023-07-21T22:20:38.305676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:52.425113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298198+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:20:38.298198+0000","last_peered":"2023-07-21T22:20:38.298198+0000","last_clean":"2023-07-21T22:20:38.298198+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:20:38.298198+0000","last_undegraded":"2023-07-21T22:20:38.298198+0000","last_fullsized":"2023-07-21T22:20:38.298198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243894+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:21:06.243894+0000","last_peered":"2023-07-21T22:21:06.243894+0000","last_clean":"2023-07-21T22:21:06.243894+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:21:06.243894+0000","last_undegraded":"2023-07-21T22:21:06.243894+0000","last_fullsized":"2023-07-21T22:21:06.243894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:06.576092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.417884+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:20:38.417884+0000","last_peered":"2023-07-21T22:20:38.417884+0000","last_clean":"2023-07-21T22:20:38.417884+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:20:38.417884+0000","last_undegraded":"2023-07-21T22:20:38.417884+0000","last_fullsized":"2023-07-21T22:20:38.417884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:32:22.146374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.660121+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:20:38.660121+0000","last_peered":"2023-07-21T22:20:38.660121+0000","last_clean":"2023-07-21T22:20:38.660121+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:20:38.660121+0000","last_undegraded":"2023-07-21T22:20:38.660121+0000","last_fullsized":"2023-07-21T22:20:38.660121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:01:15.547748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.246264+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:21:06.246264+0000","last_peered":"2023-07-21T22:21:06.246264+0000","last_clean":"2023-07-21T22:21:06.246264+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:21:06.246264+0000","last_undegraded":"2023-07-21T22:21:06.246264+0000","last_fullsized":"2023-07-21T22:21:06.246264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:37.797343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418128+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:20:38.418128+0000","last_peered":"2023-07-21T22:20:38.418128+0000","last_clean":"2023-07-21T22:20:38.418128+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:20:38.418128+0000","last_undegraded":"2023-07-21T22:20:38.418128+0000","last_fullsized":"2023-07-21T22:20:38.418128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:12.532381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658825+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:20:38.658825+0000","last_peered":"2023-07-21T22:20:38.658825+0000","last_clean":"2023-07-21T22:20:38.658825+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:20:38.658825+0000","last_undegraded":"2023-07-21T22:20:38.658825+0000","last_fullsized":"2023-07-21T22:20:38.658825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:42:37.381662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243097+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:21:06.243097+0000","last_peered":"2023-07-21T22:21:06.243097+0000","last_clean":"2023-07-21T22:21:06.243097+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:21:06.243097+0000","last_undegraded":"2023-07-21T22:21:06.243097+0000","last_fullsized":"2023-07-21T22:21:06.243097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:35:44.049591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418190+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:20:38.418190+0000","last_peered":"2023-07-21T22:20:38.418190+0000","last_clean":"2023-07-21T22:20:38.418190+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:20:38.418190+0000","last_undegraded":"2023-07-21T22:20:38.418190+0000","last_fullsized":"2023-07-21T22:20:38.418190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:53:46.778497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658900+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:20:38.658900+0000","last_peered":"2023-07-21T22:20:38.658900+0000","last_clean":"2023-07-21T22:20:38.658900+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:20:38.658900+0000","last_undegraded":"2023-07-21T22:20:38.658900+0000","last_fullsized":"2023-07-21T22:20:38.658900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:41:42.378836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229401+0000","last_change":"2023-07-21T22:20:38.439768+0000","last_active":"2023-07-21T22:21:06.229401+0000","last_peered":"2023-07-21T22:21:06.229401+0000","last_clean":"2023-07-21T22:21:06.229401+0000","last_became_active":"2023-07-21T22:20:38.439322+0000","last_became_peered":"2023-07-21T22:20:38.439322+0000","last_unstale":"2023-07-21T22:21:06.229401+0000","last_undegraded":"2023-07-21T22:21:06.229401+0000","last_fullsized":"2023-07-21T22:21:06.229401+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:07.041475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305701+0000","last_change":"2023-07-21T22:19:37.404292+0000","last_active":"2023-07-21T22:20:38.305701+0000","last_peered":"2023-07-21T22:20:38.305701+0000","last_clean":"2023-07-21T22:20:38.305701+0000","last_became_active":"2023-07-21T22:19:37.403826+0000","last_became_peered":"2023-07-21T22:19:37.403826+0000","last_unstale":"2023-07-21T22:20:38.305701+0000","last_undegraded":"2023-07-21T22:20:38.305701+0000","last_fullsized":"2023-07-21T22:20:38.305701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:58.935037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":141,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:10.459881+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:21:10.459881+0000","last_peered":"2023-07-21T22:21:10.459881+0000","last_clean":"2023-07-21T22:21:10.459881+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:21:10.459881+0000","last_undegraded":"2023-07-21T22:21:10.459881+0000","last_fullsized":"2023-07-21T22:21:10.459881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:08.289972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.235446+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:21:06.235446+0000","last_peered":"2023-07-21T22:21:06.235446+0000","last_clean":"2023-07-21T22:21:06.235446+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:21:06.235446+0000","last_undegraded":"2023-07-21T22:21:06.235446+0000","last_fullsized":"2023-07-21T22:21:06.235446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:46:49.373988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305565+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:20:38.305565+0000","last_peered":"2023-07-21T22:20:38.305565+0000","last_clean":"2023-07-21T22:20:38.305565+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:20:38.305565+0000","last_undegraded":"2023-07-21T22:20:38.305565+0000","last_fullsized":"2023-07-21T22:20:38.305565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:38:55.810627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298031+0000","last_change":"2023-07-21T22:19:37.328942+0000","last_active":"2023-07-21T22:20:38.298031+0000","last_peered":"2023-07-21T22:20:38.298031+0000","last_clean":"2023-07-21T22:20:38.298031+0000","last_became_active":"2023-07-21T22:19:37.325738+0000","last_became_peered":"2023-07-21T22:19:37.325738+0000","last_unstale":"2023-07-21T22:20:38.298031+0000","last_undegraded":"2023-07-21T22:20:38.298031+0000","last_fullsized":"2023-07-21T22:20:38.298031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:18:17.573518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230305+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:21:06.230305+0000","last_peered":"2023-07-21T22:21:06.230305+0000","last_clean":"2023-07-21T22:21:06.230305+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:21:06.230305+0000","last_undegraded":"2023-07-21T22:21:06.230305+0000","last_fullsized":"2023-07-21T22:21:06.230305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:08:02.749178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.299090+0000","last_change":"2023-07-21T22:19:37.326119+0000","last_active":"2023-07-21T22:20:38.299090+0000","last_peered":"2023-07-21T22:20:38.299090+0000","last_clean":"2023-07-21T22:20:38.299090+0000","last_became_active":"2023-07-21T22:19:37.325675+0000","last_became_peered":"2023-07-21T22:19:37.325675+0000","last_unstale":"2023-07-21T22:20:38.299090+0000","last_undegraded":"2023-07-21T22:20:38.299090+0000","last_fullsized":"2023-07-21T22:20:38.299090+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:44:14.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295247+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:20:38.295247+0000","last_peered":"2023-07-21T22:20:38.295247+0000","last_clean":"2023-07-21T22:20:38.295247+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:20:38.295247+0000","last_undegraded":"2023-07-21T22:20:38.295247+0000","last_fullsized":"2023-07-21T22:20:38.295247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:41:42.444573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.242024+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:21:06.242024+0000","last_peered":"2023-07-21T22:21:06.242024+0000","last_clean":"2023-07-21T22:21:06.242024+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:21:06.242024+0000","last_undegraded":"2023-07-21T22:21:06.242024+0000","last_fullsized":"2023-07-21T22:21:06.242024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:32:03.234308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.660186+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:20:38.660186+0000","last_peered":"2023-07-21T22:20:38.660186+0000","last_clean":"2023-07-21T22:20:38.660186+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:20:38.660186+0000","last_undegraded":"2023-07-21T22:20:38.660186+0000","last_fullsized":"2023-07-21T22:20:38.660186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:23:22.856670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296564+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:20:38.296564+0000","last_peered":"2023-07-21T22:20:38.296564+0000","last_clean":"2023-07-21T22:20:38.296564+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:20:38.296564+0000","last_undegraded":"2023-07-21T22:20:38.296564+0000","last_fullsized":"2023-07-21T22:20:38.296564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:13:30.074340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233367+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:21:06.233367+0000","last_peered":"2023-07-21T22:21:06.233367+0000","last_clean":"2023-07-21T22:21:06.233367+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:21:06.233367+0000","last_undegraded":"2023-07-21T22:21:06.233367+0000","last_fullsized":"2023-07-21T22:21:06.233367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:03.385071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659286+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:20:38.659286+0000","last_peered":"2023-07-21T22:20:38.659286+0000","last_clean":"2023-07-21T22:20:38.659286+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:20:38.659286+0000","last_undegraded":"2023-07-21T22:20:38.659286+0000","last_fullsized":"2023-07-21T22:20:38.659286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:37.149270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":133,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:11.460791+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:21:11.460791+0000","last_peered":"2023-07-21T22:21:11.460791+0000","last_clean":"2023-07-21T22:21:11.460791+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:21:11.460791+0000","last_undegraded":"2023-07-21T22:21:11.460791+0000","last_fullsized":"2023-07-21T22:21:11.460791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:18:00.045946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.235609+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:21:06.235609+0000","last_peered":"2023-07-21T22:21:06.235609+0000","last_clean":"2023-07-21T22:21:06.235609+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:21:06.235609+0000","last_undegraded":"2023-07-21T22:21:06.235609+0000","last_fullsized":"2023-07-21T22:21:06.235609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:24:37.041577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297787+0000","last_change":"2023-07-21T22:19:37.326317+0000","last_active":"2023-07-21T22:20:38.297787+0000","last_peered":"2023-07-21T22:20:38.297787+0000","last_clean":"2023-07-21T22:20:38.297787+0000","last_became_active":"2023-07-21T22:19:37.325791+0000","last_became_peered":"2023-07-21T22:19:37.325791+0000","last_unstale":"2023-07-21T22:20:38.297787+0000","last_undegraded":"2023-07-21T22:20:38.297787+0000","last_fullsized":"2023-07-21T22:20:38.297787+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:28.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659430+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:20:38.659430+0000","last_peered":"2023-07-21T22:20:38.659430+0000","last_clean":"2023-07-21T22:20:38.659430+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:20:38.659430+0000","last_undegraded":"2023-07-21T22:20:38.659430+0000","last_fullsized":"2023-07-21T22:20:38.659430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:08.684209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.246230+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:21:06.246230+0000","last_peered":"2023-07-21T22:21:06.246230+0000","last_clean":"2023-07-21T22:21:06.246230+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:21:06.246230+0000","last_undegraded":"2023-07-21T22:21:06.246230+0000","last_fullsized":"2023-07-21T22:21:06.246230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:48.990870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418435+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:20:38.418435+0000","last_peered":"2023-07-21T22:20:38.418435+0000","last_clean":"2023-07-21T22:20:38.418435+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:20:38.418435+0000","last_undegraded":"2023-07-21T22:20:38.418435+0000","last_fullsized":"2023-07-21T22:20:38.418435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:20:47.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296928+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:20:38.296928+0000","last_peered":"2023-07-21T22:20:38.296928+0000","last_clean":"2023-07-21T22:20:38.296928+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:20:38.296928+0000","last_undegraded":"2023-07-21T22:20:38.296928+0000","last_fullsized":"2023-07-21T22:20:38.296928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:48.614307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243632+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:21:06.243632+0000","last_peered":"2023-07-21T22:21:06.243632+0000","last_clean":"2023-07-21T22:21:06.243632+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:21:06.243632+0000","last_undegraded":"2023-07-21T22:21:06.243632+0000","last_fullsized":"2023-07-21T22:21:06.243632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:59.106308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297543+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:20:38.297543+0000","last_peered":"2023-07-21T22:20:38.297543+0000","last_clean":"2023-07-21T22:20:38.297543+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:20:38.297543+0000","last_undegraded":"2023-07-21T22:20:38.297543+0000","last_fullsized":"2023-07-21T22:20:38.297543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:44.231916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":273,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:12.463095+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:21:12.463095+0000","last_peered":"2023-07-21T22:21:12.463095+0000","last_clean":"2023-07-21T22:21:12.463095+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:21:12.463095+0000","last_undegraded":"2023-07-21T22:21:12.463095+0000","last_fullsized":"2023-07-21T22:21:12.463095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:30.817175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63587,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":166,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.228970+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:21:06.228970+0000","last_peered":"2023-07-21T22:21:06.228970+0000","last_clean":"2023-07-21T22:21:06.228970+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:21:06.228970+0000","last_undegraded":"2023-07-21T22:21:06.228970+0000","last_fullsized":"2023-07-21T22:21:06.228970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:04:11.923390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297581+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:20:38.297581+0000","last_peered":"2023-07-21T22:20:38.297581+0000","last_clean":"2023-07-21T22:20:38.297581+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:20:38.297581+0000","last_undegraded":"2023-07-21T22:20:38.297581+0000","last_fullsized":"2023-07-21T22:20:38.297581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:24:31.302884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305466+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:20:38.305466+0000","last_peered":"2023-07-21T22:20:38.305466+0000","last_clean":"2023-07-21T22:20:38.305466+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:20:38.305466+0000","last_undegraded":"2023-07-21T22:20:38.305466+0000","last_fullsized":"2023-07-21T22:20:38.305466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:21.244524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245019+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:21:06.245019+0000","last_peered":"2023-07-21T22:21:06.245019+0000","last_clean":"2023-07-21T22:21:06.245019+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:21:06.245019+0000","last_undegraded":"2023-07-21T22:21:06.245019+0000","last_fullsized":"2023-07-21T22:21:06.245019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:37.737964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298975+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:20:38.298975+0000","last_peered":"2023-07-21T22:20:38.298975+0000","last_clean":"2023-07-21T22:20:38.298975+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:20:38.298975+0000","last_undegraded":"2023-07-21T22:20:38.298975+0000","last_fullsized":"2023-07-21T22:20:38.298975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:10:07.260019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296895+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:20:38.296895+0000","last_peered":"2023-07-21T22:20:38.296895+0000","last_clean":"2023-07-21T22:20:38.296895+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:20:38.296895+0000","last_undegraded":"2023-07-21T22:20:38.296895+0000","last_fullsized":"2023-07-21T22:20:38.296895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:20:47.729624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232411+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:21:06.232411+0000","last_peered":"2023-07-21T22:21:06.232411+0000","last_clean":"2023-07-21T22:21:06.232411+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:21:06.232411+0000","last_undegraded":"2023-07-21T22:21:06.232411+0000","last_fullsized":"2023-07-21T22:21:06.232411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:09.067826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658927+0000","last_change":"2023-07-21T22:19:37.403648+0000","last_active":"2023-07-21T22:20:38.658927+0000","last_peered":"2023-07-21T22:20:38.658927+0000","last_clean":"2023-07-21T22:20:38.658927+0000","last_became_active":"2023-07-21T22:19:37.403252+0000","last_became_peered":"2023-07-21T22:19:37.403252+0000","last_unstale":"2023-07-21T22:20:38.658927+0000","last_undegraded":"2023-07-21T22:20:38.658927+0000","last_fullsized":"2023-07-21T22:20:38.658927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:55.264540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:09.474149+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:21:09.474149+0000","last_peered":"2023-07-21T22:21:09.474149+0000","last_clean":"2023-07-21T22:21:09.474149+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:21:09.474149+0000","last_undegraded":"2023-07-21T22:21:09.474149+0000","last_fullsized":"2023-07-21T22:21:09.474149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:34:54.366109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.221294+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:21:06.221294+0000","last_peered":"2023-07-21T22:21:06.221294+0000","last_clean":"2023-07-21T22:21:06.221294+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:21:06.221294+0000","last_undegraded":"2023-07-21T22:21:06.221294+0000","last_fullsized":"2023-07-21T22:21:06.221294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:04.512567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295715+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:20:38.295715+0000","last_peered":"2023-07-21T22:20:38.295715+0000","last_clean":"2023-07-21T22:20:38.295715+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:20:38.295715+0000","last_undegraded":"2023-07-21T22:20:38.295715+0000","last_fullsized":"2023-07-21T22:20:38.295715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:17:47.809370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658510+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:20:38.658510+0000","last_peered":"2023-07-21T22:20:38.658510+0000","last_clean":"2023-07-21T22:20:38.658510+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:20:38.658510+0000","last_undegraded":"2023-07-21T22:20:38.658510+0000","last_fullsized":"2023-07-21T22:20:38.658510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:20:14.245284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230387+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:21:06.230387+0000","last_peered":"2023-07-21T22:21:06.230387+0000","last_clean":"2023-07-21T22:21:06.230387+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:21:06.230387+0000","last_undegraded":"2023-07-21T22:21:06.230387+0000","last_fullsized":"2023-07-21T22:21:06.230387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:31:10.400632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296528+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:20:38.296528+0000","last_peered":"2023-07-21T22:20:38.296528+0000","last_clean":"2023-07-21T22:20:38.296528+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:20:38.296528+0000","last_undegraded":"2023-07-21T22:20:38.296528+0000","last_fullsized":"2023-07-21T22:20:38.296528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:16.962575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659163+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:20:38.659163+0000","last_peered":"2023-07-21T22:20:38.659163+0000","last_clean":"2023-07-21T22:20:38.659163+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:20:38.659163+0000","last_undegraded":"2023-07-21T22:20:38.659163+0000","last_fullsized":"2023-07-21T22:20:38.659163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:11:28.407010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233495+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:21:06.233495+0000","last_peered":"2023-07-21T22:21:06.233495+0000","last_clean":"2023-07-21T22:21:06.233495+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:21:06.233495+0000","last_undegraded":"2023-07-21T22:21:06.233495+0000","last_fullsized":"2023-07-21T22:21:06.233495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:30.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658592+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:20:38.658592+0000","last_peered":"2023-07-21T22:20:38.658592+0000","last_clean":"2023-07-21T22:20:38.658592+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:20:38.658592+0000","last_undegraded":"2023-07-21T22:20:38.658592+0000","last_fullsized":"2023-07-21T22:20:38.658592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:55:48.187244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297860+0000","last_change":"2023-07-21T22:19:37.312431+0000","last_active":"2023-07-21T22:20:38.297860+0000","last_peered":"2023-07-21T22:20:38.297860+0000","last_clean":"2023-07-21T22:20:38.297860+0000","last_became_active":"2023-07-21T22:19:37.312038+0000","last_became_peered":"2023-07-21T22:19:37.312038+0000","last_unstale":"2023-07-21T22:20:38.297860+0000","last_undegraded":"2023-07-21T22:20:38.297860+0000","last_fullsized":"2023-07-21T22:20:38.297860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:13:00.069534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244472+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:21:06.244472+0000","last_peered":"2023-07-21T22:21:06.244472+0000","last_clean":"2023-07-21T22:21:06.244472+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:21:06.244472+0000","last_undegraded":"2023-07-21T22:21:06.244472+0000","last_fullsized":"2023-07-21T22:21:06.244472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:49.866646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295482+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:20:38.295482+0000","last_peered":"2023-07-21T22:20:38.295482+0000","last_clean":"2023-07-21T22:20:38.295482+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:20:38.295482+0000","last_undegraded":"2023-07-21T22:20:38.295482+0000","last_fullsized":"2023-07-21T22:20:38.295482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:51.240441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659099+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:20:38.659099+0000","last_peered":"2023-07-21T22:20:38.659099+0000","last_clean":"2023-07-21T22:20:38.659099+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:20:38.659099+0000","last_undegraded":"2023-07-21T22:20:38.659099+0000","last_fullsized":"2023-07-21T22:20:38.659099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:34:49.901496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237830+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:21:06.237830+0000","last_peered":"2023-07-21T22:21:06.237830+0000","last_clean":"2023-07-21T22:21:06.237830+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:21:06.237830+0000","last_undegraded":"2023-07-21T22:21:06.237830+0000","last_fullsized":"2023-07-21T22:21:06.237830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:47.649059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295392+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:20:38.295392+0000","last_peered":"2023-07-21T22:20:38.295392+0000","last_clean":"2023-07-21T22:20:38.295392+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:20:38.295392+0000","last_undegraded":"2023-07-21T22:20:38.295392+0000","last_fullsized":"2023-07-21T22:20:38.295392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:53:12.871041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230716+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:21:06.230716+0000","last_peered":"2023-07-21T22:21:06.230716+0000","last_clean":"2023-07-21T22:21:06.230716+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:21:06.230716+0000","last_undegraded":"2023-07-21T22:21:06.230716+0000","last_fullsized":"2023-07-21T22:21:06.230716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:07:20.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:12.478815+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:21:12.478815+0000","last_peered":"2023-07-21T22:21:12.478815+0000","last_clean":"2023-07-21T22:21:12.478815+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:21:12.478815+0000","last_undegraded":"2023-07-21T22:21:12.478815+0000","last_fullsized":"2023-07-21T22:21:12.478815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:09:04.461436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659421+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:20:38.659421+0000","last_peered":"2023-07-21T22:20:38.659421+0000","last_clean":"2023-07-21T22:20:38.659421+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:20:38.659421+0000","last_undegraded":"2023-07-21T22:20:38.659421+0000","last_fullsized":"2023-07-21T22:20:38.659421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:31.504021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243705+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:21:06.243705+0000","last_peered":"2023-07-21T22:21:06.243705+0000","last_clean":"2023-07-21T22:21:06.243705+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:21:06.243705+0000","last_undegraded":"2023-07-21T22:21:06.243705+0000","last_fullsized":"2023-07-21T22:21:06.243705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:21:28.483320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418342+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:20:38.418342+0000","last_peered":"2023-07-21T22:20:38.418342+0000","last_clean":"2023-07-21T22:20:38.418342+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:20:38.418342+0000","last_undegraded":"2023-07-21T22:20:38.418342+0000","last_fullsized":"2023-07-21T22:20:38.418342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:03:40.298453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658878+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:20:38.658878+0000","last_peered":"2023-07-21T22:20:38.658878+0000","last_clean":"2023-07-21T22:20:38.658878+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:20:38.658878+0000","last_undegraded":"2023-07-21T22:20:38.658878+0000","last_fullsized":"2023-07-21T22:20:38.658878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:55:41.830722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297102+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:20:38.297102+0000","last_peered":"2023-07-21T22:20:38.297102+0000","last_clean":"2023-07-21T22:20:38.297102+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:20:38.297102+0000","last_undegraded":"2023-07-21T22:20:38.297102+0000","last_fullsized":"2023-07-21T22:20:38.297102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:48.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237467+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:21:06.237467+0000","last_peered":"2023-07-21T22:21:06.237467+0000","last_clean":"2023-07-21T22:21:06.237467+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:21:06.237467+0000","last_undegraded":"2023-07-21T22:21:06.237467+0000","last_fullsized":"2023-07-21T22:21:06.237467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:00:22.036485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658972+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:20:38.658972+0000","last_peered":"2023-07-21T22:20:38.658972+0000","last_clean":"2023-07-21T22:20:38.658972+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:20:38.658972+0000","last_undegraded":"2023-07-21T22:20:38.658972+0000","last_fullsized":"2023-07-21T22:20:38.658972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:30.610746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298509+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:20:38.298509+0000","last_peered":"2023-07-21T22:20:38.298509+0000","last_clean":"2023-07-21T22:20:38.298509+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:20:38.298509+0000","last_undegraded":"2023-07-21T22:20:38.298509+0000","last_fullsized":"2023-07-21T22:20:38.298509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:03.929894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658954+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:20:38.658954+0000","last_peered":"2023-07-21T22:20:38.658954+0000","last_clean":"2023-07-21T22:20:38.658954+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:20:38.658954+0000","last_undegraded":"2023-07-21T22:20:38.658954+0000","last_fullsized":"2023-07-21T22:20:38.658954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:49.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240119+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:21:06.240119+0000","last_peered":"2023-07-21T22:21:06.240119+0000","last_clean":"2023-07-21T22:21:06.240119+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:21:06.240119+0000","last_undegraded":"2023-07-21T22:21:06.240119+0000","last_fullsized":"2023-07-21T22:21:06.240119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:33.312382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659732+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:20:38.659732+0000","last_peered":"2023-07-21T22:20:38.659732+0000","last_clean":"2023-07-21T22:20:38.659732+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:20:38.659732+0000","last_undegraded":"2023-07-21T22:20:38.659732+0000","last_fullsized":"2023-07-21T22:20:38.659732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:39.197537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298123+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:20:38.298123+0000","last_peered":"2023-07-21T22:20:38.298123+0000","last_clean":"2023-07-21T22:20:38.298123+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:20:38.298123+0000","last_undegraded":"2023-07-21T22:20:38.298123+0000","last_fullsized":"2023-07-21T22:20:38.298123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:05.244612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243542+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:21:06.243542+0000","last_peered":"2023-07-21T22:21:06.243542+0000","last_clean":"2023-07-21T22:21:06.243542+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:21:06.243542+0000","last_undegraded":"2023-07-21T22:21:06.243542+0000","last_fullsized":"2023-07-21T22:21:06.243542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:14:42.196678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305178+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:20:38.305178+0000","last_peered":"2023-07-21T22:20:38.305178+0000","last_clean":"2023-07-21T22:20:38.305178+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:20:38.305178+0000","last_undegraded":"2023-07-21T22:20:38.305178+0000","last_fullsized":"2023-07-21T22:20:38.305178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:41:59.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418371+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:20:38.418371+0000","last_peered":"2023-07-21T22:20:38.418371+0000","last_clean":"2023-07-21T22:20:38.418371+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:20:38.418371+0000","last_undegraded":"2023-07-21T22:20:38.418371+0000","last_fullsized":"2023-07-21T22:20:38.418371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:32.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245652+0000","last_change":"2023-07-21T22:19:37.404445+0000","last_active":"2023-07-21T22:21:06.245652+0000","last_peered":"2023-07-21T22:21:06.245652+0000","last_clean":"2023-07-21T22:21:06.245652+0000","last_became_active":"2023-07-21T22:19:37.404011+0000","last_became_peered":"2023-07-21T22:19:37.404011+0000","last_unstale":"2023-07-21T22:21:06.245652+0000","last_undegraded":"2023-07-21T22:21:06.245652+0000","last_fullsized":"2023-07-21T22:21:06.245652+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:04.926506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418255+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:20:38.418255+0000","last_peered":"2023-07-21T22:20:38.418255+0000","last_clean":"2023-07-21T22:20:38.418255+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:20:38.418255+0000","last_undegraded":"2023-07-21T22:20:38.418255+0000","last_fullsized":"2023-07-21T22:20:38.418255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:26:43.576746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.438152+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:20:38.438152+0000","last_peered":"2023-07-21T22:20:38.438152+0000","last_clean":"2023-07-21T22:20:38.438152+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:20:38.438152+0000","last_undegraded":"2023-07-21T22:20:38.438152+0000","last_fullsized":"2023-07-21T22:20:38.438152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:51:01.655488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.204620+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:21:06.204620+0000","last_peered":"2023-07-21T22:21:06.204620+0000","last_clean":"2023-07-21T22:21:06.204620+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:21:06.204620+0000","last_undegraded":"2023-07-21T22:21:06.204620+0000","last_fullsized":"2023-07-21T22:21:06.204620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:36:08.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298007+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:20:38.298007+0000","last_peered":"2023-07-21T22:20:38.298007+0000","last_clean":"2023-07-21T22:20:38.298007+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:20:38.298007+0000","last_undegraded":"2023-07-21T22:20:38.298007+0000","last_fullsized":"2023-07-21T22:20:38.298007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:54.100394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659458+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:20:38.659458+0000","last_peered":"2023-07-21T22:20:38.659458+0000","last_clean":"2023-07-21T22:20:38.659458+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:20:38.659458+0000","last_undegraded":"2023-07-21T22:20:38.659458+0000","last_fullsized":"2023-07-21T22:20:38.659458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:11:22.743920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244209+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:21:06.244209+0000","last_peered":"2023-07-21T22:21:06.244209+0000","last_clean":"2023-07-21T22:21:06.244209+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:21:06.244209+0000","last_undegraded":"2023-07-21T22:21:06.244209+0000","last_fullsized":"2023-07-21T22:21:06.244209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:10:19.440093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418254+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:20:38.418254+0000","last_peered":"2023-07-21T22:20:38.418254+0000","last_clean":"2023-07-21T22:20:38.418254+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:20:38.418254+0000","last_undegraded":"2023-07-21T22:20:38.418254+0000","last_fullsized":"2023-07-21T22:20:38.418254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:52.621668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298467+0000","last_change":"2023-07-21T22:19:37.315809+0000","last_active":"2023-07-21T22:20:38.298467+0000","last_peered":"2023-07-21T22:20:38.298467+0000","last_clean":"2023-07-21T22:20:38.298467+0000","last_became_active":"2023-07-21T22:19:37.312331+0000","last_became_peered":"2023-07-21T22:19:37.312331+0000","last_unstale":"2023-07-21T22:20:38.298467+0000","last_undegraded":"2023-07-21T22:20:38.298467+0000","last_fullsized":"2023-07-21T22:20:38.298467+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:01:40.724543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243178+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:21:06.243178+0000","last_peered":"2023-07-21T22:21:06.243178+0000","last_clean":"2023-07-21T22:21:06.243178+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:21:06.243178+0000","last_undegraded":"2023-07-21T22:21:06.243178+0000","last_fullsized":"2023-07-21T22:21:06.243178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:55.735567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418468+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:20:38.418468+0000","last_peered":"2023-07-21T22:20:38.418468+0000","last_clean":"2023-07-21T22:20:38.418468+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:20:38.418468+0000","last_undegraded":"2023-07-21T22:20:38.418468+0000","last_fullsized":"2023-07-21T22:20:38.418468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:59.817080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658662+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:20:38.658662+0000","last_peered":"2023-07-21T22:20:38.658662+0000","last_clean":"2023-07-21T22:20:38.658662+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:20:38.658662+0000","last_undegraded":"2023-07-21T22:20:38.658662+0000","last_fullsized":"2023-07-21T22:20:38.658662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:59:29.846402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229474+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:21:06.229474+0000","last_peered":"2023-07-21T22:21:06.229474+0000","last_clean":"2023-07-21T22:21:06.229474+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:21:06.229474+0000","last_undegraded":"2023-07-21T22:21:06.229474+0000","last_fullsized":"2023-07-21T22:21:06.229474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:37.964833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.438255+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:20:38.438255+0000","last_peered":"2023-07-21T22:20:38.438255+0000","last_clean":"2023-07-21T22:20:38.438255+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:20:38.438255+0000","last_undegraded":"2023-07-21T22:20:38.438255+0000","last_fullsized":"2023-07-21T22:20:38.438255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:30.756639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305874+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:20:38.305874+0000","last_peered":"2023-07-21T22:20:38.305874+0000","last_clean":"2023-07-21T22:20:38.305874+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:20:38.305874+0000","last_undegraded":"2023-07-21T22:20:38.305874+0000","last_fullsized":"2023-07-21T22:20:38.305874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:39:12.174231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230951+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:21:06.230951+0000","last_peered":"2023-07-21T22:21:06.230951+0000","last_clean":"2023-07-21T22:21:06.230951+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:21:06.230951+0000","last_undegraded":"2023-07-21T22:21:06.230951+0000","last_fullsized":"2023-07-21T22:21:06.230951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:11:25.905492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659463+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:20:38.659463+0000","last_peered":"2023-07-21T22:20:38.659463+0000","last_clean":"2023-07-21T22:20:38.659463+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:20:38.659463+0000","last_undegraded":"2023-07-21T22:20:38.659463+0000","last_fullsized":"2023-07-21T22:20:38.659463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:09:28.212992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297292+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:20:38.297292+0000","last_peered":"2023-07-21T22:20:38.297292+0000","last_clean":"2023-07-21T22:20:38.297292+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:20:38.297292+0000","last_undegraded":"2023-07-21T22:20:38.297292+0000","last_fullsized":"2023-07-21T22:20:38.297292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:43:21.204017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.226015+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:21:06.226015+0000","last_peered":"2023-07-21T22:21:06.226015+0000","last_clean":"2023-07-21T22:21:06.226015+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:21:06.226015+0000","last_undegraded":"2023-07-21T22:21:06.226015+0000","last_fullsized":"2023-07-21T22:21:06.226015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:16.718601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296263+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:20:38.296263+0000","last_peered":"2023-07-21T22:20:38.296263+0000","last_clean":"2023-07-21T22:20:38.296263+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:20:38.296263+0000","last_undegraded":"2023-07-21T22:20:38.296263+0000","last_fullsized":"2023-07-21T22:20:38.296263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:36.550687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295758+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:20:38.295758+0000","last_peered":"2023-07-21T22:20:38.295758+0000","last_clean":"2023-07-21T22:20:38.295758+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:20:38.295758+0000","last_undegraded":"2023-07-21T22:20:38.295758+0000","last_fullsized":"2023-07-21T22:20:38.295758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:13.984980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.227990+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:21:06.227990+0000","last_peered":"2023-07-21T22:21:06.227990+0000","last_clean":"2023-07-21T22:21:06.227990+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:21:06.227990+0000","last_undegraded":"2023-07-21T22:21:06.227990+0000","last_fullsized":"2023-07-21T22:21:06.227990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:11.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237254+0000","last_change":"2023-07-21T22:20:38.306667+0000","last_active":"2023-07-21T22:21:06.237254+0000","last_peered":"2023-07-21T22:21:06.237254+0000","last_clean":"2023-07-21T22:21:06.237254+0000","last_became_active":"2023-07-21T22:20:38.306285+0000","last_became_peered":"2023-07-21T22:20:38.306285+0000","last_unstale":"2023-07-21T22:21:06.237254+0000","last_undegraded":"2023-07-21T22:21:06.237254+0000","last_fullsized":"2023-07-21T22:21:06.237254+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:32:37.690707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236563+0000","last_change":"2023-07-21T22:19:37.326004+0000","last_active":"2023-07-21T22:21:06.236563+0000","last_peered":"2023-07-21T22:21:06.236563+0000","last_clean":"2023-07-21T22:21:06.236563+0000","last_became_active":"2023-07-21T22:19:37.325611+0000","last_became_peered":"2023-07-21T22:19:37.325611+0000","last_unstale":"2023-07-21T22:21:06.236563+0000","last_undegraded":"2023-07-21T22:21:06.236563+0000","last_fullsized":"2023-07-21T22:21:06.236563+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:02.980321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.212111+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:21:06.212111+0000","last_peered":"2023-07-21T22:21:06.212111+0000","last_clean":"2023-07-21T22:21:06.212111+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:21:06.212111+0000","last_undegraded":"2023-07-21T22:21:06.212111+0000","last_fullsized":"2023-07-21T22:21:06.212111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:22:52.562457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243872+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:21:06.243872+0000","last_peered":"2023-07-21T22:21:06.243872+0000","last_clean":"2023-07-21T22:21:06.243872+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:21:06.243872+0000","last_undegraded":"2023-07-21T22:21:06.243872+0000","last_fullsized":"2023-07-21T22:21:06.243872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:39.140948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240603+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:21:06.240603+0000","last_peered":"2023-07-21T22:21:06.240603+0000","last_clean":"2023-07-21T22:21:06.240603+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:21:06.240603+0000","last_undegraded":"2023-07-21T22:21:06.240603+0000","last_fullsized":"2023-07-21T22:21:06.240603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:52.883722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.226450+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:21:06.226450+0000","last_peered":"2023-07-21T22:21:06.226450+0000","last_clean":"2023-07-21T22:21:06.226450+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:21:06.226450+0000","last_undegraded":"2023-07-21T22:21:06.226450+0000","last_fullsized":"2023-07-21T22:21:06.226450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:44:28.022785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236553+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:21:06.236553+0000","last_peered":"2023-07-21T22:21:06.236553+0000","last_clean":"2023-07-21T22:21:06.236553+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:21:06.236553+0000","last_undegraded":"2023-07-21T22:21:06.236553+0000","last_fullsized":"2023-07-21T22:21:06.236553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:57.476981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.228554+0000","last_change":"2023-07-21T22:19:37.314152+0000","last_active":"2023-07-21T22:21:06.228554+0000","last_peered":"2023-07-21T22:21:06.228554+0000","last_clean":"2023-07-21T22:21:06.228554+0000","last_became_active":"2023-07-21T22:19:37.313795+0000","last_became_peered":"2023-07-21T22:19:37.313795+0000","last_unstale":"2023-07-21T22:21:06.228554+0000","last_undegraded":"2023-07-21T22:21:06.228554+0000","last_fullsized":"2023-07-21T22:21:06.228554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:52:37.184312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233179+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:21:06.233179+0000","last_peered":"2023-07-21T22:21:06.233179+0000","last_clean":"2023-07-21T22:21:06.233179+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:21:06.233179+0000","last_undegraded":"2023-07-21T22:21:06.233179+0000","last_fullsized":"2023-07-21T22:21:06.233179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:23.465390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232972+0000","last_change":"2023-07-21T22:20:38.307235+0000","last_active":"2023-07-21T22:21:06.232972+0000","last_peered":"2023-07-21T22:21:06.232972+0000","last_clean":"2023-07-21T22:21:06.232972+0000","last_became_active":"2023-07-21T22:20:38.306829+0000","last_became_peered":"2023-07-21T22:20:38.306829+0000","last_unstale":"2023-07-21T22:21:06.232972+0000","last_undegraded":"2023-07-21T22:21:06.232972+0000","last_fullsized":"2023-07-21T22:21:06.232972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:18:35.611527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245317+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:21:06.245317+0000","last_peered":"2023-07-21T22:21:06.245317+0000","last_clean":"2023-07-21T22:21:06.245317+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:21:06.245317+0000","last_undegraded":"2023-07-21T22:21:06.245317+0000","last_fullsized":"2023-07-21T22:21:06.245317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:17.933758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.242741+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:21:06.242741+0000","last_peered":"2023-07-21T22:21:06.242741+0000","last_clean":"2023-07-21T22:21:06.242741+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:21:06.242741+0000","last_undegraded":"2023-07-21T22:21:06.242741+0000","last_fullsized":"2023-07-21T22:21:06.242741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:42:34.683114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233743+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:21:06.233743+0000","last_peered":"2023-07-21T22:21:06.233743+0000","last_clean":"2023-07-21T22:21:06.233743+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:21:06.233743+0000","last_undegraded":"2023-07-21T22:21:06.233743+0000","last_fullsized":"2023-07-21T22:21:06.233743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:02:26.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68479,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":624,"num_read_kb":387,"num_write":165,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":515866,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89000000000000001,"5min":1.21,"15min":1.21},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90200000000000002,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81599999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.48199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.879,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69399999999999995,"15min":0.68999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.99299999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93500000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.754}]},{"osd":7,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95799999999999996,"5min":1.294,"15min":1.294},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76700000000000002,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.01,"5min":1.024,"15min":1.024},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97699999999999998,"5min":1.431,"15min":1.431},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.64400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.462,"15min":1.462},"last":0.83999999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.79900000000000004},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.622,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97599999999999998,"5min":1.5,"15min":1.5},"last":0.82499999999999996}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1163264,"data_stored":1167204,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":1.143,"15min":1.2370000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.754,"5min":17.132000000000001,"15min":17.132000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.1439999999999999,"15min":1.234},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":17.084,"15min":17.084},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.44700000000000001,"15min":0.46300000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56000000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61699999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78100000000000003,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.747,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80100000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.747}]},{"osd":6,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.363,"15min":1.363},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77400000000000002,"15min":0.80200000000000005},"min":{"1min":0.45400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84599999999999997,"5min":1.161,"15min":1.161},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86399999999999999,"5min":1.024,"15min":1.024},"last":0.88300000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91700000000000004,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.95899999999999996,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88400000000000001,"5min":17.163,"15min":17.163},"last":0.91000000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81200000000000006,"15min":0.82299999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99399999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.113,"15min":1.113},"last":0.95399999999999996}]}]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":520198,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.437},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85899999999999999,"15min":0.91700000000000004},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77800000000000002,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.81799999999999995},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.84499999999999997,"15min":0.89100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.48799999999999999,"5min":0.433,"15min":0.433},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.626},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":0.92900000000000005},"min":{"1min":0.57899999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.877,"15min":0.92500000000000004},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.94599999999999995},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.97499999999999998,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.006,"15min":1.077},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.585,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.754}]},{"osd":8,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.052,"15min":1.133},"min":{"1min":0.61599999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.611,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.069},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.9019999999999999,"5min":1.99,"15min":1.99},"last":0.79700000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.94199999999999995,"15min":0.96299999999999997},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.29,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.91800000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.0389999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":516022,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83999999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.629},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.753,"15min":0.745},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.747},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.59999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.627,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.80200000000000005,"15min":0.77000000000000002},"min":{"1min":0.73299999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.82799999999999996,"15min":0.79600000000000004},"min":{"1min":0.78500000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.97099999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.58399999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93400000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.79700000000000004,"15min":0.75700000000000001},"min":{"1min":0.66500000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.90400000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.879},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89600000000000002,"15min":0.89500000000000002},"min":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.94799999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":1112822,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.49099999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.36799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.503,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75900000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88700000000000001,"5min":1.399,"15min":1.399},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.498,"15min":0.51200000000000001},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.441}]},{"osd":5,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.5,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77400000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.57499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91700000000000004,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.57799999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.66100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.62,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96599999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.60199999999999998,"5min":0.499,"15min":0.499},"max":{"1min":0.95599999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.69899999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78900000000000003,"15min":0.78100000000000003},"min":{"1min":0.69099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77000000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":511278,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.365,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.498},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.76400000000000001,"15min":0.72699999999999998},"min":{"1min":0.63200000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.139,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73599999999999999,"15min":0.70499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.258,"5min":1.284,"15min":1.284},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.65300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.746},"min":{"1min":0.55500000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.80000000000000004},"min":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2170000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.871},{"interface":"front","average":{"1min":0.996,"5min":0.91900000000000004,"15min":0.88300000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.202,"5min":3.202,"15min":3.202},"last":1.0069999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.93200000000000005,"15min":0.93500000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1279999999999999,"5min":1.599,"15min":1.599},"last":1.026},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.66400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1799999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:20:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.0109999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.091},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.94099999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.84299999999999997,"15min":0.85599999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2629999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.018,"15min":1.036},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1120000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.048}]},{"osd":11,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.94199999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0369999999999999,"5min":1.585,"15min":1.585},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.0660000000000001,"15min":1.083},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.163,"5min":1.373,"15min":1.373},"last":1.069}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":574775,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.317,"5min":0.31,"15min":0.31},"max":{"1min":0.69199999999999995,"5min":0.95099999999999996,"15min":0.98799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.88300000000000001,"15min":1.1100000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60599999999999998,"15min":0.57399999999999995},"min":{"1min":0.51000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84199999999999997,"5min":0.871,"15min":1.0289999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58799999999999997,"15min":0.56699999999999995},"min":{"1min":0.47399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":0.93400000000000005},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.66400000000000003,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.0119999999999996,"5min":6.0119999999999996,"15min":6.0119999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.57799999999999996},"min":{"1min":0.45300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.46200000000000002},"min":{"1min":0.38600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56499999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69299999999999995,"15min":0.66400000000000003},"min":{"1min":0.64100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.66900000000000004},"min":{"1min":0.51100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0169999999999999,"5min":1.45,"15min":1.45},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58999999999999997,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85699999999999998,"5min":1.077,"15min":1.077},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.59399999999999997},"min":{"1min":0.56999999999999995,"5min":0.313,"15min":0.313},"max":{"1min":0.93600000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99199999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.94299999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.75,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80800000000000005},"min":{"1min":0.69799999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.755},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.753,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.98799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":0.93600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.877,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.93000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":512666,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.67900000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35999999999999999,"15min":0.35899999999999999},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73499999999999999,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.66500000000000004},"min":{"1min":0.52500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.56200000000000006,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.90000000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.2250000000000001,"5min":1.23,"15min":1.23},"last":0.37}]},{"osd":4,"last update":"Fri Jul 21 22:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.74399999999999999,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.72999999999999998,"15min":0.68000000000000005},"min":{"1min":0.60299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.149,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.73599999999999999},"min":{"1min":0.55400000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.246,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:20:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.78200000000000003,"15min":0.72499999999999998},"min":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68700000000000006,"15min":0.63700000000000001},"min":{"1min":0.58299999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.496}]},{"osd":7,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.97099999999999997,"15min":0.98899999999999999},"min":{"1min":0.65700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.226,"5min":0.88800000000000001,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.0640000000000001,"5min":6.0640000000000001,"15min":6.0640000000000001},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98399999999999999,"15min":0.99399999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4199999999999999,"5min":1.526,"15min":1.526},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.871,"15min":0.84899999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:20:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.86899999999999999,"15min":0.84599999999999997},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.71399999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.88}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1105920,"data_stored":576924,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.79300000000000004,"5min":1.9530000000000001,"15min":3.988},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.754,"5min":1.421,"15min":1.421},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.626,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.006,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.54500000000000004},"min":{"1min":0.51600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":2.044,"15min":2.044},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":1.984,"15min":1.984},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.45800000000000002},"min":{"1min":0.36899999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.82799999999999996,"5min":1.542,"15min":1.542},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.47299999999999998},"min":{"1min":0.39200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.84499999999999997,"5min":1.76,"15min":1.76},"last":0.755}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71399999999999997,"15min":0.68500000000000005},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.079,"5min":1.907,"15min":1.907},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72399999999999998,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.0489999999999999,"5min":1.484,"15min":1.484},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98199999999999998,"5min":1.609,"15min":1.609},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0620000000000001,"5min":1.859,"15min":1.859},"last":0.496}]},{"osd":9,"last update":"Fri Jul 21 22:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.117,"5min":2.016,"15min":2.016},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.69099999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.022,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.81699999999999995,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.149,"5min":1.887,"15min":1.887},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.82099999999999995},"min":{"1min":0.71099999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.101,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88500000000000001,"15min":0.88200000000000001},"min":{"1min":0.71899999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":511472,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.501,"15min":0.41299999999999998},"min":{"1min":0.39100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.42299999999999999},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.63800000000000001,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.67200000000000004,"15min":0.57599999999999996},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.734,"5min":2.734,"15min":2.734},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67000000000000004,"15min":0.60999999999999999},"min":{"1min":0.55200000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86499999999999999,"5min":1.139,"15min":1.139},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.177,"5min":0.76900000000000002,"15min":0.65900000000000003},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.0199999999999996,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.68500000000000005,"15min":0.61199999999999999},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.746,"15min":0.64600000000000002},"min":{"1min":0.57699999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69999999999999996,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":1.002,"15min":1.002},"last":0.623},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.71899999999999997},"min":{"1min":0.56100000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.073,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.80500000000000005,"15min":0.71899999999999997},"min":{"1min":0.61399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.375,"15min":0.375},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.78200000000000003,"15min":0.71099999999999997},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.755},{"interface":"front","average":{"1min":1.1080000000000001,"5min":0.84399999999999997,"15min":0.76900000000000002},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.90600000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:20:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.88100000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:20:39 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":0.98799999999999999,"15min":0.97299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:21:02 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.98299999999999998,"15min":0.93799999999999994},"min":{"1min":0.80300000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.8399999999999999,"5min":2.8399999999999999,"15min":2.8399999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89100000000000001,"15min":0.88400000000000001},"min":{"1min":0.71899999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.85599999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3700000000000001,"5min":2.883,"15min":2.883},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":0.93300000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5109999999999999,"5min":1.716,"15min":1.716},"last":0.82599999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":516411,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38600000000000001},"min":{"1min":0.35799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59699999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.44800000000000001,"15min":0.42999999999999999},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.377}]},{"osd":2,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":1.3,"15min":1.3},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.57399999999999995},"min":{"1min":0.50900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.68400000000000005,"15min":0.625},"min":{"1min":0.56599999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.65500000000000003,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65700000000000003,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.63700000000000001},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80700000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82099999999999995,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.75,"15min":0.71299999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.501}]},{"osd":8,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.95199999999999996,"5min":2.202,"15min":2.202},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.752,"15min":0.72799999999999998},"min":{"1min":0.65400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.038,"5min":1.526,"15min":1.526},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.68700000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.63300000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":1.097,"15min":1.097},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92300000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66600000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.68999999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72199999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":1107927,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81899999999999995,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90100000000000002,"5min":3.319,"15min":3.319},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42899999999999999,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4330000000000001,"5min":1.647,"15min":1.647},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.60299999999999998},"min":{"1min":0.503,"5min":0.436,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.67700000000000005},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.796,"15min":1.796},"last":0.745},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.64500000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95399999999999996,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66400000000000003,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.96999999999999997,"5min":1.623,"15min":1.623},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.628},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.82699999999999996},"min":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0489999999999999,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98799999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.57299999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.008,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.874},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93300000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:21:14.673 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:21:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.861+0000 7fcbb79ff700 1 -- 172.21.15.66:0/531656635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb00ffb10 msgr2=0x7fcbb00fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.861+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/531656635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb00ffb10 0x7fcbb00fff20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009990 tx=0x7fcbac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 -- 172.21.15.66:0/531656635 shutdown_connections 2023-07-21T22:21:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/531656635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb0101690 0x7fcbb0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/531656635 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb00ffb10 0x7fcbb00fff20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/531656635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbb0100d00 0x7fcbb0101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 -- 172.21.15.66:0/531656635 >> 172.21.15.66:0/531656635 conn(0x7fcbb00fb110 msgr2=0x7fcbb00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:14.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 -- 172.21.15.66:0/531656635 shutdown_connections 2023-07-21T22:21:14.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.862+0000 7fcbb79ff700 1 -- 172.21.15.66:0/531656635 wait complete. 2023-07-21T22:21:14.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 Processor -- start 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 -- start start 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 0x7fcbb0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb0100d00 0x7fcbb0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbb0101690 0x7fcbb019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019f5a0 con 0x7fcbb0100d00 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019f6e0 con 0x7fcbb0101690 2023-07-21T22:21:14.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.863+0000 7fcbb79ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb019f870 con 0x7fcbb00ffb10 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb0100d00 0x7fcbb0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 0x7fcbb0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 0x7fcbb0197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53288/0 (socket says 172.21.15.66:53288) 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 -- 172.21.15.66:0/1321272659 learned_addr learned my addr 172.21.15.66:0/1321272659 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb5f9c700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbb0101690 0x7fcbb019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 -- 172.21.15.66:0/1321272659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbb0101690 msgr2=0x7fcbb019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbb0101690 0x7fcbb019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 -- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb0100d00 msgr2=0x7fcbb0197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb0100d00 0x7fcbb0197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 -- 172.21.15.66:0/1321272659 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0095f0 con 0x7fcbb00ffb10 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb4f9a700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb0100d00 0x7fcbb0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.864+0000 7fcbb579b700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 0x7fcbb0197a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b620 tx=0x7fcbac005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.865+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac01d070 con 0x7fcbb00ffb10 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.865+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbac021410 con 0x7fcbb00ffb10 2023-07-21T22:21:14.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.865+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb019faf0 con 0x7fcbb00ffb10 2023-07-21T22:21:14.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.865+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac017650 con 0x7fcbb00ffb10 2023-07-21T22:21:14.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.865+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb019ff70 con 0x7fcbb00ffb10 2023-07-21T22:21:14.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.866+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb010b330 con 0x7fcbb00ffb10 2023-07-21T22:21:14.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.868+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcbac021580 con 0x7fcbb00ffb10 2023-07-21T22:21:14.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.869+0000 7fcba67fc700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fcb9c0a1220 0x7fcb9c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:14.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.869+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fcbac0ce0b0 con 0x7fcbb00ffb10 2023-07-21T22:21:14.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.869+0000 7fcbb4f9a700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fcb9c0a1220 0x7fcb9c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:14.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.870+0000 7fcbb4f9a700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fcb9c0a1220 0x7fcb9c0a36d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcba0011490 tx=0x7fcba0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:14.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:14.873+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcbac0974c0 con 0x7fcbb00ffb10 2023-07-21T22:21:14.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/716335015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:14.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:14 smithi066 ceph-mon[126379]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-07-21T22:21:15.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/716335015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:15.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:14 smithi104 ceph-mon[129021]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-07-21T22:21:15.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.081+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcbb0107c50 con 0x7fcb9c0a1220 2023-07-21T22:21:15.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.094+0000 7fcba67fc700 1 -- 172.21.15.66:0/1321272659 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427379 (secure 0 0 0) 0x7fcbb0107c50 con 0x7fcb9c0a1220 2023-07-21T22:21:15.144 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:15.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fcb9c0a1220 msgr2=0x7fcb9c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fcb9c0a1220 0x7fcb9c0a36d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcba0011490 tx=0x7fcba0009260 comp rx=0 tx=0).stop 2023-07-21T22:21:15.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 msgr2=0x7fcbb0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 0x7fcbb0197a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcbac00b620 tx=0x7fcbac005740 comp rx=0 tx=0).stop 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 shutdown_connections 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fcb9c0a1220 0x7fcb9c0a36d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbb0101690 0x7fcbb019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbb00ffb10 0x7fcbb0197a20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 --2- 172.21.15.66:0/1321272659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbb0100d00 0x7fcbb0197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 >> 172.21.15.66:0/1321272659 conn(0x7fcbb00fb110 msgr2=0x7fcbb0106530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 shutdown_connections 2023-07-21T22:21:15.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.099+0000 7fcbb79ff700 1 -- 172.21.15.66:0/1321272659 wait complete. 2023-07-21T22:21:15.148 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:21:15.163 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2023-07-21T22:21:14.210615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659093,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":760,"num_read_kb":506,"num_write":348,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335828,"kb_used_data":12404,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605996,"statfs":{"total":1151940427776,"available":1151596539904,"internally_reserved":0,"allocated":12701696,"data_stored":8143565,"data_compressed":389043,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5483,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":55,"num_write":10,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003385"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.239002+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:21:06.239002+0000","last_peered":"2023-07-21T22:21:06.239002+0000","last_clean":"2023-07-21T22:21:06.239002+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:21:06.239002+0000","last_undegraded":"2023-07-21T22:21:06.239002+0000","last_fullsized":"2023-07-21T22:21:06.239002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:18:11.706021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241220+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:21:06.241220+0000","last_peered":"2023-07-21T22:21:06.241220+0000","last_clean":"2023-07-21T22:21:06.241220+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:21:06.241220+0000","last_undegraded":"2023-07-21T22:21:06.241220+0000","last_fullsized":"2023-07-21T22:21:06.241220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:12.418124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244777+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:21:06.244777+0000","last_peered":"2023-07-21T22:21:06.244777+0000","last_clean":"2023-07-21T22:21:06.244777+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:21:06.244777+0000","last_undegraded":"2023-07-21T22:21:06.244777+0000","last_fullsized":"2023-07-21T22:21:06.244777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:49:28.752633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.231962+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:21:06.231962+0000","last_peered":"2023-07-21T22:21:06.231962+0000","last_clean":"2023-07-21T22:21:06.231962+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:21:06.231962+0000","last_undegraded":"2023-07-21T22:21:06.231962+0000","last_fullsized":"2023-07-21T22:21:06.231962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:10.307052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.238638+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:21:06.238638+0000","last_peered":"2023-07-21T22:21:06.238638+0000","last_clean":"2023-07-21T22:21:06.238638+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:21:06.238638+0000","last_undegraded":"2023-07-21T22:21:06.238638+0000","last_fullsized":"2023-07-21T22:21:06.238638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:34.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241211+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:21:06.241211+0000","last_peered":"2023-07-21T22:21:06.241211+0000","last_clean":"2023-07-21T22:21:06.241211+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:21:06.241211+0000","last_undegraded":"2023-07-21T22:21:06.241211+0000","last_fullsized":"2023-07-21T22:21:06.241211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:56:26.017768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233052+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:21:06.233052+0000","last_peered":"2023-07-21T22:21:06.233052+0000","last_clean":"2023-07-21T22:21:06.233052+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:21:06.233052+0000","last_undegraded":"2023-07-21T22:21:06.233052+0000","last_fullsized":"2023-07-21T22:21:06.233052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:39:12.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243697+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:21:06.243697+0000","last_peered":"2023-07-21T22:21:06.243697+0000","last_clean":"2023-07-21T22:21:06.243697+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:21:06.243697+0000","last_undegraded":"2023-07-21T22:21:06.243697+0000","last_fullsized":"2023-07-21T22:21:06.243697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:55.560292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236208+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:21:06.236208+0000","last_peered":"2023-07-21T22:21:06.236208+0000","last_clean":"2023-07-21T22:21:06.236208+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:21:06.236208+0000","last_undegraded":"2023-07-21T22:21:06.236208+0000","last_fullsized":"2023-07-21T22:21:06.236208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:25:01.311975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241324+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:21:06.241324+0000","last_peered":"2023-07-21T22:21:06.241324+0000","last_clean":"2023-07-21T22:21:06.241324+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:21:06.241324+0000","last_undegraded":"2023-07-21T22:21:06.241324+0000","last_fullsized":"2023-07-21T22:21:06.241324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:45:28.789239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229830+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:21:06.229830+0000","last_peered":"2023-07-21T22:21:06.229830+0000","last_clean":"2023-07-21T22:21:06.229830+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:21:06.229830+0000","last_undegraded":"2023-07-21T22:21:06.229830+0000","last_fullsized":"2023-07-21T22:21:06.229830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:42.902545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233939+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:21:06.233939+0000","last_peered":"2023-07-21T22:21:06.233939+0000","last_clean":"2023-07-21T22:21:06.233939+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:21:06.233939+0000","last_undegraded":"2023-07-21T22:21:06.233939+0000","last_fullsized":"2023-07-21T22:21:06.233939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:04:05.043773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240104+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:21:06.240104+0000","last_peered":"2023-07-21T22:21:06.240104+0000","last_clean":"2023-07-21T22:21:06.240104+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:21:06.240104+0000","last_undegraded":"2023-07-21T22:21:06.240104+0000","last_fullsized":"2023-07-21T22:21:06.240104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:42:32.982654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244272+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:21:06.244272+0000","last_peered":"2023-07-21T22:21:06.244272+0000","last_clean":"2023-07-21T22:21:06.244272+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:21:06.244272+0000","last_undegraded":"2023-07-21T22:21:06.244272+0000","last_fullsized":"2023-07-21T22:21:06.244272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:42.745730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241909+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:21:06.241909+0000","last_peered":"2023-07-21T22:21:06.241909+0000","last_clean":"2023-07-21T22:21:06.241909+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:21:06.241909+0000","last_undegraded":"2023-07-21T22:21:06.241909+0000","last_fullsized":"2023-07-21T22:21:06.241909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:19:27.316051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243456+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:21:06.243456+0000","last_peered":"2023-07-21T22:21:06.243456+0000","last_clean":"2023-07-21T22:21:06.243456+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:21:06.243456+0000","last_undegraded":"2023-07-21T22:21:06.243456+0000","last_fullsized":"2023-07-21T22:21:06.243456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:43.668214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.239421+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:21:06.239421+0000","last_peered":"2023-07-21T22:21:06.239421+0000","last_clean":"2023-07-21T22:21:06.239421+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:21:06.239421+0000","last_undegraded":"2023-07-21T22:21:06.239421+0000","last_fullsized":"2023-07-21T22:21:06.239421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:38:36.512107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232295+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:21:06.232295+0000","last_peered":"2023-07-21T22:21:06.232295+0000","last_clean":"2023-07-21T22:21:06.232295+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:21:06.232295+0000","last_undegraded":"2023-07-21T22:21:06.232295+0000","last_fullsized":"2023-07-21T22:21:06.232295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:01.043172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.210433+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:21:06.210433+0000","last_peered":"2023-07-21T22:21:06.210433+0000","last_clean":"2023-07-21T22:21:06.210433+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:21:06.210433+0000","last_undegraded":"2023-07-21T22:21:06.210433+0000","last_fullsized":"2023-07-21T22:21:06.210433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:10.034128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305676+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:20:38.305676+0000","last_peered":"2023-07-21T22:20:38.305676+0000","last_clean":"2023-07-21T22:20:38.305676+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:20:38.305676+0000","last_undegraded":"2023-07-21T22:20:38.305676+0000","last_fullsized":"2023-07-21T22:20:38.305676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:52.425113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298198+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:20:38.298198+0000","last_peered":"2023-07-21T22:20:38.298198+0000","last_clean":"2023-07-21T22:20:38.298198+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:20:38.298198+0000","last_undegraded":"2023-07-21T22:20:38.298198+0000","last_fullsized":"2023-07-21T22:20:38.298198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243894+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:21:06.243894+0000","last_peered":"2023-07-21T22:21:06.243894+0000","last_clean":"2023-07-21T22:21:06.243894+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:21:06.243894+0000","last_undegraded":"2023-07-21T22:21:06.243894+0000","last_fullsized":"2023-07-21T22:21:06.243894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:06.576092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.417884+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:20:38.417884+0000","last_peered":"2023-07-21T22:20:38.417884+0000","last_clean":"2023-07-21T22:20:38.417884+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:20:38.417884+0000","last_undegraded":"2023-07-21T22:20:38.417884+0000","last_fullsized":"2023-07-21T22:20:38.417884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:32:22.146374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.660121+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:20:38.660121+0000","last_peered":"2023-07-21T22:20:38.660121+0000","last_clean":"2023-07-21T22:20:38.660121+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:20:38.660121+0000","last_undegraded":"2023-07-21T22:20:38.660121+0000","last_fullsized":"2023-07-21T22:20:38.660121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:01:15.547748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.246264+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:21:06.246264+0000","last_peered":"2023-07-21T22:21:06.246264+0000","last_clean":"2023-07-21T22:21:06.246264+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:21:06.246264+0000","last_undegraded":"2023-07-21T22:21:06.246264+0000","last_fullsized":"2023-07-21T22:21:06.246264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:37.797343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418128+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:20:38.418128+0000","last_peered":"2023-07-21T22:20:38.418128+0000","last_clean":"2023-07-21T22:20:38.418128+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:20:38.418128+0000","last_undegraded":"2023-07-21T22:20:38.418128+0000","last_fullsized":"2023-07-21T22:20:38.418128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:12.532381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658825+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:20:38.658825+0000","last_peered":"2023-07-21T22:20:38.658825+0000","last_clean":"2023-07-21T22:20:38.658825+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:20:38.658825+0000","last_undegraded":"2023-07-21T22:20:38.658825+0000","last_fullsized":"2023-07-21T22:20:38.658825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:42:37.381662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243097+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:21:06.243097+0000","last_peered":"2023-07-21T22:21:06.243097+0000","last_clean":"2023-07-21T22:21:06.243097+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:21:06.243097+0000","last_undegraded":"2023-07-21T22:21:06.243097+0000","last_fullsized":"2023-07-21T22:21:06.243097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:35:44.049591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418190+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:20:38.418190+0000","last_peered":"2023-07-21T22:20:38.418190+0000","last_clean":"2023-07-21T22:20:38.418190+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:20:38.418190+0000","last_undegraded":"2023-07-21T22:20:38.418190+0000","last_fullsized":"2023-07-21T22:20:38.418190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:53:46.778497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658900+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:20:38.658900+0000","last_peered":"2023-07-21T22:20:38.658900+0000","last_clean":"2023-07-21T22:20:38.658900+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:20:38.658900+0000","last_undegraded":"2023-07-21T22:20:38.658900+0000","last_fullsized":"2023-07-21T22:20:38.658900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:41:42.378836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229401+0000","last_change":"2023-07-21T22:20:38.439768+0000","last_active":"2023-07-21T22:21:06.229401+0000","last_peered":"2023-07-21T22:21:06.229401+0000","last_clean":"2023-07-21T22:21:06.229401+0000","last_became_active":"2023-07-21T22:20:38.439322+0000","last_became_peered":"2023-07-21T22:20:38.439322+0000","last_unstale":"2023-07-21T22:21:06.229401+0000","last_undegraded":"2023-07-21T22:21:06.229401+0000","last_fullsized":"2023-07-21T22:21:06.229401+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:07.041475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305701+0000","last_change":"2023-07-21T22:19:37.404292+0000","last_active":"2023-07-21T22:20:38.305701+0000","last_peered":"2023-07-21T22:20:38.305701+0000","last_clean":"2023-07-21T22:20:38.305701+0000","last_became_active":"2023-07-21T22:19:37.403826+0000","last_became_peered":"2023-07-21T22:19:37.403826+0000","last_unstale":"2023-07-21T22:20:38.305701+0000","last_undegraded":"2023-07-21T22:20:38.305701+0000","last_fullsized":"2023-07-21T22:20:38.305701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:58.935037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":141,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:10.459881+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:21:10.459881+0000","last_peered":"2023-07-21T22:21:10.459881+0000","last_clean":"2023-07-21T22:21:10.459881+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:21:10.459881+0000","last_undegraded":"2023-07-21T22:21:10.459881+0000","last_fullsized":"2023-07-21T22:21:10.459881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:08.289972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.235446+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:21:06.235446+0000","last_peered":"2023-07-21T22:21:06.235446+0000","last_clean":"2023-07-21T22:21:06.235446+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:21:06.235446+0000","last_undegraded":"2023-07-21T22:21:06.235446+0000","last_fullsized":"2023-07-21T22:21:06.235446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:46:49.373988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305565+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:20:38.305565+0000","last_peered":"2023-07-21T22:20:38.305565+0000","last_clean":"2023-07-21T22:20:38.305565+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:20:38.305565+0000","last_undegraded":"2023-07-21T22:20:38.305565+0000","last_fullsized":"2023-07-21T22:20:38.305565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:38:55.810627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298031+0000","last_change":"2023-07-21T22:19:37.328942+0000","last_active":"2023-07-21T22:20:38.298031+0000","last_peered":"2023-07-21T22:20:38.298031+0000","last_clean":"2023-07-21T22:20:38.298031+0000","last_became_active":"2023-07-21T22:19:37.325738+0000","last_became_peered":"2023-07-21T22:19:37.325738+0000","last_unstale":"2023-07-21T22:20:38.298031+0000","last_undegraded":"2023-07-21T22:20:38.298031+0000","last_fullsized":"2023-07-21T22:20:38.298031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:18:17.573518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230305+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:21:06.230305+0000","last_peered":"2023-07-21T22:21:06.230305+0000","last_clean":"2023-07-21T22:21:06.230305+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:21:06.230305+0000","last_undegraded":"2023-07-21T22:21:06.230305+0000","last_fullsized":"2023-07-21T22:21:06.230305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:08:02.749178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.299090+0000","last_change":"2023-07-21T22:19:37.326119+0000","last_active":"2023-07-21T22:20:38.299090+0000","last_peered":"2023-07-21T22:20:38.299090+0000","last_clean":"2023-07-21T22:20:38.299090+0000","last_became_active":"2023-07-21T22:19:37.325675+0000","last_became_peered":"2023-07-21T22:19:37.325675+0000","last_unstale":"2023-07-21T22:20:38.299090+0000","last_undegraded":"2023-07-21T22:20:38.299090+0000","last_fullsized":"2023-07-21T22:20:38.299090+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:44:14.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295247+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:20:38.295247+0000","last_peered":"2023-07-21T22:20:38.295247+0000","last_clean":"2023-07-21T22:20:38.295247+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:20:38.295247+0000","last_undegraded":"2023-07-21T22:20:38.295247+0000","last_fullsized":"2023-07-21T22:20:38.295247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:41:42.444573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.242024+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:21:06.242024+0000","last_peered":"2023-07-21T22:21:06.242024+0000","last_clean":"2023-07-21T22:21:06.242024+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:21:06.242024+0000","last_undegraded":"2023-07-21T22:21:06.242024+0000","last_fullsized":"2023-07-21T22:21:06.242024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:32:03.234308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.660186+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:20:38.660186+0000","last_peered":"2023-07-21T22:20:38.660186+0000","last_clean":"2023-07-21T22:20:38.660186+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:20:38.660186+0000","last_undegraded":"2023-07-21T22:20:38.660186+0000","last_fullsized":"2023-07-21T22:20:38.660186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:23:22.856670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296564+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:20:38.296564+0000","last_peered":"2023-07-21T22:20:38.296564+0000","last_clean":"2023-07-21T22:20:38.296564+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:20:38.296564+0000","last_undegraded":"2023-07-21T22:20:38.296564+0000","last_fullsized":"2023-07-21T22:20:38.296564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:13:30.074340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233367+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:21:06.233367+0000","last_peered":"2023-07-21T22:21:06.233367+0000","last_clean":"2023-07-21T22:21:06.233367+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:21:06.233367+0000","last_undegraded":"2023-07-21T22:21:06.233367+0000","last_fullsized":"2023-07-21T22:21:06.233367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:03.385071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659286+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:20:38.659286+0000","last_peered":"2023-07-21T22:20:38.659286+0000","last_clean":"2023-07-21T22:20:38.659286+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:20:38.659286+0000","last_undegraded":"2023-07-21T22:20:38.659286+0000","last_fullsized":"2023-07-21T22:20:38.659286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:37.149270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":133,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:11.460791+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:21:11.460791+0000","last_peered":"2023-07-21T22:21:11.460791+0000","last_clean":"2023-07-21T22:21:11.460791+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:21:11.460791+0000","last_undegraded":"2023-07-21T22:21:11.460791+0000","last_fullsized":"2023-07-21T22:21:11.460791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:18:00.045946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.235609+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:21:06.235609+0000","last_peered":"2023-07-21T22:21:06.235609+0000","last_clean":"2023-07-21T22:21:06.235609+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:21:06.235609+0000","last_undegraded":"2023-07-21T22:21:06.235609+0000","last_fullsized":"2023-07-21T22:21:06.235609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:24:37.041577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297787+0000","last_change":"2023-07-21T22:19:37.326317+0000","last_active":"2023-07-21T22:20:38.297787+0000","last_peered":"2023-07-21T22:20:38.297787+0000","last_clean":"2023-07-21T22:20:38.297787+0000","last_became_active":"2023-07-21T22:19:37.325791+0000","last_became_peered":"2023-07-21T22:19:37.325791+0000","last_unstale":"2023-07-21T22:20:38.297787+0000","last_undegraded":"2023-07-21T22:20:38.297787+0000","last_fullsized":"2023-07-21T22:20:38.297787+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:28.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659430+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:20:38.659430+0000","last_peered":"2023-07-21T22:20:38.659430+0000","last_clean":"2023-07-21T22:20:38.659430+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:20:38.659430+0000","last_undegraded":"2023-07-21T22:20:38.659430+0000","last_fullsized":"2023-07-21T22:20:38.659430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:08.684209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.246230+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:21:06.246230+0000","last_peered":"2023-07-21T22:21:06.246230+0000","last_clean":"2023-07-21T22:21:06.246230+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:21:06.246230+0000","last_undegraded":"2023-07-21T22:21:06.246230+0000","last_fullsized":"2023-07-21T22:21:06.246230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:48.990870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418435+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:20:38.418435+0000","last_peered":"2023-07-21T22:20:38.418435+0000","last_clean":"2023-07-21T22:20:38.418435+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:20:38.418435+0000","last_undegraded":"2023-07-21T22:20:38.418435+0000","last_fullsized":"2023-07-21T22:20:38.418435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:20:47.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296928+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:20:38.296928+0000","last_peered":"2023-07-21T22:20:38.296928+0000","last_clean":"2023-07-21T22:20:38.296928+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:20:38.296928+0000","last_undegraded":"2023-07-21T22:20:38.296928+0000","last_fullsized":"2023-07-21T22:20:38.296928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:48.614307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243632+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:21:06.243632+0000","last_peered":"2023-07-21T22:21:06.243632+0000","last_clean":"2023-07-21T22:21:06.243632+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:21:06.243632+0000","last_undegraded":"2023-07-21T22:21:06.243632+0000","last_fullsized":"2023-07-21T22:21:06.243632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:59.106308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297543+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:20:38.297543+0000","last_peered":"2023-07-21T22:20:38.297543+0000","last_clean":"2023-07-21T22:20:38.297543+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:20:38.297543+0000","last_undegraded":"2023-07-21T22:20:38.297543+0000","last_fullsized":"2023-07-21T22:20:38.297543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:44.231916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":273,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:12.463095+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:21:12.463095+0000","last_peered":"2023-07-21T22:21:12.463095+0000","last_clean":"2023-07-21T22:21:12.463095+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:21:12.463095+0000","last_undegraded":"2023-07-21T22:21:12.463095+0000","last_fullsized":"2023-07-21T22:21:12.463095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:30.817175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63587,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":166,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.228970+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:21:06.228970+0000","last_peered":"2023-07-21T22:21:06.228970+0000","last_clean":"2023-07-21T22:21:06.228970+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:21:06.228970+0000","last_undegraded":"2023-07-21T22:21:06.228970+0000","last_fullsized":"2023-07-21T22:21:06.228970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:04:11.923390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297581+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:20:38.297581+0000","last_peered":"2023-07-21T22:20:38.297581+0000","last_clean":"2023-07-21T22:20:38.297581+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:20:38.297581+0000","last_undegraded":"2023-07-21T22:20:38.297581+0000","last_fullsized":"2023-07-21T22:20:38.297581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:24:31.302884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305466+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:20:38.305466+0000","last_peered":"2023-07-21T22:20:38.305466+0000","last_clean":"2023-07-21T22:20:38.305466+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:20:38.305466+0000","last_undegraded":"2023-07-21T22:20:38.305466+0000","last_fullsized":"2023-07-21T22:20:38.305466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:21.244524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245019+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:21:06.245019+0000","last_peered":"2023-07-21T22:21:06.245019+0000","last_clean":"2023-07-21T22:21:06.245019+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:21:06.245019+0000","last_undegraded":"2023-07-21T22:21:06.245019+0000","last_fullsized":"2023-07-21T22:21:06.245019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:37.737964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298975+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:20:38.298975+0000","last_peered":"2023-07-21T22:20:38.298975+0000","last_clean":"2023-07-21T22:20:38.298975+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:20:38.298975+0000","last_undegraded":"2023-07-21T22:20:38.298975+0000","last_fullsized":"2023-07-21T22:20:38.298975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:10:07.260019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296895+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:20:38.296895+0000","last_peered":"2023-07-21T22:20:38.296895+0000","last_clean":"2023-07-21T22:20:38.296895+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:20:38.296895+0000","last_undegraded":"2023-07-21T22:20:38.296895+0000","last_fullsized":"2023-07-21T22:20:38.296895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:20:47.729624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232411+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:21:06.232411+0000","last_peered":"2023-07-21T22:21:06.232411+0000","last_clean":"2023-07-21T22:21:06.232411+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:21:06.232411+0000","last_undegraded":"2023-07-21T22:21:06.232411+0000","last_fullsized":"2023-07-21T22:21:06.232411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:09.067826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658927+0000","last_change":"2023-07-21T22:19:37.403648+0000","last_active":"2023-07-21T22:20:38.658927+0000","last_peered":"2023-07-21T22:20:38.658927+0000","last_clean":"2023-07-21T22:20:38.658927+0000","last_became_active":"2023-07-21T22:19:37.403252+0000","last_became_peered":"2023-07-21T22:19:37.403252+0000","last_unstale":"2023-07-21T22:20:38.658927+0000","last_undegraded":"2023-07-21T22:20:38.658927+0000","last_fullsized":"2023-07-21T22:20:38.658927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:55.264540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:09.474149+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:21:09.474149+0000","last_peered":"2023-07-21T22:21:09.474149+0000","last_clean":"2023-07-21T22:21:09.474149+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:21:09.474149+0000","last_undegraded":"2023-07-21T22:21:09.474149+0000","last_fullsized":"2023-07-21T22:21:09.474149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:34:54.366109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.221294+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:21:06.221294+0000","last_peered":"2023-07-21T22:21:06.221294+0000","last_clean":"2023-07-21T22:21:06.221294+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:21:06.221294+0000","last_undegraded":"2023-07-21T22:21:06.221294+0000","last_fullsized":"2023-07-21T22:21:06.221294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:04.512567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295715+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:20:38.295715+0000","last_peered":"2023-07-21T22:20:38.295715+0000","last_clean":"2023-07-21T22:20:38.295715+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:20:38.295715+0000","last_undegraded":"2023-07-21T22:20:38.295715+0000","last_fullsized":"2023-07-21T22:20:38.295715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:17:47.809370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658510+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:20:38.658510+0000","last_peered":"2023-07-21T22:20:38.658510+0000","last_clean":"2023-07-21T22:20:38.658510+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:20:38.658510+0000","last_undegraded":"2023-07-21T22:20:38.658510+0000","last_fullsized":"2023-07-21T22:20:38.658510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:20:14.245284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230387+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:21:06.230387+0000","last_peered":"2023-07-21T22:21:06.230387+0000","last_clean":"2023-07-21T22:21:06.230387+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:21:06.230387+0000","last_undegraded":"2023-07-21T22:21:06.230387+0000","last_fullsized":"2023-07-21T22:21:06.230387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:31:10.400632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296528+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:20:38.296528+0000","last_peered":"2023-07-21T22:20:38.296528+0000","last_clean":"2023-07-21T22:20:38.296528+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:20:38.296528+0000","last_undegraded":"2023-07-21T22:20:38.296528+0000","last_fullsized":"2023-07-21T22:20:38.296528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:16.962575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659163+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:20:38.659163+0000","last_peered":"2023-07-21T22:20:38.659163+0000","last_clean":"2023-07-21T22:20:38.659163+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:20:38.659163+0000","last_undegraded":"2023-07-21T22:20:38.659163+0000","last_fullsized":"2023-07-21T22:20:38.659163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:11:28.407010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233495+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:21:06.233495+0000","last_peered":"2023-07-21T22:21:06.233495+0000","last_clean":"2023-07-21T22:21:06.233495+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:21:06.233495+0000","last_undegraded":"2023-07-21T22:21:06.233495+0000","last_fullsized":"2023-07-21T22:21:06.233495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:30.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658592+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:20:38.658592+0000","last_peered":"2023-07-21T22:20:38.658592+0000","last_clean":"2023-07-21T22:20:38.658592+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:20:38.658592+0000","last_undegraded":"2023-07-21T22:20:38.658592+0000","last_fullsized":"2023-07-21T22:20:38.658592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:55:48.187244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297860+0000","last_change":"2023-07-21T22:19:37.312431+0000","last_active":"2023-07-21T22:20:38.297860+0000","last_peered":"2023-07-21T22:20:38.297860+0000","last_clean":"2023-07-21T22:20:38.297860+0000","last_became_active":"2023-07-21T22:19:37.312038+0000","last_became_peered":"2023-07-21T22:19:37.312038+0000","last_unstale":"2023-07-21T22:20:38.297860+0000","last_undegraded":"2023-07-21T22:20:38.297860+0000","last_fullsized":"2023-07-21T22:20:38.297860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:13:00.069534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244472+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:21:06.244472+0000","last_peered":"2023-07-21T22:21:06.244472+0000","last_clean":"2023-07-21T22:21:06.244472+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:21:06.244472+0000","last_undegraded":"2023-07-21T22:21:06.244472+0000","last_fullsized":"2023-07-21T22:21:06.244472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:49.866646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295482+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:20:38.295482+0000","last_peered":"2023-07-21T22:20:38.295482+0000","last_clean":"2023-07-21T22:20:38.295482+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:20:38.295482+0000","last_undegraded":"2023-07-21T22:20:38.295482+0000","last_fullsized":"2023-07-21T22:20:38.295482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:51.240441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659099+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:20:38.659099+0000","last_peered":"2023-07-21T22:20:38.659099+0000","last_clean":"2023-07-21T22:20:38.659099+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:20:38.659099+0000","last_undegraded":"2023-07-21T22:20:38.659099+0000","last_fullsized":"2023-07-21T22:20:38.659099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:34:49.901496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237830+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:21:06.237830+0000","last_peered":"2023-07-21T22:21:06.237830+0000","last_clean":"2023-07-21T22:21:06.237830+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:21:06.237830+0000","last_undegraded":"2023-07-21T22:21:06.237830+0000","last_fullsized":"2023-07-21T22:21:06.237830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:47.649059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295392+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:20:38.295392+0000","last_peered":"2023-07-21T22:20:38.295392+0000","last_clean":"2023-07-21T22:20:38.295392+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:20:38.295392+0000","last_undegraded":"2023-07-21T22:20:38.295392+0000","last_fullsized":"2023-07-21T22:20:38.295392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:53:12.871041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230716+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:21:06.230716+0000","last_peered":"2023-07-21T22:21:06.230716+0000","last_clean":"2023-07-21T22:21:06.230716+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:21:06.230716+0000","last_undegraded":"2023-07-21T22:21:06.230716+0000","last_fullsized":"2023-07-21T22:21:06.230716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:07:20.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:12.478815+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:21:12.478815+0000","last_peered":"2023-07-21T22:21:12.478815+0000","last_clean":"2023-07-21T22:21:12.478815+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:21:12.478815+0000","last_undegraded":"2023-07-21T22:21:12.478815+0000","last_fullsized":"2023-07-21T22:21:12.478815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:09:04.461436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659421+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:20:38.659421+0000","last_peered":"2023-07-21T22:20:38.659421+0000","last_clean":"2023-07-21T22:20:38.659421+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:20:38.659421+0000","last_undegraded":"2023-07-21T22:20:38.659421+0000","last_fullsized":"2023-07-21T22:20:38.659421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:31.504021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243705+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:21:06.243705+0000","last_peered":"2023-07-21T22:21:06.243705+0000","last_clean":"2023-07-21T22:21:06.243705+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:21:06.243705+0000","last_undegraded":"2023-07-21T22:21:06.243705+0000","last_fullsized":"2023-07-21T22:21:06.243705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:21:28.483320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418342+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:20:38.418342+0000","last_peered":"2023-07-21T22:20:38.418342+0000","last_clean":"2023-07-21T22:20:38.418342+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:20:38.418342+0000","last_undegraded":"2023-07-21T22:20:38.418342+0000","last_fullsized":"2023-07-21T22:20:38.418342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:03:40.298453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658878+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:20:38.658878+0000","last_peered":"2023-07-21T22:20:38.658878+0000","last_clean":"2023-07-21T22:20:38.658878+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:20:38.658878+0000","last_undegraded":"2023-07-21T22:20:38.658878+0000","last_fullsized":"2023-07-21T22:20:38.658878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:55:41.830722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297102+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:20:38.297102+0000","last_peered":"2023-07-21T22:20:38.297102+0000","last_clean":"2023-07-21T22:20:38.297102+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:20:38.297102+0000","last_undegraded":"2023-07-21T22:20:38.297102+0000","last_fullsized":"2023-07-21T22:20:38.297102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:48.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237467+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:21:06.237467+0000","last_peered":"2023-07-21T22:21:06.237467+0000","last_clean":"2023-07-21T22:21:06.237467+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:21:06.237467+0000","last_undegraded":"2023-07-21T22:21:06.237467+0000","last_fullsized":"2023-07-21T22:21:06.237467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:00:22.036485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658972+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:20:38.658972+0000","last_peered":"2023-07-21T22:20:38.658972+0000","last_clean":"2023-07-21T22:20:38.658972+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:20:38.658972+0000","last_undegraded":"2023-07-21T22:20:38.658972+0000","last_fullsized":"2023-07-21T22:20:38.658972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:30.610746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298509+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:20:38.298509+0000","last_peered":"2023-07-21T22:20:38.298509+0000","last_clean":"2023-07-21T22:20:38.298509+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:20:38.298509+0000","last_undegraded":"2023-07-21T22:20:38.298509+0000","last_fullsized":"2023-07-21T22:20:38.298509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:03.929894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658954+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:20:38.658954+0000","last_peered":"2023-07-21T22:20:38.658954+0000","last_clean":"2023-07-21T22:20:38.658954+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:20:38.658954+0000","last_undegraded":"2023-07-21T22:20:38.658954+0000","last_fullsized":"2023-07-21T22:20:38.658954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:49.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240119+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:21:06.240119+0000","last_peered":"2023-07-21T22:21:06.240119+0000","last_clean":"2023-07-21T22:21:06.240119+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:21:06.240119+0000","last_undegraded":"2023-07-21T22:21:06.240119+0000","last_fullsized":"2023-07-21T22:21:06.240119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:33.312382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659732+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:20:38.659732+0000","last_peered":"2023-07-21T22:20:38.659732+0000","last_clean":"2023-07-21T22:20:38.659732+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:20:38.659732+0000","last_undegraded":"2023-07-21T22:20:38.659732+0000","last_fullsized":"2023-07-21T22:20:38.659732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:39.197537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298123+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:20:38.298123+0000","last_peered":"2023-07-21T22:20:38.298123+0000","last_clean":"2023-07-21T22:20:38.298123+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:20:38.298123+0000","last_undegraded":"2023-07-21T22:20:38.298123+0000","last_fullsized":"2023-07-21T22:20:38.298123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:05.244612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243542+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:21:06.243542+0000","last_peered":"2023-07-21T22:21:06.243542+0000","last_clean":"2023-07-21T22:21:06.243542+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:21:06.243542+0000","last_undegraded":"2023-07-21T22:21:06.243542+0000","last_fullsized":"2023-07-21T22:21:06.243542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:14:42.196678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305178+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:20:38.305178+0000","last_peered":"2023-07-21T22:20:38.305178+0000","last_clean":"2023-07-21T22:20:38.305178+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:20:38.305178+0000","last_undegraded":"2023-07-21T22:20:38.305178+0000","last_fullsized":"2023-07-21T22:20:38.305178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:41:59.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418371+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:20:38.418371+0000","last_peered":"2023-07-21T22:20:38.418371+0000","last_clean":"2023-07-21T22:20:38.418371+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:20:38.418371+0000","last_undegraded":"2023-07-21T22:20:38.418371+0000","last_fullsized":"2023-07-21T22:20:38.418371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:32.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245652+0000","last_change":"2023-07-21T22:19:37.404445+0000","last_active":"2023-07-21T22:21:06.245652+0000","last_peered":"2023-07-21T22:21:06.245652+0000","last_clean":"2023-07-21T22:21:06.245652+0000","last_became_active":"2023-07-21T22:19:37.404011+0000","last_became_peered":"2023-07-21T22:19:37.404011+0000","last_unstale":"2023-07-21T22:21:06.245652+0000","last_undegraded":"2023-07-21T22:21:06.245652+0000","last_fullsized":"2023-07-21T22:21:06.245652+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:04.926506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418255+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:20:38.418255+0000","last_peered":"2023-07-21T22:20:38.418255+0000","last_clean":"2023-07-21T22:20:38.418255+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:20:38.418255+0000","last_undegraded":"2023-07-21T22:20:38.418255+0000","last_fullsized":"2023-07-21T22:20:38.418255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:26:43.576746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.438152+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:20:38.438152+0000","last_peered":"2023-07-21T22:20:38.438152+0000","last_clean":"2023-07-21T22:20:38.438152+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:20:38.438152+0000","last_undegraded":"2023-07-21T22:20:38.438152+0000","last_fullsized":"2023-07-21T22:20:38.438152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:51:01.655488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.204620+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:21:06.204620+0000","last_peered":"2023-07-21T22:21:06.204620+0000","last_clean":"2023-07-21T22:21:06.204620+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:21:06.204620+0000","last_undegraded":"2023-07-21T22:21:06.204620+0000","last_fullsized":"2023-07-21T22:21:06.204620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:36:08.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298007+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:20:38.298007+0000","last_peered":"2023-07-21T22:20:38.298007+0000","last_clean":"2023-07-21T22:20:38.298007+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:20:38.298007+0000","last_undegraded":"2023-07-21T22:20:38.298007+0000","last_fullsized":"2023-07-21T22:20:38.298007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:54.100394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659458+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:20:38.659458+0000","last_peered":"2023-07-21T22:20:38.659458+0000","last_clean":"2023-07-21T22:20:38.659458+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:20:38.659458+0000","last_undegraded":"2023-07-21T22:20:38.659458+0000","last_fullsized":"2023-07-21T22:20:38.659458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:11:22.743920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244209+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:21:06.244209+0000","last_peered":"2023-07-21T22:21:06.244209+0000","last_clean":"2023-07-21T22:21:06.244209+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:21:06.244209+0000","last_undegraded":"2023-07-21T22:21:06.244209+0000","last_fullsized":"2023-07-21T22:21:06.244209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:10:19.440093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418254+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:20:38.418254+0000","last_peered":"2023-07-21T22:20:38.418254+0000","last_clean":"2023-07-21T22:20:38.418254+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:20:38.418254+0000","last_undegraded":"2023-07-21T22:20:38.418254+0000","last_fullsized":"2023-07-21T22:20:38.418254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:52.621668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298467+0000","last_change":"2023-07-21T22:19:37.315809+0000","last_active":"2023-07-21T22:20:38.298467+0000","last_peered":"2023-07-21T22:20:38.298467+0000","last_clean":"2023-07-21T22:20:38.298467+0000","last_became_active":"2023-07-21T22:19:37.312331+0000","last_became_peered":"2023-07-21T22:19:37.312331+0000","last_unstale":"2023-07-21T22:20:38.298467+0000","last_undegraded":"2023-07-21T22:20:38.298467+0000","last_fullsized":"2023-07-21T22:20:38.298467+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:01:40.724543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243178+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:21:06.243178+0000","last_peered":"2023-07-21T22:21:06.243178+0000","last_clean":"2023-07-21T22:21:06.243178+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:21:06.243178+0000","last_undegraded":"2023-07-21T22:21:06.243178+0000","last_fullsized":"2023-07-21T22:21:06.243178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:55.735567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418468+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:20:38.418468+0000","last_peered":"2023-07-21T22:20:38.418468+0000","last_clean":"2023-07-21T22:20:38.418468+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:20:38.418468+0000","last_undegraded":"2023-07-21T22:20:38.418468+0000","last_fullsized":"2023-07-21T22:20:38.418468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:59.817080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658662+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:20:38.658662+0000","last_peered":"2023-07-21T22:20:38.658662+0000","last_clean":"2023-07-21T22:20:38.658662+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:20:38.658662+0000","last_undegraded":"2023-07-21T22:20:38.658662+0000","last_fullsized":"2023-07-21T22:20:38.658662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:59:29.846402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229474+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:21:06.229474+0000","last_peered":"2023-07-21T22:21:06.229474+0000","last_clean":"2023-07-21T22:21:06.229474+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:21:06.229474+0000","last_undegraded":"2023-07-21T22:21:06.229474+0000","last_fullsized":"2023-07-21T22:21:06.229474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:37.964833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.438255+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:20:38.438255+0000","last_peered":"2023-07-21T22:20:38.438255+0000","last_clean":"2023-07-21T22:20:38.438255+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:20:38.438255+0000","last_undegraded":"2023-07-21T22:20:38.438255+0000","last_fullsized":"2023-07-21T22:20:38.438255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:30.756639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305874+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:20:38.305874+0000","last_peered":"2023-07-21T22:20:38.305874+0000","last_clean":"2023-07-21T22:20:38.305874+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:20:38.305874+0000","last_undegraded":"2023-07-21T22:20:38.305874+0000","last_fullsized":"2023-07-21T22:20:38.305874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:39:12.174231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230951+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:21:06.230951+0000","last_peered":"2023-07-21T22:21:06.230951+0000","last_clean":"2023-07-21T22:21:06.230951+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:21:06.230951+0000","last_undegraded":"2023-07-21T22:21:06.230951+0000","last_fullsized":"2023-07-21T22:21:06.230951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:11:25.905492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659463+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:20:38.659463+0000","last_peered":"2023-07-21T22:20:38.659463+0000","last_clean":"2023-07-21T22:20:38.659463+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:20:38.659463+0000","last_undegraded":"2023-07-21T22:20:38.659463+0000","last_fullsized":"2023-07-21T22:20:38.659463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:09:28.212992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297292+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:20:38.297292+0000","last_peered":"2023-07-21T22:20:38.297292+0000","last_clean":"2023-07-21T22:20:38.297292+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:20:38.297292+0000","last_undegraded":"2023-07-21T22:20:38.297292+0000","last_fullsized":"2023-07-21T22:20:38.297292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:43:21.204017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.226015+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:21:06.226015+0000","last_peered":"2023-07-21T22:21:06.226015+0000","last_clean":"2023-07-21T22:21:06.226015+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:21:06.226015+0000","last_undegraded":"2023-07-21T22:21:06.226015+0000","last_fullsized":"2023-07-21T22:21:06.226015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:16.718601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296263+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:20:38.296263+0000","last_peered":"2023-07-21T22:20:38.296263+0000","last_clean":"2023-07-21T22:20:38.296263+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:20:38.296263+0000","last_undegraded":"2023-07-21T22:20:38.296263+0000","last_fullsized":"2023-07-21T22:20:38.296263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:36.550687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295758+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:20:38.295758+0000","last_peered":"2023-07-21T22:20:38.295758+0000","last_clean":"2023-07-21T22:20:38.295758+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:20:38.295758+0000","last_undegraded":"2023-07-21T22:20:38.295758+0000","last_fullsized":"2023-07-21T22:20:38.295758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:13.984980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.227990+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:21:06.227990+0000","last_peered":"2023-07-21T22:21:06.227990+0000","last_clean":"2023-07-21T22:21:06.227990+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:21:06.227990+0000","last_undegraded":"2023-07-21T22:21:06.227990+0000","last_fullsized":"2023-07-21T22:21:06.227990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:11.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237254+0000","last_change":"2023-07-21T22:20:38.306667+0000","last_active":"2023-07-21T22:21:06.237254+0000","last_peered":"2023-07-21T22:21:06.237254+0000","last_clean":"2023-07-21T22:21:06.237254+0000","last_became_active":"2023-07-21T22:20:38.306285+0000","last_became_peered":"2023-07-21T22:20:38.306285+0000","last_unstale":"2023-07-21T22:21:06.237254+0000","last_undegraded":"2023-07-21T22:21:06.237254+0000","last_fullsized":"2023-07-21T22:21:06.237254+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:32:37.690707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236563+0000","last_change":"2023-07-21T22:19:37.326004+0000","last_active":"2023-07-21T22:21:06.236563+0000","last_peered":"2023-07-21T22:21:06.236563+0000","last_clean":"2023-07-21T22:21:06.236563+0000","last_became_active":"2023-07-21T22:19:37.325611+0000","last_became_peered":"2023-07-21T22:19:37.325611+0000","last_unstale":"2023-07-21T22:21:06.236563+0000","last_undegraded":"2023-07-21T22:21:06.236563+0000","last_fullsized":"2023-07-21T22:21:06.236563+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:02.980321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.212111+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:21:06.212111+0000","last_peered":"2023-07-21T22:21:06.212111+0000","last_clean":"2023-07-21T22:21:06.212111+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:21:06.212111+0000","last_undegraded":"2023-07-21T22:21:06.212111+0000","last_fullsized":"2023-07-21T22:21:06.212111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:22:52.562457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243872+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:21:06.243872+0000","last_peered":"2023-07-21T22:21:06.243872+0000","last_clean":"2023-07-21T22:21:06.243872+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:21:06.243872+0000","last_undegraded":"2023-07-21T22:21:06.243872+0000","last_fullsized":"2023-07-21T22:21:06.243872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:39.140948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240603+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:21:06.240603+0000","last_peered":"2023-07-21T22:21:06.240603+0000","last_clean":"2023-07-21T22:21:06.240603+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:21:06.240603+0000","last_undegraded":"2023-07-21T22:21:06.240603+0000","last_fullsized":"2023-07-21T22:21:06.240603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:52.883722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.226450+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:21:06.226450+0000","last_peered":"2023-07-21T22:21:06.226450+0000","last_clean":"2023-07-21T22:21:06.226450+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:21:06.226450+0000","last_undegraded":"2023-07-21T22:21:06.226450+0000","last_fullsized":"2023-07-21T22:21:06.226450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:44:28.022785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236553+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:21:06.236553+0000","last_peered":"2023-07-21T22:21:06.236553+0000","last_clean":"2023-07-21T22:21:06.236553+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:21:06.236553+0000","last_undegraded":"2023-07-21T22:21:06.236553+0000","last_fullsized":"2023-07-21T22:21:06.236553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:57.476981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.228554+0000","last_change":"2023-07-21T22:19:37.314152+0000","last_active":"2023-07-21T22:21:06.228554+0000","last_peered":"2023-07-21T22:21:06.228554+0000","last_clean":"2023-07-21T22:21:06.228554+0000","last_became_active":"2023-07-21T22:19:37.313795+0000","last_became_peered":"2023-07-21T22:19:37.313795+0000","last_unstale":"2023-07-21T22:21:06.228554+0000","last_undegraded":"2023-07-21T22:21:06.228554+0000","last_fullsized":"2023-07-21T22:21:06.228554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:52:37.184312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233179+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:21:06.233179+0000","last_peered":"2023-07-21T22:21:06.233179+0000","last_clean":"2023-07-21T22:21:06.233179+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:21:06.233179+0000","last_undegraded":"2023-07-21T22:21:06.233179+0000","last_fullsized":"2023-07-21T22:21:06.233179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:23.465390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232972+0000","last_change":"2023-07-21T22:20:38.307235+0000","last_active":"2023-07-21T22:21:06.232972+0000","last_peered":"2023-07-21T22:21:06.232972+0000","last_clean":"2023-07-21T22:21:06.232972+0000","last_became_active":"2023-07-21T22:20:38.306829+0000","last_became_peered":"2023-07-21T22:20:38.306829+0000","last_unstale":"2023-07-21T22:21:06.232972+0000","last_undegraded":"2023-07-21T22:21:06.232972+0000","last_fullsized":"2023-07-21T22:21:06.232972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:18:35.611527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245317+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:21:06.245317+0000","last_peered":"2023-07-21T22:21:06.245317+0000","last_clean":"2023-07-21T22:21:06.245317+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:21:06.245317+0000","last_undegraded":"2023-07-21T22:21:06.245317+0000","last_fullsized":"2023-07-21T22:21:06.245317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:17.933758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.242741+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:21:06.242741+0000","last_peered":"2023-07-21T22:21:06.242741+0000","last_clean":"2023-07-21T22:21:06.242741+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:21:06.242741+0000","last_undegraded":"2023-07-21T22:21:06.242741+0000","last_fullsized":"2023-07-21T22:21:06.242741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:42:34.683114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233743+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:21:06.233743+0000","last_peered":"2023-07-21T22:21:06.233743+0000","last_clean":"2023-07-21T22:21:06.233743+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:21:06.233743+0000","last_undegraded":"2023-07-21T22:21:06.233743+0000","last_fullsized":"2023-07-21T22:21:06.233743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:02:26.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68479,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":624,"num_read_kb":387,"num_write":165,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":515866,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89000000000000001,"5min":1.21,"15min":1.21},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90200000000000002,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81599999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.48199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.879,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69399999999999995,"15min":0.68999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.99299999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93500000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.754}]},{"osd":7,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95799999999999996,"5min":1.294,"15min":1.294},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76700000000000002,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.01,"5min":1.024,"15min":1.024},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97699999999999998,"5min":1.431,"15min":1.431},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.64400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.462,"15min":1.462},"last":0.83999999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.79900000000000004},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.622,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97599999999999998,"5min":1.5,"15min":1.5},"last":0.82499999999999996}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1163264,"data_stored":1167204,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":1.143,"15min":1.2370000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.754,"5min":17.132000000000001,"15min":17.132000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.1439999999999999,"15min":1.234},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":17.084,"15min":17.084},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.44700000000000001,"15min":0.46300000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56000000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61699999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78100000000000003,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.747,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80100000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.747}]},{"osd":6,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.363,"15min":1.363},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77400000000000002,"15min":0.80200000000000005},"min":{"1min":0.45400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84599999999999997,"5min":1.161,"15min":1.161},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86399999999999999,"5min":1.024,"15min":1.024},"last":0.88300000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91700000000000004,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.95899999999999996,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88400000000000001,"5min":17.163,"15min":17.163},"last":0.91000000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81200000000000006,"15min":0.82299999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99399999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.113,"15min":1.113},"last":0.95399999999999996}]}]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":520198,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.437},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85899999999999999,"15min":0.91700000000000004},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77800000000000002,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.81799999999999995},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.84499999999999997,"15min":0.89100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.48799999999999999,"5min":0.433,"15min":0.433},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.626},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":0.92900000000000005},"min":{"1min":0.57899999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.877,"15min":0.92500000000000004},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.94599999999999995},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.97499999999999998,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.006,"15min":1.077},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.585,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.754}]},{"osd":8,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.052,"15min":1.133},"min":{"1min":0.61599999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.611,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.069},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.9019999999999999,"5min":1.99,"15min":1.99},"last":0.79700000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.94199999999999995,"15min":0.96299999999999997},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.29,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.91800000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.0389999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":516022,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83999999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.629},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.753,"15min":0.745},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.747},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.59999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.627,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.80200000000000005,"15min":0.77000000000000002},"min":{"1min":0.73299999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.82799999999999996,"15min":0.79600000000000004},"min":{"1min":0.78500000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.97099999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.58399999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93400000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.79700000000000004,"15min":0.75700000000000001},"min":{"1min":0.66500000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.90400000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.879},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89600000000000002,"15min":0.89500000000000002},"min":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.94799999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":1112822,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.49099999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.36799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.503,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75900000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88700000000000001,"5min":1.399,"15min":1.399},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.498,"15min":0.51200000000000001},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.441}]},{"osd":5,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.5,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77400000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.57499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91700000000000004,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.57799999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.66100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.62,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96599999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.60199999999999998,"5min":0.499,"15min":0.499},"max":{"1min":0.95599999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.69899999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78900000000000003,"15min":0.78100000000000003},"min":{"1min":0.69099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77000000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":511278,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.365,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.498},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.76400000000000001,"15min":0.72699999999999998},"min":{"1min":0.63200000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.139,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73599999999999999,"15min":0.70499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.258,"5min":1.284,"15min":1.284},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.65300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.746},"min":{"1min":0.55500000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.80000000000000004},"min":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2170000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.871},{"interface":"front","average":{"1min":0.996,"5min":0.91900000000000004,"15min":0.88300000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.202,"5min":3.202,"15min":3.202},"last":1.0069999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.93200000000000005,"15min":0.93500000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1279999999999999,"5min":1.599,"15min":1.599},"last":1.026},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.66400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1799999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:20:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.0109999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.091},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.94099999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.84299999999999997,"15min":0.85599999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2629999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.018,"15min":1.036},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1120000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.048}]},{"osd":11,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.94199999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0369999999999999,"5min":1.585,"15min":1.585},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.0660000000000001,"15min":1.083},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.163,"5min":1.373,"15min":1.373},"last":1.069}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":574775,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.317,"5min":0.31,"15min":0.31},"max":{"1min":0.69199999999999995,"5min":0.95099999999999996,"15min":0.98799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.88300000000000001,"15min":1.1100000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60599999999999998,"15min":0.57399999999999995},"min":{"1min":0.51000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84199999999999997,"5min":0.871,"15min":1.0289999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58799999999999997,"15min":0.56699999999999995},"min":{"1min":0.47399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":0.93400000000000005},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.66400000000000003,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.0119999999999996,"5min":6.0119999999999996,"15min":6.0119999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.57799999999999996},"min":{"1min":0.45300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.46200000000000002},"min":{"1min":0.38600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56499999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69299999999999995,"15min":0.66400000000000003},"min":{"1min":0.64100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.66900000000000004},"min":{"1min":0.51100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0169999999999999,"5min":1.45,"15min":1.45},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58999999999999997,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85699999999999998,"5min":1.077,"15min":1.077},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.59399999999999997},"min":{"1min":0.56999999999999995,"5min":0.313,"15min":0.313},"max":{"1min":0.93600000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99199999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.94299999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.75,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80800000000000005},"min":{"1min":0.69799999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.755},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.753,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.98799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":0.93600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.877,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.93000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":512666,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.67900000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35999999999999999,"15min":0.35899999999999999},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73499999999999999,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.66500000000000004},"min":{"1min":0.52500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.56200000000000006,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.90000000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.2250000000000001,"5min":1.23,"15min":1.23},"last":0.37}]},{"osd":4,"last update":"Fri Jul 21 22:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.74399999999999999,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.72999999999999998,"15min":0.68000000000000005},"min":{"1min":0.60299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.149,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.73599999999999999},"min":{"1min":0.55400000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.246,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:20:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.78200000000000003,"15min":0.72499999999999998},"min":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68700000000000006,"15min":0.63700000000000001},"min":{"1min":0.58299999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.496}]},{"osd":7,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.97099999999999997,"15min":0.98899999999999999},"min":{"1min":0.65700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.226,"5min":0.88800000000000001,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.0640000000000001,"5min":6.0640000000000001,"15min":6.0640000000000001},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98399999999999999,"15min":0.99399999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4199999999999999,"5min":1.526,"15min":1.526},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.871,"15min":0.84899999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:20:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.86899999999999999,"15min":0.84599999999999997},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.71399999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.88}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1105920,"data_stored":576924,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.79300000000000004,"5min":1.9530000000000001,"15min":3.988},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.754,"5min":1.421,"15min":1.421},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.626,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.006,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.54500000000000004},"min":{"1min":0.51600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":2.044,"15min":2.044},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":1.984,"15min":1.984},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.45800000000000002},"min":{"1min":0.36899999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.82799999999999996,"5min":1.542,"15min":1.542},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.47299999999999998},"min":{"1min":0.39200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.84499999999999997,"5min":1.76,"15min":1.76},"last":0.755}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71399999999999997,"15min":0.68500000000000005},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.079,"5min":1.907,"15min":1.907},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72399999999999998,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.0489999999999999,"5min":1.484,"15min":1.484},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98199999999999998,"5min":1.609,"15min":1.609},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0620000000000001,"5min":1.859,"15min":1.859},"last":0.496}]},{"osd":9,"last update":"Fri Jul 21 22:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.117,"5min":2.016,"15min":2.016},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.69099999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.022,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.81699999999999995,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.149,"5min":1.887,"15min":1.887},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.82099999999999995},"min":{"1min":0.71099999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.101,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88500000000000001,"15min":0.88200000000000001},"min":{"1min":0.71899999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":511472,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.501,"15min":0.41299999999999998},"min":{"1min":0.39100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.42299999999999999},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.63800000000000001,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.67200000000000004,"15min":0.57599999999999996},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.734,"5min":2.734,"15min":2.734},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67000000000000004,"15min":0.60999999999999999},"min":{"1min":0.55200000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86499999999999999,"5min":1.139,"15min":1.139},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.177,"5min":0.76900000000000002,"15min":0.65900000000000003},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.0199999999999996,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.68500000000000005,"15min":0.61199999999999999},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.746,"15min":0.64600000000000002},"min":{"1min":0.57699999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69999999999999996,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":1.002,"15min":1.002},"last":0.623},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.71899999999999997},"min":{"1min":0.56100000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.073,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.80500000000000005,"15min":0.71899999999999997},"min":{"1min":0.61399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.375,"15min":0.375},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.78200000000000003,"15min":0.71099999999999997},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.755},{"interface":"front","average":{"1min":1.1080000000000001,"5min":0.84399999999999997,"15min":0.76900000000000002},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.90600000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:20:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.88100000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:20:39 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":0.98799999999999999,"15min":0.97299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:21:02 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.98299999999999998,"15min":0.93799999999999994},"min":{"1min":0.80300000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.8399999999999999,"5min":2.8399999999999999,"15min":2.8399999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89100000000000001,"15min":0.88400000000000001},"min":{"1min":0.71899999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.85599999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3700000000000001,"5min":2.883,"15min":2.883},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":0.93300000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5109999999999999,"5min":1.716,"15min":1.716},"last":0.82599999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":516411,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38600000000000001},"min":{"1min":0.35799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59699999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.44800000000000001,"15min":0.42999999999999999},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.377}]},{"osd":2,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":1.3,"15min":1.3},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.57399999999999995},"min":{"1min":0.50900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.68400000000000005,"15min":0.625},"min":{"1min":0.56599999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.65500000000000003,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65700000000000003,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.63700000000000001},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80700000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82099999999999995,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.75,"15min":0.71299999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.501}]},{"osd":8,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.95199999999999996,"5min":2.202,"15min":2.202},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.752,"15min":0.72799999999999998},"min":{"1min":0.65400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.038,"5min":1.526,"15min":1.526},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.68700000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.63300000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":1.097,"15min":1.097},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92300000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66600000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.68999999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72199999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":1107927,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81899999999999995,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90100000000000002,"5min":3.319,"15min":3.319},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42899999999999999,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4330000000000001,"5min":1.647,"15min":1.647},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.60299999999999998},"min":{"1min":0.503,"5min":0.436,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.67700000000000005},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.796,"15min":1.796},"last":0.745},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.64500000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95399999999999996,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66400000000000003,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.96999999999999997,"5min":1.623,"15min":1.623},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.628},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.82699999999999996},"min":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0489999999999999,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98799999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.57299999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.008,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.874},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93300000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:21:15.173 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-21T22:21:15.173 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T22:21:15.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/716335015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:15.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:14 smithi149 ceph-mon[129268]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-07-21T22:21:15.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 -- 172.21.15.66:0/3788973881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294100cf0 msgr2=0x7f0294101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 --2- 172.21.15.66:0/3788973881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294100cf0 0x7f0294101140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02840098f0 tx=0x7f0284009c00 comp rx=0 tx=0).stop 2023-07-21T22:21:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 -- 172.21.15.66:0/3788973881 shutdown_connections 2023-07-21T22:21:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 --2- 172.21.15.66:0/3788973881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f02940ffb00 0x7f02940fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 --2- 172.21.15.66:0/3788973881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0294101680 0x7f0294107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 --2- 172.21.15.66:0/3788973881 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294100cf0 0x7f0294101140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 -- 172.21.15.66:0/3788973881 >> 172.21.15.66:0/3788973881 conn(0x7f02940fb0e0 msgr2=0x7f02940fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:15.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 -- 172.21.15.66:0/3788973881 shutdown_connections 2023-07-21T22:21:15.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.364+0000 7f029c5ff700 1 -- 172.21.15.66:0/3788973881 wait complete. 2023-07-21T22:21:15.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.365+0000 7f029c5ff700 1 Processor -- start 2023-07-21T22:21:15.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 -- start start 2023-07-21T22:21:15.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 0x7f0294195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0294100cf0 0x7f0294195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 0x7f029419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029419d410 con 0x7f0294101680 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029419d550 con 0x7f0294100cf0 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029c5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029419d6e0 con 0x7f02940ffb00 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029a39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 0x7f0294195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029ab9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 0x7f029419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029ab9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 0x7f029419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56872/0 (socket says 172.21.15.66:56872) 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029a39b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 0x7f0294195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53306/0 (socket says 172.21.15.66:53306) 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f029ab9c700 1 -- 172.21.15.66:0/1782222332 learned_addr learned my addr 172.21.15.66:0/1782222332 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.366+0000 7f0299b9a700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0294100cf0 0x7f0294195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029a39b700 1 -- 172.21.15.66:0/1782222332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0294100cf0 msgr2=0x7f0294195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029a39b700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0294100cf0 0x7f0294195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029a39b700 1 -- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 msgr2=0x7f029419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029a39b700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 0x7f029419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029a39b700 1 -- 172.21.15.66:0/1782222332 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02900095f0 con 0x7f02940ffb00 2023-07-21T22:21:15.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029ab9c700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 0x7f029419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:15.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029a39b700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 0x7f0294195890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0290005fd0 tx=0x7f029000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:15.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029000cbd0 con 0x7f02940ffb00 2023-07-21T22:21:15.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0284009520 con 0x7f02940ffb00 2023-07-21T22:21:15.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f029000cd30 con 0x7f02940ffb00 2023-07-21T22:21:15.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.367+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0290018710 con 0x7f02940ffb00 2023-07-21T22:21:15.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.368+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029419dd40 con 0x7f02940ffb00 2023-07-21T22:21:15.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.368+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029410b320 con 0x7f02940ffb00 2023-07-21T22:21:15.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.371+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f0290010980 con 0x7f02940ffb00 2023-07-21T22:21:15.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.373+0000 7f028b7fe700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02800a11a0 0x7f02800a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.373+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0290014070 con 0x7f02940ffb00 2023-07-21T22:21:15.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.373+0000 7f0299b9a700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02800a11a0 0x7f02800a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.374+0000 7f0299b9a700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02800a11a0 0x7f02800a3650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0284005e90 tx=0x7f0284019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:15.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.376+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f02900968b0 con 0x7f02940ffb00 2023-07-21T22:21:15.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.588+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0294060950 con 0x7f02800a11a0 2023-07-21T22:21:15.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.601+0000 7f028b7fe700 1 -- 172.21.15.66:0/1782222332 <== mgr.14150 v2:172.21.15.66:6800/1262827329 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427379 (secure 0 0 0) 0x7f0294060950 con 0x7f02800a11a0 2023-07-21T22:21:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:15.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02800a11a0 msgr2=0x7f02800a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02800a11a0 0x7f02800a3650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0284005e90 tx=0x7f0284019040 comp rx=0 tx=0).stop 2023-07-21T22:21:15.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 msgr2=0x7f0294195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 0x7f0294195890 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0290005fd0 tx=0x7f029000ee70 comp rx=0 tx=0).stop 2023-07-21T22:21:15.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 shutdown_connections 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f02800a11a0 0x7f02800a3650 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0294101680 0x7f029419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f02940ffb00 0x7f0294195890 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 --2- 172.21.15.66:0/1782222332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0294100cf0 0x7f0294195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 >> 172.21.15.66:0/1782222332 conn(0x7f02940fb0e0 msgr2=0x7f02940fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 shutdown_connections 2023-07-21T22:21:15.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.607+0000 7f029c5ff700 1 -- 172.21.15.66:0/1782222332 wait complete. 2023-07-21T22:21:15.657 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2023-07-21T22:21:15.671 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2023-07-21T22:21:14.210615+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659093,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":760,"num_read_kb":506,"num_write":348,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335828,"kb_used_data":12404,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124605996,"statfs":{"total":1151940427776,"available":1151596539904,"internally_reserved":0,"allocated":12701696,"data_stored":8143565,"data_compressed":389043,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5483,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":55,"num_write":10,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003385"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.239002+0000","last_change":"2023-07-21T22:19:07.369434+0000","last_active":"2023-07-21T22:21:06.239002+0000","last_peered":"2023-07-21T22:21:06.239002+0000","last_clean":"2023-07-21T22:21:06.239002+0000","last_became_active":"2023-07-21T22:19:07.369156+0000","last_became_peered":"2023-07-21T22:19:07.369156+0000","last_unstale":"2023-07-21T22:21:06.239002+0000","last_undegraded":"2023-07-21T22:21:06.239002+0000","last_fullsized":"2023-07-21T22:21:06.239002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:18:11.706021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241220+0000","last_change":"2023-07-21T22:19:07.376798+0000","last_active":"2023-07-21T22:21:06.241220+0000","last_peered":"2023-07-21T22:21:06.241220+0000","last_clean":"2023-07-21T22:21:06.241220+0000","last_became_active":"2023-07-21T22:19:07.376466+0000","last_became_peered":"2023-07-21T22:19:07.376466+0000","last_unstale":"2023-07-21T22:21:06.241220+0000","last_undegraded":"2023-07-21T22:21:06.241220+0000","last_fullsized":"2023-07-21T22:21:06.241220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:36:12.418124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244777+0000","last_change":"2023-07-21T22:19:07.369347+0000","last_active":"2023-07-21T22:21:06.244777+0000","last_peered":"2023-07-21T22:21:06.244777+0000","last_clean":"2023-07-21T22:21:06.244777+0000","last_became_active":"2023-07-21T22:19:07.368898+0000","last_became_peered":"2023-07-21T22:19:07.368898+0000","last_unstale":"2023-07-21T22:21:06.244777+0000","last_undegraded":"2023-07-21T22:21:06.244777+0000","last_fullsized":"2023-07-21T22:21:06.244777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:49:28.752633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.231962+0000","last_change":"2023-07-21T22:19:07.370128+0000","last_active":"2023-07-21T22:21:06.231962+0000","last_peered":"2023-07-21T22:21:06.231962+0000","last_clean":"2023-07-21T22:21:06.231962+0000","last_became_active":"2023-07-21T22:19:07.369248+0000","last_became_peered":"2023-07-21T22:19:07.369248+0000","last_unstale":"2023-07-21T22:21:06.231962+0000","last_undegraded":"2023-07-21T22:21:06.231962+0000","last_fullsized":"2023-07-21T22:21:06.231962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:10.307052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.238638+0000","last_change":"2023-07-21T22:19:07.380668+0000","last_active":"2023-07-21T22:21:06.238638+0000","last_peered":"2023-07-21T22:21:06.238638+0000","last_clean":"2023-07-21T22:21:06.238638+0000","last_became_active":"2023-07-21T22:19:07.380384+0000","last_became_peered":"2023-07-21T22:19:07.380384+0000","last_unstale":"2023-07-21T22:21:06.238638+0000","last_undegraded":"2023-07-21T22:21:06.238638+0000","last_fullsized":"2023-07-21T22:21:06.238638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:42:34.571283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241211+0000","last_change":"2023-07-21T22:19:07.376013+0000","last_active":"2023-07-21T22:21:06.241211+0000","last_peered":"2023-07-21T22:21:06.241211+0000","last_clean":"2023-07-21T22:21:06.241211+0000","last_became_active":"2023-07-21T22:19:07.375705+0000","last_became_peered":"2023-07-21T22:19:07.375705+0000","last_unstale":"2023-07-21T22:21:06.241211+0000","last_undegraded":"2023-07-21T22:21:06.241211+0000","last_fullsized":"2023-07-21T22:21:06.241211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:56:26.017768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233052+0000","last_change":"2023-07-21T22:19:07.377569+0000","last_active":"2023-07-21T22:21:06.233052+0000","last_peered":"2023-07-21T22:21:06.233052+0000","last_clean":"2023-07-21T22:21:06.233052+0000","last_became_active":"2023-07-21T22:19:07.377182+0000","last_became_peered":"2023-07-21T22:19:07.377182+0000","last_unstale":"2023-07-21T22:21:06.233052+0000","last_undegraded":"2023-07-21T22:21:06.233052+0000","last_fullsized":"2023-07-21T22:21:06.233052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:39:12.625342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243697+0000","last_change":"2023-07-21T22:19:07.378953+0000","last_active":"2023-07-21T22:21:06.243697+0000","last_peered":"2023-07-21T22:21:06.243697+0000","last_clean":"2023-07-21T22:21:06.243697+0000","last_became_active":"2023-07-21T22:19:07.378674+0000","last_became_peered":"2023-07-21T22:19:07.378674+0000","last_unstale":"2023-07-21T22:21:06.243697+0000","last_undegraded":"2023-07-21T22:21:06.243697+0000","last_fullsized":"2023-07-21T22:21:06.243697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:22:55.560292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236208+0000","last_change":"2023-07-21T22:19:07.367506+0000","last_active":"2023-07-21T22:21:06.236208+0000","last_peered":"2023-07-21T22:21:06.236208+0000","last_clean":"2023-07-21T22:21:06.236208+0000","last_became_active":"2023-07-21T22:19:07.367191+0000","last_became_peered":"2023-07-21T22:19:07.367191+0000","last_unstale":"2023-07-21T22:21:06.236208+0000","last_undegraded":"2023-07-21T22:21:06.236208+0000","last_fullsized":"2023-07-21T22:21:06.236208+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:25:01.311975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241324+0000","last_change":"2023-07-21T22:19:07.375282+0000","last_active":"2023-07-21T22:21:06.241324+0000","last_peered":"2023-07-21T22:21:06.241324+0000","last_clean":"2023-07-21T22:21:06.241324+0000","last_became_active":"2023-07-21T22:19:07.374996+0000","last_became_peered":"2023-07-21T22:19:07.374996+0000","last_unstale":"2023-07-21T22:21:06.241324+0000","last_undegraded":"2023-07-21T22:21:06.241324+0000","last_fullsized":"2023-07-21T22:21:06.241324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:45:28.789239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229830+0000","last_change":"2023-07-21T22:19:07.371505+0000","last_active":"2023-07-21T22:21:06.229830+0000","last_peered":"2023-07-21T22:21:06.229830+0000","last_clean":"2023-07-21T22:21:06.229830+0000","last_became_active":"2023-07-21T22:19:07.370925+0000","last_became_peered":"2023-07-21T22:19:07.370925+0000","last_unstale":"2023-07-21T22:21:06.229830+0000","last_undegraded":"2023-07-21T22:21:06.229830+0000","last_fullsized":"2023-07-21T22:21:06.229830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:42.902545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233939+0000","last_change":"2023-07-21T22:19:07.371451+0000","last_active":"2023-07-21T22:21:06.233939+0000","last_peered":"2023-07-21T22:21:06.233939+0000","last_clean":"2023-07-21T22:21:06.233939+0000","last_became_active":"2023-07-21T22:19:07.371162+0000","last_became_peered":"2023-07-21T22:19:07.371162+0000","last_unstale":"2023-07-21T22:21:06.233939+0000","last_undegraded":"2023-07-21T22:21:06.233939+0000","last_fullsized":"2023-07-21T22:21:06.233939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:04:05.043773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240104+0000","last_change":"2023-07-21T22:19:07.372976+0000","last_active":"2023-07-21T22:21:06.240104+0000","last_peered":"2023-07-21T22:21:06.240104+0000","last_clean":"2023-07-21T22:21:06.240104+0000","last_became_active":"2023-07-21T22:19:07.369043+0000","last_became_peered":"2023-07-21T22:19:07.369043+0000","last_unstale":"2023-07-21T22:21:06.240104+0000","last_undegraded":"2023-07-21T22:21:06.240104+0000","last_fullsized":"2023-07-21T22:21:06.240104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:42:32.982654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244272+0000","last_change":"2023-07-21T22:19:07.367846+0000","last_active":"2023-07-21T22:21:06.244272+0000","last_peered":"2023-07-21T22:21:06.244272+0000","last_clean":"2023-07-21T22:21:06.244272+0000","last_became_active":"2023-07-21T22:19:07.367479+0000","last_became_peered":"2023-07-21T22:19:07.367479+0000","last_unstale":"2023-07-21T22:21:06.244272+0000","last_undegraded":"2023-07-21T22:21:06.244272+0000","last_fullsized":"2023-07-21T22:21:06.244272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:42.745730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.241909+0000","last_change":"2023-07-21T22:19:07.375479+0000","last_active":"2023-07-21T22:21:06.241909+0000","last_peered":"2023-07-21T22:21:06.241909+0000","last_clean":"2023-07-21T22:21:06.241909+0000","last_became_active":"2023-07-21T22:19:07.375091+0000","last_became_peered":"2023-07-21T22:19:07.375091+0000","last_unstale":"2023-07-21T22:21:06.241909+0000","last_undegraded":"2023-07-21T22:21:06.241909+0000","last_fullsized":"2023-07-21T22:21:06.241909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:19:27.316051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243456+0000","last_change":"2023-07-21T22:19:07.370214+0000","last_active":"2023-07-21T22:21:06.243456+0000","last_peered":"2023-07-21T22:21:06.243456+0000","last_clean":"2023-07-21T22:21:06.243456+0000","last_became_active":"2023-07-21T22:19:07.369941+0000","last_became_peered":"2023-07-21T22:19:07.369941+0000","last_unstale":"2023-07-21T22:21:06.243456+0000","last_undegraded":"2023-07-21T22:21:06.243456+0000","last_fullsized":"2023-07-21T22:21:06.243456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:33:43.668214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.239421+0000","last_change":"2023-07-21T22:19:07.369491+0000","last_active":"2023-07-21T22:21:06.239421+0000","last_peered":"2023-07-21T22:21:06.239421+0000","last_clean":"2023-07-21T22:21:06.239421+0000","last_became_active":"2023-07-21T22:19:07.369204+0000","last_became_peered":"2023-07-21T22:19:07.369204+0000","last_unstale":"2023-07-21T22:21:06.239421+0000","last_undegraded":"2023-07-21T22:21:06.239421+0000","last_fullsized":"2023-07-21T22:21:06.239421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:38:36.512107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232295+0000","last_change":"2023-07-21T22:19:07.378378+0000","last_active":"2023-07-21T22:21:06.232295+0000","last_peered":"2023-07-21T22:21:06.232295+0000","last_clean":"2023-07-21T22:21:06.232295+0000","last_became_active":"2023-07-21T22:19:07.378017+0000","last_became_peered":"2023-07-21T22:19:07.378017+0000","last_unstale":"2023-07-21T22:21:06.232295+0000","last_undegraded":"2023-07-21T22:21:06.232295+0000","last_fullsized":"2023-07-21T22:21:06.232295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:50:01.043172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.210433+0000","last_change":"2023-07-21T22:19:07.380319+0000","last_active":"2023-07-21T22:21:06.210433+0000","last_peered":"2023-07-21T22:21:06.210433+0000","last_clean":"2023-07-21T22:21:06.210433+0000","last_became_active":"2023-07-21T22:19:07.380019+0000","last_became_peered":"2023-07-21T22:19:07.380019+0000","last_unstale":"2023-07-21T22:21:06.210433+0000","last_undegraded":"2023-07-21T22:21:06.210433+0000","last_fullsized":"2023-07-21T22:21:06.210433+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:27:10.034128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305676+0000","last_change":"2023-07-21T22:19:11.387978+0000","last_active":"2023-07-21T22:20:38.305676+0000","last_peered":"2023-07-21T22:20:38.305676+0000","last_clean":"2023-07-21T22:20:38.305676+0000","last_became_active":"2023-07-21T22:19:11.387558+0000","last_became_peered":"2023-07-21T22:19:11.387558+0000","last_unstale":"2023-07-21T22:20:38.305676+0000","last_undegraded":"2023-07-21T22:20:38.305676+0000","last_fullsized":"2023-07-21T22:20:38.305676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:52.425113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298198+0000","last_change":"2023-07-21T22:19:06.368957+0000","last_active":"2023-07-21T22:20:38.298198+0000","last_peered":"2023-07-21T22:20:38.298198+0000","last_clean":"2023-07-21T22:20:38.298198+0000","last_became_active":"2023-07-21T22:19:06.368534+0000","last_became_peered":"2023-07-21T22:19:06.368534+0000","last_unstale":"2023-07-21T22:20:38.298198+0000","last_undegraded":"2023-07-21T22:20:38.298198+0000","last_fullsized":"2023-07-21T22:20:38.298198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243894+0000","last_change":"2023-07-21T22:19:07.380251+0000","last_active":"2023-07-21T22:21:06.243894+0000","last_peered":"2023-07-21T22:21:06.243894+0000","last_clean":"2023-07-21T22:21:06.243894+0000","last_became_active":"2023-07-21T22:19:07.379931+0000","last_became_peered":"2023-07-21T22:19:07.379931+0000","last_unstale":"2023-07-21T22:21:06.243894+0000","last_undegraded":"2023-07-21T22:21:06.243894+0000","last_fullsized":"2023-07-21T22:21:06.243894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:23:06.576092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.417884+0000","last_change":"2023-07-21T22:19:11.394186+0000","last_active":"2023-07-21T22:20:38.417884+0000","last_peered":"2023-07-21T22:20:38.417884+0000","last_clean":"2023-07-21T22:20:38.417884+0000","last_became_active":"2023-07-21T22:19:11.393806+0000","last_became_peered":"2023-07-21T22:19:11.393806+0000","last_unstale":"2023-07-21T22:20:38.417884+0000","last_undegraded":"2023-07-21T22:20:38.417884+0000","last_fullsized":"2023-07-21T22:20:38.417884+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:32:22.146374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.660121+0000","last_change":"2023-07-21T22:19:06.376770+0000","last_active":"2023-07-21T22:20:38.660121+0000","last_peered":"2023-07-21T22:20:38.660121+0000","last_clean":"2023-07-21T22:20:38.660121+0000","last_became_active":"2023-07-21T22:19:06.376202+0000","last_became_peered":"2023-07-21T22:19:06.376202+0000","last_unstale":"2023-07-21T22:20:38.660121+0000","last_undegraded":"2023-07-21T22:20:38.660121+0000","last_fullsized":"2023-07-21T22:20:38.660121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:01:15.547748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.246264+0000","last_change":"2023-07-21T22:19:07.380240+0000","last_active":"2023-07-21T22:21:06.246264+0000","last_peered":"2023-07-21T22:21:06.246264+0000","last_clean":"2023-07-21T22:21:06.246264+0000","last_became_active":"2023-07-21T22:19:07.379874+0000","last_became_peered":"2023-07-21T22:19:07.379874+0000","last_unstale":"2023-07-21T22:21:06.246264+0000","last_undegraded":"2023-07-21T22:21:06.246264+0000","last_fullsized":"2023-07-21T22:21:06.246264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:37.797343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418128+0000","last_change":"2023-07-21T22:19:11.393079+0000","last_active":"2023-07-21T22:20:38.418128+0000","last_peered":"2023-07-21T22:20:38.418128+0000","last_clean":"2023-07-21T22:20:38.418128+0000","last_became_active":"2023-07-21T22:19:11.392699+0000","last_became_peered":"2023-07-21T22:19:11.392699+0000","last_unstale":"2023-07-21T22:20:38.418128+0000","last_undegraded":"2023-07-21T22:20:38.418128+0000","last_fullsized":"2023-07-21T22:20:38.418128+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:33:12.532381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658825+0000","last_change":"2023-07-21T22:19:06.367960+0000","last_active":"2023-07-21T22:20:38.658825+0000","last_peered":"2023-07-21T22:20:38.658825+0000","last_clean":"2023-07-21T22:20:38.658825+0000","last_became_active":"2023-07-21T22:19:06.367671+0000","last_became_peered":"2023-07-21T22:19:06.367671+0000","last_unstale":"2023-07-21T22:20:38.658825+0000","last_undegraded":"2023-07-21T22:20:38.658825+0000","last_fullsized":"2023-07-21T22:20:38.658825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:42:37.381662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243097+0000","last_change":"2023-07-21T22:19:07.364302+0000","last_active":"2023-07-21T22:21:06.243097+0000","last_peered":"2023-07-21T22:21:06.243097+0000","last_clean":"2023-07-21T22:21:06.243097+0000","last_became_active":"2023-07-21T22:19:07.364010+0000","last_became_peered":"2023-07-21T22:19:07.364010+0000","last_unstale":"2023-07-21T22:21:06.243097+0000","last_undegraded":"2023-07-21T22:21:06.243097+0000","last_fullsized":"2023-07-21T22:21:06.243097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:35:44.049591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418190+0000","last_change":"2023-07-21T22:19:11.394165+0000","last_active":"2023-07-21T22:20:38.418190+0000","last_peered":"2023-07-21T22:20:38.418190+0000","last_clean":"2023-07-21T22:20:38.418190+0000","last_became_active":"2023-07-21T22:19:11.393761+0000","last_became_peered":"2023-07-21T22:19:11.393761+0000","last_unstale":"2023-07-21T22:20:38.418190+0000","last_undegraded":"2023-07-21T22:20:38.418190+0000","last_fullsized":"2023-07-21T22:20:38.418190+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:53:46.778497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658900+0000","last_change":"2023-07-21T22:19:06.371320+0000","last_active":"2023-07-21T22:20:38.658900+0000","last_peered":"2023-07-21T22:20:38.658900+0000","last_clean":"2023-07-21T22:20:38.658900+0000","last_became_active":"2023-07-21T22:19:06.371031+0000","last_became_peered":"2023-07-21T22:19:06.371031+0000","last_unstale":"2023-07-21T22:20:38.658900+0000","last_undegraded":"2023-07-21T22:20:38.658900+0000","last_fullsized":"2023-07-21T22:20:38.658900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:41:42.378836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":49,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229401+0000","last_change":"2023-07-21T22:20:38.439768+0000","last_active":"2023-07-21T22:21:06.229401+0000","last_peered":"2023-07-21T22:21:06.229401+0000","last_clean":"2023-07-21T22:21:06.229401+0000","last_became_active":"2023-07-21T22:20:38.439322+0000","last_became_peered":"2023-07-21T22:20:38.439322+0000","last_unstale":"2023-07-21T22:21:06.229401+0000","last_undegraded":"2023-07-21T22:21:06.229401+0000","last_fullsized":"2023-07-21T22:21:06.229401+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:07.041475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305701+0000","last_change":"2023-07-21T22:19:37.404292+0000","last_active":"2023-07-21T22:20:38.305701+0000","last_peered":"2023-07-21T22:20:38.305701+0000","last_clean":"2023-07-21T22:20:38.305701+0000","last_became_active":"2023-07-21T22:19:37.403826+0000","last_became_peered":"2023-07-21T22:19:37.403826+0000","last_unstale":"2023-07-21T22:20:38.305701+0000","last_undegraded":"2023-07-21T22:20:38.305701+0000","last_fullsized":"2023-07-21T22:20:38.305701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:31:58.935037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":141,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:10.459881+0000","last_change":"2023-07-21T22:19:06.374128+0000","last_active":"2023-07-21T22:21:10.459881+0000","last_peered":"2023-07-21T22:21:10.459881+0000","last_clean":"2023-07-21T22:21:10.459881+0000","last_became_active":"2023-07-21T22:19:06.373800+0000","last_became_peered":"2023-07-21T22:19:06.373800+0000","last_unstale":"2023-07-21T22:21:10.459881+0000","last_undegraded":"2023-07-21T22:21:10.459881+0000","last_fullsized":"2023-07-21T22:21:10.459881+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:59:08.289972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1421,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.235446+0000","last_change":"2023-07-21T22:19:07.375461+0000","last_active":"2023-07-21T22:21:06.235446+0000","last_peered":"2023-07-21T22:21:06.235446+0000","last_clean":"2023-07-21T22:21:06.235446+0000","last_became_active":"2023-07-21T22:19:07.375147+0000","last_became_peered":"2023-07-21T22:19:07.375147+0000","last_unstale":"2023-07-21T22:21:06.235446+0000","last_undegraded":"2023-07-21T22:21:06.235446+0000","last_fullsized":"2023-07-21T22:21:06.235446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:46:49.373988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305565+0000","last_change":"2023-07-21T22:19:11.396877+0000","last_active":"2023-07-21T22:20:38.305565+0000","last_peered":"2023-07-21T22:20:38.305565+0000","last_clean":"2023-07-21T22:20:38.305565+0000","last_became_active":"2023-07-21T22:19:11.396339+0000","last_became_peered":"2023-07-21T22:19:11.396339+0000","last_unstale":"2023-07-21T22:20:38.305565+0000","last_undegraded":"2023-07-21T22:20:38.305565+0000","last_fullsized":"2023-07-21T22:20:38.305565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:38:55.810627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298031+0000","last_change":"2023-07-21T22:19:37.328942+0000","last_active":"2023-07-21T22:20:38.298031+0000","last_peered":"2023-07-21T22:20:38.298031+0000","last_clean":"2023-07-21T22:20:38.298031+0000","last_became_active":"2023-07-21T22:19:37.325738+0000","last_became_peered":"2023-07-21T22:19:37.325738+0000","last_unstale":"2023-07-21T22:20:38.298031+0000","last_undegraded":"2023-07-21T22:20:38.298031+0000","last_fullsized":"2023-07-21T22:20:38.298031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:18:17.573518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230305+0000","last_change":"2023-07-21T22:19:07.373813+0000","last_active":"2023-07-21T22:21:06.230305+0000","last_peered":"2023-07-21T22:21:06.230305+0000","last_clean":"2023-07-21T22:21:06.230305+0000","last_became_active":"2023-07-21T22:19:07.373467+0000","last_became_peered":"2023-07-21T22:19:07.373467+0000","last_unstale":"2023-07-21T22:21:06.230305+0000","last_undegraded":"2023-07-21T22:21:06.230305+0000","last_fullsized":"2023-07-21T22:21:06.230305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:08:02.749178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.299090+0000","last_change":"2023-07-21T22:19:37.326119+0000","last_active":"2023-07-21T22:20:38.299090+0000","last_peered":"2023-07-21T22:20:38.299090+0000","last_clean":"2023-07-21T22:20:38.299090+0000","last_became_active":"2023-07-21T22:19:37.325675+0000","last_became_peered":"2023-07-21T22:19:37.325675+0000","last_unstale":"2023-07-21T22:20:38.299090+0000","last_undegraded":"2023-07-21T22:20:38.299090+0000","last_fullsized":"2023-07-21T22:20:38.299090+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:44:14.465291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295247+0000","last_change":"2023-07-21T22:19:06.368220+0000","last_active":"2023-07-21T22:20:38.295247+0000","last_peered":"2023-07-21T22:20:38.295247+0000","last_clean":"2023-07-21T22:20:38.295247+0000","last_became_active":"2023-07-21T22:19:06.367853+0000","last_became_peered":"2023-07-21T22:19:06.367853+0000","last_unstale":"2023-07-21T22:20:38.295247+0000","last_undegraded":"2023-07-21T22:20:38.295247+0000","last_fullsized":"2023-07-21T22:20:38.295247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:41:42.444573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.242024+0000","last_change":"2023-07-21T22:19:07.381055+0000","last_active":"2023-07-21T22:21:06.242024+0000","last_peered":"2023-07-21T22:21:06.242024+0000","last_clean":"2023-07-21T22:21:06.242024+0000","last_became_active":"2023-07-21T22:19:07.380678+0000","last_became_peered":"2023-07-21T22:19:07.380678+0000","last_unstale":"2023-07-21T22:21:06.242024+0000","last_undegraded":"2023-07-21T22:21:06.242024+0000","last_fullsized":"2023-07-21T22:21:06.242024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:32:03.234308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.660186+0000","last_change":"2023-07-21T22:19:11.395609+0000","last_active":"2023-07-21T22:20:38.660186+0000","last_peered":"2023-07-21T22:20:38.660186+0000","last_clean":"2023-07-21T22:20:38.660186+0000","last_became_active":"2023-07-21T22:19:11.395217+0000","last_became_peered":"2023-07-21T22:19:11.395217+0000","last_unstale":"2023-07-21T22:20:38.660186+0000","last_undegraded":"2023-07-21T22:20:38.660186+0000","last_fullsized":"2023-07-21T22:20:38.660186+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:23:22.856670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296564+0000","last_change":"2023-07-21T22:19:06.370281+0000","last_active":"2023-07-21T22:20:38.296564+0000","last_peered":"2023-07-21T22:20:38.296564+0000","last_clean":"2023-07-21T22:20:38.296564+0000","last_became_active":"2023-07-21T22:19:06.369925+0000","last_became_peered":"2023-07-21T22:19:06.369925+0000","last_unstale":"2023-07-21T22:20:38.296564+0000","last_undegraded":"2023-07-21T22:20:38.296564+0000","last_fullsized":"2023-07-21T22:20:38.296564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:13:30.074340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233367+0000","last_change":"2023-07-21T22:19:07.383851+0000","last_active":"2023-07-21T22:21:06.233367+0000","last_peered":"2023-07-21T22:21:06.233367+0000","last_clean":"2023-07-21T22:21:06.233367+0000","last_became_active":"2023-07-21T22:19:07.383260+0000","last_became_peered":"2023-07-21T22:19:07.383260+0000","last_unstale":"2023-07-21T22:21:06.233367+0000","last_undegraded":"2023-07-21T22:21:06.233367+0000","last_fullsized":"2023-07-21T22:21:06.233367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:58:03.385071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659286+0000","last_change":"2023-07-21T22:19:11.394347+0000","last_active":"2023-07-21T22:20:38.659286+0000","last_peered":"2023-07-21T22:20:38.659286+0000","last_clean":"2023-07-21T22:20:38.659286+0000","last_became_active":"2023-07-21T22:19:11.393515+0000","last_became_peered":"2023-07-21T22:19:11.393515+0000","last_unstale":"2023-07-21T22:20:38.659286+0000","last_undegraded":"2023-07-21T22:20:38.659286+0000","last_fullsized":"2023-07-21T22:20:38.659286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:37.149270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":133,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:11.460791+0000","last_change":"2023-07-21T22:19:06.369377+0000","last_active":"2023-07-21T22:21:11.460791+0000","last_peered":"2023-07-21T22:21:11.460791+0000","last_clean":"2023-07-21T22:21:11.460791+0000","last_became_active":"2023-07-21T22:19:06.369069+0000","last_became_peered":"2023-07-21T22:19:06.369069+0000","last_unstale":"2023-07-21T22:21:11.460791+0000","last_undegraded":"2023-07-21T22:21:11.460791+0000","last_fullsized":"2023-07-21T22:21:11.460791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:18:00.045946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.235609+0000","last_change":"2023-07-21T22:19:07.375370+0000","last_active":"2023-07-21T22:21:06.235609+0000","last_peered":"2023-07-21T22:21:06.235609+0000","last_clean":"2023-07-21T22:21:06.235609+0000","last_became_active":"2023-07-21T22:19:07.375045+0000","last_became_peered":"2023-07-21T22:19:07.375045+0000","last_unstale":"2023-07-21T22:21:06.235609+0000","last_undegraded":"2023-07-21T22:21:06.235609+0000","last_fullsized":"2023-07-21T22:21:06.235609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:24:37.041577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297787+0000","last_change":"2023-07-21T22:19:37.326317+0000","last_active":"2023-07-21T22:20:38.297787+0000","last_peered":"2023-07-21T22:20:38.297787+0000","last_clean":"2023-07-21T22:20:38.297787+0000","last_became_active":"2023-07-21T22:19:37.325791+0000","last_became_peered":"2023-07-21T22:19:37.325791+0000","last_unstale":"2023-07-21T22:20:38.297787+0000","last_undegraded":"2023-07-21T22:20:38.297787+0000","last_fullsized":"2023-07-21T22:20:38.297787+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:24:28.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,0,2,4],"acting":[10,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659430+0000","last_change":"2023-07-21T22:19:06.371422+0000","last_active":"2023-07-21T22:20:38.659430+0000","last_peered":"2023-07-21T22:20:38.659430+0000","last_clean":"2023-07-21T22:20:38.659430+0000","last_became_active":"2023-07-21T22:19:06.371094+0000","last_became_peered":"2023-07-21T22:19:06.371094+0000","last_unstale":"2023-07-21T22:20:38.659430+0000","last_undegraded":"2023-07-21T22:20:38.659430+0000","last_fullsized":"2023-07-21T22:20:38.659430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:08.684209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.246230+0000","last_change":"2023-07-21T22:19:07.381779+0000","last_active":"2023-07-21T22:21:06.246230+0000","last_peered":"2023-07-21T22:21:06.246230+0000","last_clean":"2023-07-21T22:21:06.246230+0000","last_became_active":"2023-07-21T22:19:07.381386+0000","last_became_peered":"2023-07-21T22:19:07.381386+0000","last_unstale":"2023-07-21T22:21:06.246230+0000","last_undegraded":"2023-07-21T22:21:06.246230+0000","last_fullsized":"2023-07-21T22:21:06.246230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:48.990870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418435+0000","last_change":"2023-07-21T22:19:11.389753+0000","last_active":"2023-07-21T22:20:38.418435+0000","last_peered":"2023-07-21T22:20:38.418435+0000","last_clean":"2023-07-21T22:20:38.418435+0000","last_became_active":"2023-07-21T22:19:11.389200+0000","last_became_peered":"2023-07-21T22:19:11.389200+0000","last_unstale":"2023-07-21T22:20:38.418435+0000","last_undegraded":"2023-07-21T22:20:38.418435+0000","last_fullsized":"2023-07-21T22:20:38.418435+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:20:47.024250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296928+0000","last_change":"2023-07-21T22:19:06.374122+0000","last_active":"2023-07-21T22:20:38.296928+0000","last_peered":"2023-07-21T22:20:38.296928+0000","last_clean":"2023-07-21T22:20:38.296928+0000","last_became_active":"2023-07-21T22:19:06.373844+0000","last_became_peered":"2023-07-21T22:19:06.373844+0000","last_unstale":"2023-07-21T22:20:38.296928+0000","last_undegraded":"2023-07-21T22:20:38.296928+0000","last_fullsized":"2023-07-21T22:20:38.296928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:54:48.614307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243632+0000","last_change":"2023-07-21T22:19:07.371200+0000","last_active":"2023-07-21T22:21:06.243632+0000","last_peered":"2023-07-21T22:21:06.243632+0000","last_clean":"2023-07-21T22:21:06.243632+0000","last_became_active":"2023-07-21T22:19:07.370887+0000","last_became_peered":"2023-07-21T22:19:07.370887+0000","last_unstale":"2023-07-21T22:21:06.243632+0000","last_undegraded":"2023-07-21T22:21:06.243632+0000","last_fullsized":"2023-07-21T22:21:06.243632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:11:59.106308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297543+0000","last_change":"2023-07-21T22:19:11.392222+0000","last_active":"2023-07-21T22:20:38.297543+0000","last_peered":"2023-07-21T22:20:38.297543+0000","last_clean":"2023-07-21T22:20:38.297543+0000","last_became_active":"2023-07-21T22:19:11.390186+0000","last_became_peered":"2023-07-21T22:19:11.390186+0000","last_unstale":"2023-07-21T22:20:38.297543+0000","last_undegraded":"2023-07-21T22:20:38.297543+0000","last_fullsized":"2023-07-21T22:20:38.297543+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:08:44.231916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"93'35","reported_seq":273,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:12.463095+0000","last_change":"2023-07-21T22:19:06.372927+0000","last_active":"2023-07-21T22:21:12.463095+0000","last_peered":"2023-07-21T22:21:12.463095+0000","last_clean":"2023-07-21T22:21:12.463095+0000","last_became_active":"2023-07-21T22:19:06.372403+0000","last_became_peered":"2023-07-21T22:19:06.372403+0000","last_unstale":"2023-07-21T22:21:12.463095+0000","last_undegraded":"2023-07-21T22:21:12.463095+0000","last_fullsized":"2023-07-21T22:21:12.463095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:30.817175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63587,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":208,"num_read_kb":166,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.228970+0000","last_change":"2023-07-21T22:19:07.370984+0000","last_active":"2023-07-21T22:21:06.228970+0000","last_peered":"2023-07-21T22:21:06.228970+0000","last_clean":"2023-07-21T22:21:06.228970+0000","last_became_active":"2023-07-21T22:19:07.370687+0000","last_became_peered":"2023-07-21T22:19:07.370687+0000","last_unstale":"2023-07-21T22:21:06.228970+0000","last_undegraded":"2023-07-21T22:21:06.228970+0000","last_fullsized":"2023-07-21T22:21:06.228970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:04:11.923390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297581+0000","last_change":"2023-07-21T22:19:11.393710+0000","last_active":"2023-07-21T22:20:38.297581+0000","last_peered":"2023-07-21T22:20:38.297581+0000","last_clean":"2023-07-21T22:20:38.297581+0000","last_became_active":"2023-07-21T22:19:11.393313+0000","last_became_peered":"2023-07-21T22:19:11.393313+0000","last_unstale":"2023-07-21T22:20:38.297581+0000","last_undegraded":"2023-07-21T22:20:38.297581+0000","last_fullsized":"2023-07-21T22:20:38.297581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:24:31.302884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305466+0000","last_change":"2023-07-21T22:19:06.370627+0000","last_active":"2023-07-21T22:20:38.305466+0000","last_peered":"2023-07-21T22:20:38.305466+0000","last_clean":"2023-07-21T22:20:38.305466+0000","last_became_active":"2023-07-21T22:19:06.370334+0000","last_became_peered":"2023-07-21T22:19:06.370334+0000","last_unstale":"2023-07-21T22:20:38.305466+0000","last_undegraded":"2023-07-21T22:20:38.305466+0000","last_fullsized":"2023-07-21T22:20:38.305466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:21.244524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245019+0000","last_change":"2023-07-21T22:19:07.370482+0000","last_active":"2023-07-21T22:21:06.245019+0000","last_peered":"2023-07-21T22:21:06.245019+0000","last_clean":"2023-07-21T22:21:06.245019+0000","last_became_active":"2023-07-21T22:19:07.370086+0000","last_became_peered":"2023-07-21T22:19:07.370086+0000","last_unstale":"2023-07-21T22:21:06.245019+0000","last_undegraded":"2023-07-21T22:21:06.245019+0000","last_fullsized":"2023-07-21T22:21:06.245019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:37.737964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298975+0000","last_change":"2023-07-21T22:19:11.389777+0000","last_active":"2023-07-21T22:20:38.298975+0000","last_peered":"2023-07-21T22:20:38.298975+0000","last_clean":"2023-07-21T22:20:38.298975+0000","last_became_active":"2023-07-21T22:19:11.389389+0000","last_became_peered":"2023-07-21T22:19:11.389389+0000","last_unstale":"2023-07-21T22:20:38.298975+0000","last_undegraded":"2023-07-21T22:20:38.298975+0000","last_fullsized":"2023-07-21T22:20:38.298975+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:10:07.260019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296895+0000","last_change":"2023-07-21T22:19:06.376714+0000","last_active":"2023-07-21T22:20:38.296895+0000","last_peered":"2023-07-21T22:20:38.296895+0000","last_clean":"2023-07-21T22:20:38.296895+0000","last_became_active":"2023-07-21T22:19:06.376413+0000","last_became_peered":"2023-07-21T22:19:06.376413+0000","last_unstale":"2023-07-21T22:20:38.296895+0000","last_undegraded":"2023-07-21T22:20:38.296895+0000","last_fullsized":"2023-07-21T22:20:38.296895+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:20:47.729624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232411+0000","last_change":"2023-07-21T22:19:07.376452+0000","last_active":"2023-07-21T22:21:06.232411+0000","last_peered":"2023-07-21T22:21:06.232411+0000","last_clean":"2023-07-21T22:21:06.232411+0000","last_became_active":"2023-07-21T22:19:07.376149+0000","last_became_peered":"2023-07-21T22:19:07.376149+0000","last_unstale":"2023-07-21T22:21:06.232411+0000","last_undegraded":"2023-07-21T22:21:06.232411+0000","last_fullsized":"2023-07-21T22:21:06.232411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:09.067826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658927+0000","last_change":"2023-07-21T22:19:37.403648+0000","last_active":"2023-07-21T22:20:38.658927+0000","last_peered":"2023-07-21T22:20:38.658927+0000","last_clean":"2023-07-21T22:20:38.658927+0000","last_became_active":"2023-07-21T22:19:37.403252+0000","last_became_peered":"2023-07-21T22:19:37.403252+0000","last_unstale":"2023-07-21T22:20:38.658927+0000","last_undegraded":"2023-07-21T22:20:38.658927+0000","last_fullsized":"2023-07-21T22:20:38.658927+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:55.264540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"93'5","reported_seq":136,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:09.474149+0000","last_change":"2023-07-21T22:19:06.369873+0000","last_active":"2023-07-21T22:21:09.474149+0000","last_peered":"2023-07-21T22:21:09.474149+0000","last_clean":"2023-07-21T22:21:09.474149+0000","last_became_active":"2023-07-21T22:19:06.369581+0000","last_became_peered":"2023-07-21T22:19:06.369581+0000","last_unstale":"2023-07-21T22:21:09.474149+0000","last_undegraded":"2023-07-21T22:21:09.474149+0000","last_fullsized":"2023-07-21T22:21:09.474149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:34:54.366109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.221294+0000","last_change":"2023-07-21T22:19:07.370715+0000","last_active":"2023-07-21T22:21:06.221294+0000","last_peered":"2023-07-21T22:21:06.221294+0000","last_clean":"2023-07-21T22:21:06.221294+0000","last_became_active":"2023-07-21T22:19:07.370403+0000","last_became_peered":"2023-07-21T22:19:07.370403+0000","last_unstale":"2023-07-21T22:21:06.221294+0000","last_undegraded":"2023-07-21T22:21:06.221294+0000","last_fullsized":"2023-07-21T22:21:06.221294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:29:04.512567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295715+0000","last_change":"2023-07-21T22:19:11.398294+0000","last_active":"2023-07-21T22:20:38.295715+0000","last_peered":"2023-07-21T22:20:38.295715+0000","last_clean":"2023-07-21T22:20:38.295715+0000","last_became_active":"2023-07-21T22:19:11.397896+0000","last_became_peered":"2023-07-21T22:19:11.397896+0000","last_unstale":"2023-07-21T22:20:38.295715+0000","last_undegraded":"2023-07-21T22:20:38.295715+0000","last_fullsized":"2023-07-21T22:20:38.295715+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:17:47.809370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658510+0000","last_change":"2023-07-21T22:19:06.372625+0000","last_active":"2023-07-21T22:20:38.658510+0000","last_peered":"2023-07-21T22:20:38.658510+0000","last_clean":"2023-07-21T22:20:38.658510+0000","last_became_active":"2023-07-21T22:19:06.372047+0000","last_became_peered":"2023-07-21T22:19:06.372047+0000","last_unstale":"2023-07-21T22:20:38.658510+0000","last_undegraded":"2023-07-21T22:20:38.658510+0000","last_fullsized":"2023-07-21T22:20:38.658510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:20:14.245284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230387+0000","last_change":"2023-07-21T22:19:07.373330+0000","last_active":"2023-07-21T22:21:06.230387+0000","last_peered":"2023-07-21T22:21:06.230387+0000","last_clean":"2023-07-21T22:21:06.230387+0000","last_became_active":"2023-07-21T22:19:07.373014+0000","last_became_peered":"2023-07-21T22:19:07.373014+0000","last_unstale":"2023-07-21T22:21:06.230387+0000","last_undegraded":"2023-07-21T22:21:06.230387+0000","last_fullsized":"2023-07-21T22:21:06.230387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:31:10.400632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296528+0000","last_change":"2023-07-21T22:19:11.387103+0000","last_active":"2023-07-21T22:20:38.296528+0000","last_peered":"2023-07-21T22:20:38.296528+0000","last_clean":"2023-07-21T22:20:38.296528+0000","last_became_active":"2023-07-21T22:19:11.386727+0000","last_became_peered":"2023-07-21T22:19:11.386727+0000","last_unstale":"2023-07-21T22:20:38.296528+0000","last_undegraded":"2023-07-21T22:20:38.296528+0000","last_fullsized":"2023-07-21T22:20:38.296528+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:36:16.962575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659163+0000","last_change":"2023-07-21T22:19:06.364504+0000","last_active":"2023-07-21T22:20:38.659163+0000","last_peered":"2023-07-21T22:20:38.659163+0000","last_clean":"2023-07-21T22:20:38.659163+0000","last_became_active":"2023-07-21T22:19:06.364182+0000","last_became_peered":"2023-07-21T22:19:06.364182+0000","last_unstale":"2023-07-21T22:20:38.659163+0000","last_undegraded":"2023-07-21T22:20:38.659163+0000","last_fullsized":"2023-07-21T22:20:38.659163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:11:28.407010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233495+0000","last_change":"2023-07-21T22:19:07.372596+0000","last_active":"2023-07-21T22:21:06.233495+0000","last_peered":"2023-07-21T22:21:06.233495+0000","last_clean":"2023-07-21T22:21:06.233495+0000","last_became_active":"2023-07-21T22:19:07.372285+0000","last_became_peered":"2023-07-21T22:19:07.372285+0000","last_unstale":"2023-07-21T22:21:06.233495+0000","last_undegraded":"2023-07-21T22:21:06.233495+0000","last_fullsized":"2023-07-21T22:21:06.233495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:19:30.865477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658592+0000","last_change":"2023-07-21T22:19:11.671595+0000","last_active":"2023-07-21T22:20:38.658592+0000","last_peered":"2023-07-21T22:20:38.658592+0000","last_clean":"2023-07-21T22:20:38.658592+0000","last_became_active":"2023-07-21T22:19:11.671176+0000","last_became_peered":"2023-07-21T22:19:11.671176+0000","last_unstale":"2023-07-21T22:20:38.658592+0000","last_undegraded":"2023-07-21T22:20:38.658592+0000","last_fullsized":"2023-07-21T22:20:38.658592+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:55:48.187244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297860+0000","last_change":"2023-07-21T22:19:37.312431+0000","last_active":"2023-07-21T22:20:38.297860+0000","last_peered":"2023-07-21T22:20:38.297860+0000","last_clean":"2023-07-21T22:20:38.297860+0000","last_became_active":"2023-07-21T22:19:37.312038+0000","last_became_peered":"2023-07-21T22:19:37.312038+0000","last_unstale":"2023-07-21T22:20:38.297860+0000","last_undegraded":"2023-07-21T22:20:38.297860+0000","last_fullsized":"2023-07-21T22:20:38.297860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:13:00.069534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244472+0000","last_change":"2023-07-21T22:19:07.382034+0000","last_active":"2023-07-21T22:21:06.244472+0000","last_peered":"2023-07-21T22:21:06.244472+0000","last_clean":"2023-07-21T22:21:06.244472+0000","last_became_active":"2023-07-21T22:19:07.381746+0000","last_became_peered":"2023-07-21T22:19:07.381746+0000","last_unstale":"2023-07-21T22:21:06.244472+0000","last_undegraded":"2023-07-21T22:21:06.244472+0000","last_fullsized":"2023-07-21T22:21:06.244472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:49.866646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295482+0000","last_change":"2023-07-21T22:19:11.391808+0000","last_active":"2023-07-21T22:20:38.295482+0000","last_peered":"2023-07-21T22:20:38.295482+0000","last_clean":"2023-07-21T22:20:38.295482+0000","last_became_active":"2023-07-21T22:19:11.390899+0000","last_became_peered":"2023-07-21T22:19:11.390899+0000","last_unstale":"2023-07-21T22:20:38.295482+0000","last_undegraded":"2023-07-21T22:20:38.295482+0000","last_fullsized":"2023-07-21T22:20:38.295482+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:32:51.240441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659099+0000","last_change":"2023-07-21T22:19:06.374310+0000","last_active":"2023-07-21T22:20:38.659099+0000","last_peered":"2023-07-21T22:20:38.659099+0000","last_clean":"2023-07-21T22:20:38.659099+0000","last_became_active":"2023-07-21T22:19:06.373984+0000","last_became_peered":"2023-07-21T22:19:06.373984+0000","last_unstale":"2023-07-21T22:20:38.659099+0000","last_undegraded":"2023-07-21T22:20:38.659099+0000","last_fullsized":"2023-07-21T22:20:38.659099+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:34:49.901496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237830+0000","last_change":"2023-07-21T22:19:07.374389+0000","last_active":"2023-07-21T22:21:06.237830+0000","last_peered":"2023-07-21T22:21:06.237830+0000","last_clean":"2023-07-21T22:21:06.237830+0000","last_became_active":"2023-07-21T22:19:07.374096+0000","last_became_peered":"2023-07-21T22:19:07.374096+0000","last_unstale":"2023-07-21T22:21:06.237830+0000","last_undegraded":"2023-07-21T22:21:06.237830+0000","last_fullsized":"2023-07-21T22:21:06.237830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:47.649059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295392+0000","last_change":"2023-07-21T22:19:11.672125+0000","last_active":"2023-07-21T22:20:38.295392+0000","last_peered":"2023-07-21T22:20:38.295392+0000","last_clean":"2023-07-21T22:20:38.295392+0000","last_became_active":"2023-07-21T22:19:11.671397+0000","last_became_peered":"2023-07-21T22:19:11.671397+0000","last_unstale":"2023-07-21T22:20:38.295392+0000","last_undegraded":"2023-07-21T22:20:38.295392+0000","last_fullsized":"2023-07-21T22:20:38.295392+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:53:12.871041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230716+0000","last_change":"2023-07-21T22:19:07.368203+0000","last_active":"2023-07-21T22:21:06.230716+0000","last_peered":"2023-07-21T22:21:06.230716+0000","last_clean":"2023-07-21T22:21:06.230716+0000","last_became_active":"2023-07-21T22:19:07.367917+0000","last_became_peered":"2023-07-21T22:19:07.367917+0000","last_unstale":"2023-07-21T22:21:06.230716+0000","last_undegraded":"2023-07-21T22:21:06.230716+0000","last_fullsized":"2023-07-21T22:21:06.230716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:07:20.967282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'5","reported_seq":135,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:12.478815+0000","last_change":"2023-07-21T22:19:06.380215+0000","last_active":"2023-07-21T22:21:12.478815+0000","last_peered":"2023-07-21T22:21:12.478815+0000","last_clean":"2023-07-21T22:21:12.478815+0000","last_became_active":"2023-07-21T22:19:06.379867+0000","last_became_peered":"2023-07-21T22:19:06.379867+0000","last_unstale":"2023-07-21T22:21:12.478815+0000","last_undegraded":"2023-07-21T22:21:12.478815+0000","last_fullsized":"2023-07-21T22:21:12.478815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:09:04.461436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659421+0000","last_change":"2023-07-21T22:19:11.381632+0000","last_active":"2023-07-21T22:20:38.659421+0000","last_peered":"2023-07-21T22:20:38.659421+0000","last_clean":"2023-07-21T22:20:38.659421+0000","last_became_active":"2023-07-21T22:19:11.381138+0000","last_became_peered":"2023-07-21T22:19:11.381138+0000","last_unstale":"2023-07-21T22:20:38.659421+0000","last_undegraded":"2023-07-21T22:20:38.659421+0000","last_fullsized":"2023-07-21T22:20:38.659421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:32:31.504021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243705+0000","last_change":"2023-07-21T22:19:07.383294+0000","last_active":"2023-07-21T22:21:06.243705+0000","last_peered":"2023-07-21T22:21:06.243705+0000","last_clean":"2023-07-21T22:21:06.243705+0000","last_became_active":"2023-07-21T22:19:07.382984+0000","last_became_peered":"2023-07-21T22:19:07.382984+0000","last_unstale":"2023-07-21T22:21:06.243705+0000","last_undegraded":"2023-07-21T22:21:06.243705+0000","last_fullsized":"2023-07-21T22:21:06.243705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:21:28.483320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418342+0000","last_change":"2023-07-21T22:19:06.365350+0000","last_active":"2023-07-21T22:20:38.418342+0000","last_peered":"2023-07-21T22:20:38.418342+0000","last_clean":"2023-07-21T22:20:38.418342+0000","last_became_active":"2023-07-21T22:19:06.365001+0000","last_became_peered":"2023-07-21T22:19:06.365001+0000","last_unstale":"2023-07-21T22:20:38.418342+0000","last_undegraded":"2023-07-21T22:20:38.418342+0000","last_fullsized":"2023-07-21T22:20:38.418342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:03:40.298453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658878+0000","last_change":"2023-07-21T22:18:01.213967+0000","last_active":"2023-07-21T22:20:38.658878+0000","last_peered":"2023-07-21T22:20:38.658878+0000","last_clean":"2023-07-21T22:20:38.658878+0000","last_became_active":"2023-07-21T22:18:01.213528+0000","last_became_peered":"2023-07-21T22:18:01.213528+0000","last_unstale":"2023-07-21T22:20:38.658878+0000","last_undegraded":"2023-07-21T22:20:38.658878+0000","last_fullsized":"2023-07-21T22:20:38.658878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:14:54.962562+0000","last_clean_scrub_stamp":"2023-07-21T22:14:54.962562+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:55:41.830722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297102+0000","last_change":"2023-07-21T22:19:11.672641+0000","last_active":"2023-07-21T22:20:38.297102+0000","last_peered":"2023-07-21T22:20:38.297102+0000","last_clean":"2023-07-21T22:20:38.297102+0000","last_became_active":"2023-07-21T22:19:11.672213+0000","last_became_peered":"2023-07-21T22:19:11.672213+0000","last_unstale":"2023-07-21T22:20:38.297102+0000","last_undegraded":"2023-07-21T22:20:38.297102+0000","last_fullsized":"2023-07-21T22:20:38.297102+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:39:48.308116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237467+0000","last_change":"2023-07-21T22:19:07.372537+0000","last_active":"2023-07-21T22:21:06.237467+0000","last_peered":"2023-07-21T22:21:06.237467+0000","last_clean":"2023-07-21T22:21:06.237467+0000","last_became_active":"2023-07-21T22:19:07.372156+0000","last_became_peered":"2023-07-21T22:19:07.372156+0000","last_unstale":"2023-07-21T22:21:06.237467+0000","last_undegraded":"2023-07-21T22:21:06.237467+0000","last_fullsized":"2023-07-21T22:21:06.237467+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:00:22.036485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658972+0000","last_change":"2023-07-21T22:19:06.368552+0000","last_active":"2023-07-21T22:20:38.658972+0000","last_peered":"2023-07-21T22:20:38.658972+0000","last_clean":"2023-07-21T22:20:38.658972+0000","last_became_active":"2023-07-21T22:19:06.368322+0000","last_became_peered":"2023-07-21T22:19:06.368322+0000","last_unstale":"2023-07-21T22:20:38.658972+0000","last_undegraded":"2023-07-21T22:20:38.658972+0000","last_fullsized":"2023-07-21T22:20:38.658972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:08:30.610746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298509+0000","last_change":"2023-07-21T22:19:11.384699+0000","last_active":"2023-07-21T22:20:38.298509+0000","last_peered":"2023-07-21T22:20:38.298509+0000","last_clean":"2023-07-21T22:20:38.298509+0000","last_became_active":"2023-07-21T22:19:11.384212+0000","last_became_peered":"2023-07-21T22:19:11.384212+0000","last_unstale":"2023-07-21T22:20:38.298509+0000","last_undegraded":"2023-07-21T22:20:38.298509+0000","last_fullsized":"2023-07-21T22:20:38.298509+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:51:03.929894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658954+0000","last_change":"2023-07-21T22:19:06.371507+0000","last_active":"2023-07-21T22:20:38.658954+0000","last_peered":"2023-07-21T22:20:38.658954+0000","last_clean":"2023-07-21T22:20:38.658954+0000","last_became_active":"2023-07-21T22:19:06.371226+0000","last_became_peered":"2023-07-21T22:19:06.371226+0000","last_unstale":"2023-07-21T22:20:38.658954+0000","last_undegraded":"2023-07-21T22:20:38.658954+0000","last_fullsized":"2023-07-21T22:20:38.658954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:49.193766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240119+0000","last_change":"2023-07-21T22:19:07.374799+0000","last_active":"2023-07-21T22:21:06.240119+0000","last_peered":"2023-07-21T22:21:06.240119+0000","last_clean":"2023-07-21T22:21:06.240119+0000","last_became_active":"2023-07-21T22:19:07.374461+0000","last_became_peered":"2023-07-21T22:19:07.374461+0000","last_unstale":"2023-07-21T22:21:06.240119+0000","last_undegraded":"2023-07-21T22:21:06.240119+0000","last_fullsized":"2023-07-21T22:21:06.240119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:45:33.312382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659732+0000","last_change":"2023-07-21T22:19:11.673022+0000","last_active":"2023-07-21T22:20:38.659732+0000","last_peered":"2023-07-21T22:20:38.659732+0000","last_clean":"2023-07-21T22:20:38.659732+0000","last_became_active":"2023-07-21T22:19:11.672291+0000","last_became_peered":"2023-07-21T22:19:11.672291+0000","last_unstale":"2023-07-21T22:20:38.659732+0000","last_undegraded":"2023-07-21T22:20:38.659732+0000","last_fullsized":"2023-07-21T22:20:38.659732+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:04:39.197537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298123+0000","last_change":"2023-07-21T22:19:06.369499+0000","last_active":"2023-07-21T22:20:38.298123+0000","last_peered":"2023-07-21T22:20:38.298123+0000","last_clean":"2023-07-21T22:20:38.298123+0000","last_became_active":"2023-07-21T22:19:06.369159+0000","last_became_peered":"2023-07-21T22:19:06.369159+0000","last_unstale":"2023-07-21T22:20:38.298123+0000","last_undegraded":"2023-07-21T22:20:38.298123+0000","last_fullsized":"2023-07-21T22:20:38.298123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:35:05.244612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243542+0000","last_change":"2023-07-21T22:19:07.382533+0000","last_active":"2023-07-21T22:21:06.243542+0000","last_peered":"2023-07-21T22:21:06.243542+0000","last_clean":"2023-07-21T22:21:06.243542+0000","last_became_active":"2023-07-21T22:19:07.381804+0000","last_became_peered":"2023-07-21T22:19:07.381804+0000","last_unstale":"2023-07-21T22:21:06.243542+0000","last_undegraded":"2023-07-21T22:21:06.243542+0000","last_fullsized":"2023-07-21T22:21:06.243542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:14:42.196678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305178+0000","last_change":"2023-07-21T22:19:11.381838+0000","last_active":"2023-07-21T22:20:38.305178+0000","last_peered":"2023-07-21T22:20:38.305178+0000","last_clean":"2023-07-21T22:20:38.305178+0000","last_became_active":"2023-07-21T22:19:11.381448+0000","last_became_peered":"2023-07-21T22:19:11.381448+0000","last_unstale":"2023-07-21T22:20:38.305178+0000","last_undegraded":"2023-07-21T22:20:38.305178+0000","last_fullsized":"2023-07-21T22:20:38.305178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:41:59.043778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418371+0000","last_change":"2023-07-21T22:19:06.370752+0000","last_active":"2023-07-21T22:20:38.418371+0000","last_peered":"2023-07-21T22:20:38.418371+0000","last_clean":"2023-07-21T22:20:38.418371+0000","last_became_active":"2023-07-21T22:19:06.370421+0000","last_became_peered":"2023-07-21T22:19:06.370421+0000","last_unstale":"2023-07-21T22:20:38.418371+0000","last_undegraded":"2023-07-21T22:20:38.418371+0000","last_fullsized":"2023-07-21T22:20:38.418371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:36:32.862088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245652+0000","last_change":"2023-07-21T22:19:37.404445+0000","last_active":"2023-07-21T22:21:06.245652+0000","last_peered":"2023-07-21T22:21:06.245652+0000","last_clean":"2023-07-21T22:21:06.245652+0000","last_became_active":"2023-07-21T22:19:37.404011+0000","last_became_peered":"2023-07-21T22:19:37.404011+0000","last_unstale":"2023-07-21T22:21:06.245652+0000","last_undegraded":"2023-07-21T22:21:06.245652+0000","last_fullsized":"2023-07-21T22:21:06.245652+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:04.926506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418255+0000","last_change":"2023-07-21T22:19:11.672524+0000","last_active":"2023-07-21T22:20:38.418255+0000","last_peered":"2023-07-21T22:20:38.418255+0000","last_clean":"2023-07-21T22:20:38.418255+0000","last_became_active":"2023-07-21T22:19:11.672077+0000","last_became_peered":"2023-07-21T22:19:11.672077+0000","last_unstale":"2023-07-21T22:20:38.418255+0000","last_undegraded":"2023-07-21T22:20:38.418255+0000","last_fullsized":"2023-07-21T22:20:38.418255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:26:43.576746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.438152+0000","last_change":"2023-07-21T22:19:06.374199+0000","last_active":"2023-07-21T22:20:38.438152+0000","last_peered":"2023-07-21T22:20:38.438152+0000","last_clean":"2023-07-21T22:20:38.438152+0000","last_became_active":"2023-07-21T22:19:06.373884+0000","last_became_peered":"2023-07-21T22:19:06.373884+0000","last_unstale":"2023-07-21T22:20:38.438152+0000","last_undegraded":"2023-07-21T22:20:38.438152+0000","last_fullsized":"2023-07-21T22:20:38.438152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:51:01.655488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.204620+0000","last_change":"2023-07-21T22:19:07.370907+0000","last_active":"2023-07-21T22:21:06.204620+0000","last_peered":"2023-07-21T22:21:06.204620+0000","last_clean":"2023-07-21T22:21:06.204620+0000","last_became_active":"2023-07-21T22:19:07.370604+0000","last_became_peered":"2023-07-21T22:19:07.370604+0000","last_unstale":"2023-07-21T22:21:06.204620+0000","last_undegraded":"2023-07-21T22:21:06.204620+0000","last_fullsized":"2023-07-21T22:21:06.204620+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:36:08.420123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298007+0000","last_change":"2023-07-21T22:19:11.393180+0000","last_active":"2023-07-21T22:20:38.298007+0000","last_peered":"2023-07-21T22:20:38.298007+0000","last_clean":"2023-07-21T22:20:38.298007+0000","last_became_active":"2023-07-21T22:19:11.392777+0000","last_became_peered":"2023-07-21T22:19:11.392777+0000","last_unstale":"2023-07-21T22:20:38.298007+0000","last_undegraded":"2023-07-21T22:20:38.298007+0000","last_fullsized":"2023-07-21T22:20:38.298007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:47:54.100394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659458+0000","last_change":"2023-07-21T22:19:06.370881+0000","last_active":"2023-07-21T22:20:38.659458+0000","last_peered":"2023-07-21T22:20:38.659458+0000","last_clean":"2023-07-21T22:20:38.659458+0000","last_became_active":"2023-07-21T22:19:06.370558+0000","last_became_peered":"2023-07-21T22:19:06.370558+0000","last_unstale":"2023-07-21T22:20:38.659458+0000","last_undegraded":"2023-07-21T22:20:38.659458+0000","last_fullsized":"2023-07-21T22:20:38.659458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T08:11:22.743920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.244209+0000","last_change":"2023-07-21T22:19:07.375642+0000","last_active":"2023-07-21T22:21:06.244209+0000","last_peered":"2023-07-21T22:21:06.244209+0000","last_clean":"2023-07-21T22:21:06.244209+0000","last_became_active":"2023-07-21T22:19:07.375299+0000","last_became_peered":"2023-07-21T22:19:07.375299+0000","last_unstale":"2023-07-21T22:21:06.244209+0000","last_undegraded":"2023-07-21T22:21:06.244209+0000","last_fullsized":"2023-07-21T22:21:06.244209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:10:19.440093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418254+0000","last_change":"2023-07-21T22:19:11.387119+0000","last_active":"2023-07-21T22:20:38.418254+0000","last_peered":"2023-07-21T22:20:38.418254+0000","last_clean":"2023-07-21T22:20:38.418254+0000","last_became_active":"2023-07-21T22:19:11.386648+0000","last_became_peered":"2023-07-21T22:19:11.386648+0000","last_unstale":"2023-07-21T22:20:38.418254+0000","last_undegraded":"2023-07-21T22:20:38.418254+0000","last_fullsized":"2023-07-21T22:20:38.418254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:57:52.621668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.298467+0000","last_change":"2023-07-21T22:19:37.315809+0000","last_active":"2023-07-21T22:20:38.298467+0000","last_peered":"2023-07-21T22:20:38.298467+0000","last_clean":"2023-07-21T22:20:38.298467+0000","last_became_active":"2023-07-21T22:19:37.312331+0000","last_became_peered":"2023-07-21T22:19:37.312331+0000","last_unstale":"2023-07-21T22:20:38.298467+0000","last_undegraded":"2023-07-21T22:20:38.298467+0000","last_fullsized":"2023-07-21T22:20:38.298467+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T10:01:40.724543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243178+0000","last_change":"2023-07-21T22:19:07.384332+0000","last_active":"2023-07-21T22:21:06.243178+0000","last_peered":"2023-07-21T22:21:06.243178+0000","last_clean":"2023-07-21T22:21:06.243178+0000","last_became_active":"2023-07-21T22:19:07.383866+0000","last_became_peered":"2023-07-21T22:19:07.383866+0000","last_unstale":"2023-07-21T22:21:06.243178+0000","last_undegraded":"2023-07-21T22:21:06.243178+0000","last_fullsized":"2023-07-21T22:21:06.243178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:55.735567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.418468+0000","last_change":"2023-07-21T22:19:11.671175+0000","last_active":"2023-07-21T22:20:38.418468+0000","last_peered":"2023-07-21T22:20:38.418468+0000","last_clean":"2023-07-21T22:20:38.418468+0000","last_became_active":"2023-07-21T22:19:11.670853+0000","last_became_peered":"2023-07-21T22:19:11.670853+0000","last_unstale":"2023-07-21T22:20:38.418468+0000","last_undegraded":"2023-07-21T22:20:38.418468+0000","last_fullsized":"2023-07-21T22:20:38.418468+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:18:59.817080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.658662+0000","last_change":"2023-07-21T22:19:06.368579+0000","last_active":"2023-07-21T22:20:38.658662+0000","last_peered":"2023-07-21T22:20:38.658662+0000","last_clean":"2023-07-21T22:20:38.658662+0000","last_became_active":"2023-07-21T22:19:06.368249+0000","last_became_peered":"2023-07-21T22:19:06.368249+0000","last_unstale":"2023-07-21T22:20:38.658662+0000","last_undegraded":"2023-07-21T22:20:38.658662+0000","last_fullsized":"2023-07-21T22:20:38.658662+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:59:29.846402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.229474+0000","last_change":"2023-07-21T22:19:07.377062+0000","last_active":"2023-07-21T22:21:06.229474+0000","last_peered":"2023-07-21T22:21:06.229474+0000","last_clean":"2023-07-21T22:21:06.229474+0000","last_became_active":"2023-07-21T22:19:07.376616+0000","last_became_peered":"2023-07-21T22:19:07.376616+0000","last_unstale":"2023-07-21T22:21:06.229474+0000","last_undegraded":"2023-07-21T22:21:06.229474+0000","last_fullsized":"2023-07-21T22:21:06.229474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:37:37.964833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.438255+0000","last_change":"2023-07-21T22:19:11.392777+0000","last_active":"2023-07-21T22:20:38.438255+0000","last_peered":"2023-07-21T22:20:38.438255+0000","last_clean":"2023-07-21T22:20:38.438255+0000","last_became_active":"2023-07-21T22:19:11.391866+0000","last_became_peered":"2023-07-21T22:19:11.391866+0000","last_unstale":"2023-07-21T22:20:38.438255+0000","last_undegraded":"2023-07-21T22:20:38.438255+0000","last_fullsized":"2023-07-21T22:20:38.438255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:30.756639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.305874+0000","last_change":"2023-07-21T22:19:06.370977+0000","last_active":"2023-07-21T22:20:38.305874+0000","last_peered":"2023-07-21T22:20:38.305874+0000","last_clean":"2023-07-21T22:20:38.305874+0000","last_became_active":"2023-07-21T22:19:06.370595+0000","last_became_peered":"2023-07-21T22:19:06.370595+0000","last_unstale":"2023-07-21T22:20:38.305874+0000","last_undegraded":"2023-07-21T22:20:38.305874+0000","last_fullsized":"2023-07-21T22:20:38.305874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:39:12.174231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.230951+0000","last_change":"2023-07-21T22:19:07.375355+0000","last_active":"2023-07-21T22:21:06.230951+0000","last_peered":"2023-07-21T22:21:06.230951+0000","last_clean":"2023-07-21T22:21:06.230951+0000","last_became_active":"2023-07-21T22:19:07.375054+0000","last_became_peered":"2023-07-21T22:19:07.375054+0000","last_unstale":"2023-07-21T22:21:06.230951+0000","last_undegraded":"2023-07-21T22:21:06.230951+0000","last_fullsized":"2023-07-21T22:21:06.230951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:11:25.905492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.659463+0000","last_change":"2023-07-21T22:19:11.383198+0000","last_active":"2023-07-21T22:20:38.659463+0000","last_peered":"2023-07-21T22:20:38.659463+0000","last_clean":"2023-07-21T22:20:38.659463+0000","last_became_active":"2023-07-21T22:19:11.382767+0000","last_became_peered":"2023-07-21T22:19:11.382767+0000","last_unstale":"2023-07-21T22:20:38.659463+0000","last_undegraded":"2023-07-21T22:20:38.659463+0000","last_fullsized":"2023-07-21T22:20:38.659463+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T09:09:28.212992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.297292+0000","last_change":"2023-07-21T22:19:06.377345+0000","last_active":"2023-07-21T22:20:38.297292+0000","last_peered":"2023-07-21T22:20:38.297292+0000","last_clean":"2023-07-21T22:20:38.297292+0000","last_became_active":"2023-07-21T22:19:06.377090+0000","last_became_peered":"2023-07-21T22:19:06.377090+0000","last_unstale":"2023-07-21T22:20:38.297292+0000","last_undegraded":"2023-07-21T22:20:38.297292+0000","last_fullsized":"2023-07-21T22:20:38.297292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:43:21.204017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.226015+0000","last_change":"2023-07-21T22:19:07.382180+0000","last_active":"2023-07-21T22:21:06.226015+0000","last_peered":"2023-07-21T22:21:06.226015+0000","last_clean":"2023-07-21T22:21:06.226015+0000","last_became_active":"2023-07-21T22:19:07.381833+0000","last_became_peered":"2023-07-21T22:19:07.381833+0000","last_unstale":"2023-07-21T22:21:06.226015+0000","last_undegraded":"2023-07-21T22:21:06.226015+0000","last_fullsized":"2023-07-21T22:21:06.226015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:21:16.718601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.296263+0000","last_change":"2023-07-21T22:19:11.386363+0000","last_active":"2023-07-21T22:20:38.296263+0000","last_peered":"2023-07-21T22:20:38.296263+0000","last_clean":"2023-07-21T22:20:38.296263+0000","last_became_active":"2023-07-21T22:19:11.385974+0000","last_became_peered":"2023-07-21T22:19:11.385974+0000","last_unstale":"2023-07-21T22:20:38.296263+0000","last_undegraded":"2023-07-21T22:20:38.296263+0000","last_fullsized":"2023-07-21T22:20:38.296263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:10.358827+0000","last_clean_scrub_stamp":"2023-07-21T22:19:10.358827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:01:36.550687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:20:38.295758+0000","last_change":"2023-07-21T22:19:06.380029+0000","last_active":"2023-07-21T22:20:38.295758+0000","last_peered":"2023-07-21T22:20:38.295758+0000","last_clean":"2023-07-21T22:20:38.295758+0000","last_became_active":"2023-07-21T22:19:06.379624+0000","last_became_peered":"2023-07-21T22:19:06.379624+0000","last_unstale":"2023-07-21T22:20:38.295758+0000","last_undegraded":"2023-07-21T22:20:38.295758+0000","last_fullsized":"2023-07-21T22:20:38.295758+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:05.147343+0000","last_clean_scrub_stamp":"2023-07-21T22:19:05.147343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:55:13.984980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.227990+0000","last_change":"2023-07-21T22:19:07.374999+0000","last_active":"2023-07-21T22:21:06.227990+0000","last_peered":"2023-07-21T22:21:06.227990+0000","last_clean":"2023-07-21T22:21:06.227990+0000","last_became_active":"2023-07-21T22:19:07.374671+0000","last_became_peered":"2023-07-21T22:19:07.374671+0000","last_unstale":"2023-07-21T22:21:06.227990+0000","last_undegraded":"2023-07-21T22:21:06.227990+0000","last_fullsized":"2023-07-21T22:21:06.227990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:11.723201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.237254+0000","last_change":"2023-07-21T22:20:38.306667+0000","last_active":"2023-07-21T22:21:06.237254+0000","last_peered":"2023-07-21T22:21:06.237254+0000","last_clean":"2023-07-21T22:21:06.237254+0000","last_became_active":"2023-07-21T22:20:38.306285+0000","last_became_peered":"2023-07-21T22:20:38.306285+0000","last_unstale":"2023-07-21T22:21:06.237254+0000","last_undegraded":"2023-07-21T22:21:06.237254+0000","last_fullsized":"2023-07-21T22:21:06.237254+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:32:37.690707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":47,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236563+0000","last_change":"2023-07-21T22:19:37.326004+0000","last_active":"2023-07-21T22:21:06.236563+0000","last_peered":"2023-07-21T22:21:06.236563+0000","last_clean":"2023-07-21T22:21:06.236563+0000","last_became_active":"2023-07-21T22:19:37.325611+0000","last_became_peered":"2023-07-21T22:19:37.325611+0000","last_unstale":"2023-07-21T22:21:06.236563+0000","last_undegraded":"2023-07-21T22:21:06.236563+0000","last_fullsized":"2023-07-21T22:21:06.236563+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:02.980321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.212111+0000","last_change":"2023-07-21T22:19:07.380307+0000","last_active":"2023-07-21T22:21:06.212111+0000","last_peered":"2023-07-21T22:21:06.212111+0000","last_clean":"2023-07-21T22:21:06.212111+0000","last_became_active":"2023-07-21T22:19:07.379977+0000","last_became_peered":"2023-07-21T22:19:07.379977+0000","last_unstale":"2023-07-21T22:21:06.212111+0000","last_undegraded":"2023-07-21T22:21:06.212111+0000","last_fullsized":"2023-07-21T22:21:06.212111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:22:52.562457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.243872+0000","last_change":"2023-07-21T22:19:07.375900+0000","last_active":"2023-07-21T22:21:06.243872+0000","last_peered":"2023-07-21T22:21:06.243872+0000","last_clean":"2023-07-21T22:21:06.243872+0000","last_became_active":"2023-07-21T22:19:07.375539+0000","last_became_peered":"2023-07-21T22:19:07.375539+0000","last_unstale":"2023-07-21T22:21:06.243872+0000","last_undegraded":"2023-07-21T22:21:06.243872+0000","last_fullsized":"2023-07-21T22:21:06.243872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:06:39.140948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.240603+0000","last_change":"2023-07-21T22:19:07.372543+0000","last_active":"2023-07-21T22:21:06.240603+0000","last_peered":"2023-07-21T22:21:06.240603+0000","last_clean":"2023-07-21T22:21:06.240603+0000","last_became_active":"2023-07-21T22:19:07.372168+0000","last_became_peered":"2023-07-21T22:19:07.372168+0000","last_unstale":"2023-07-21T22:21:06.240603+0000","last_undegraded":"2023-07-21T22:21:06.240603+0000","last_fullsized":"2023-07-21T22:21:06.240603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:02:52.883722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.226450+0000","last_change":"2023-07-21T22:19:07.371739+0000","last_active":"2023-07-21T22:21:06.226450+0000","last_peered":"2023-07-21T22:21:06.226450+0000","last_clean":"2023-07-21T22:21:06.226450+0000","last_became_active":"2023-07-21T22:19:07.371368+0000","last_became_peered":"2023-07-21T22:19:07.371368+0000","last_unstale":"2023-07-21T22:21:06.226450+0000","last_undegraded":"2023-07-21T22:21:06.226450+0000","last_fullsized":"2023-07-21T22:21:06.226450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:44:28.022785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":43,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.236553+0000","last_change":"2023-07-21T22:19:07.369502+0000","last_active":"2023-07-21T22:21:06.236553+0000","last_peered":"2023-07-21T22:21:06.236553+0000","last_clean":"2023-07-21T22:21:06.236553+0000","last_became_active":"2023-07-21T22:19:07.369222+0000","last_became_peered":"2023-07-21T22:19:07.369222+0000","last_unstale":"2023-07-21T22:21:06.236553+0000","last_undegraded":"2023-07-21T22:21:06.236553+0000","last_fullsized":"2023-07-21T22:21:06.236553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:50:57.476981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.228554+0000","last_change":"2023-07-21T22:19:37.314152+0000","last_active":"2023-07-21T22:21:06.228554+0000","last_peered":"2023-07-21T22:21:06.228554+0000","last_clean":"2023-07-21T22:21:06.228554+0000","last_became_active":"2023-07-21T22:19:37.313795+0000","last_became_peered":"2023-07-21T22:19:37.313795+0000","last_unstale":"2023-07-21T22:21:06.228554+0000","last_undegraded":"2023-07-21T22:21:06.228554+0000","last_fullsized":"2023-07-21T22:21:06.228554+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:52:37.184312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233179+0000","last_change":"2023-07-21T22:19:07.374678+0000","last_active":"2023-07-21T22:21:06.233179+0000","last_peered":"2023-07-21T22:21:06.233179+0000","last_clean":"2023-07-21T22:21:06.233179+0000","last_became_active":"2023-07-21T22:19:07.374357+0000","last_became_peered":"2023-07-21T22:19:07.374357+0000","last_unstale":"2023-07-21T22:21:06.233179+0000","last_undegraded":"2023-07-21T22:21:06.233179+0000","last_fullsized":"2023-07-21T22:21:06.233179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:26:23.465390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.232972+0000","last_change":"2023-07-21T22:20:38.307235+0000","last_active":"2023-07-21T22:21:06.232972+0000","last_peered":"2023-07-21T22:21:06.232972+0000","last_clean":"2023-07-21T22:21:06.232972+0000","last_became_active":"2023-07-21T22:20:38.306829+0000","last_became_peered":"2023-07-21T22:20:38.306829+0000","last_unstale":"2023-07-21T22:21:06.232972+0000","last_undegraded":"2023-07-21T22:21:06.232972+0000","last_fullsized":"2023-07-21T22:21:06.232972+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:18:35.611527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.245317+0000","last_change":"2023-07-21T22:19:07.370439+0000","last_active":"2023-07-21T22:21:06.245317+0000","last_peered":"2023-07-21T22:21:06.245317+0000","last_clean":"2023-07-21T22:21:06.245317+0000","last_became_active":"2023-07-21T22:19:07.369721+0000","last_became_peered":"2023-07-21T22:19:07.369721+0000","last_unstale":"2023-07-21T22:21:06.245317+0000","last_undegraded":"2023-07-21T22:21:06.245317+0000","last_fullsized":"2023-07-21T22:21:06.245317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:17.933758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.242741+0000","last_change":"2023-07-21T22:19:07.370522+0000","last_active":"2023-07-21T22:21:06.242741+0000","last_peered":"2023-07-21T22:21:06.242741+0000","last_clean":"2023-07-21T22:21:06.242741+0000","last_became_active":"2023-07-21T22:19:07.369933+0000","last_became_peered":"2023-07-21T22:19:07.369933+0000","last_unstale":"2023-07-21T22:21:06.242741+0000","last_undegraded":"2023-07-21T22:21:06.242741+0000","last_fullsized":"2023-07-21T22:21:06.242741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:42:34.683114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-07-21T22:21:06.233743+0000","last_change":"2023-07-21T22:19:07.370752+0000","last_active":"2023-07-21T22:21:06.233743+0000","last_peered":"2023-07-21T22:21:06.233743+0000","last_clean":"2023-07-21T22:21:06.233743+0000","last_became_active":"2023-07-21T22:19:07.370464+0000","last_became_peered":"2023-07-21T22:19:07.370464+0000","last_unstale":"2023-07-21T22:21:06.233743+0000","last_undegraded":"2023-07-21T22:21:06.233743+0000","last_fullsized":"2023-07-21T22:21:06.233743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T22:19:06.345403+0000","last_clean_scrub_stamp":"2023-07-21T22:19:06.345403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:02:26.104006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":213049,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68479,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":624,"num_read_kb":387,"num_write":165,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":205437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383718,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":515866,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65800000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55600000000000005,"15min":0.55000000000000004},"min":{"1min":0.47699999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.40300000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61299999999999999,"15min":0.60999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65100000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.496,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.89000000000000001,"5min":1.21,"15min":1.21},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66700000000000004,"15min":0.67200000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90200000000000002,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.62},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.81599999999999995,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.48199999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.879,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69399999999999995,"15min":0.68999999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.99299999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73999999999999999,"15min":0.73699999999999999},"min":{"1min":0.52700000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.93500000000000005,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.754}]},{"osd":7,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75600000000000001,"15min":0.753},"min":{"1min":0.504,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.95799999999999996,"5min":1.294,"15min":1.294},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.98799999999999999,"5min":1.103,"15min":1.103},"last":0.85399999999999998}]},{"osd":8,"last update":"Fri Jul 21 22:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.76700000000000002,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78400000000000003,"15min":0.78900000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.01,"5min":1.024,"15min":1.024},"last":0.80200000000000005}]},{"osd":9,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79500000000000004},"min":{"1min":0.64000000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97699999999999998,"5min":1.431,"15min":1.431},"last":0.874},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.64400000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.462,"15min":1.462},"last":0.83999999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.80700000000000005,"15min":0.79900000000000004},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.622,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.97599999999999998,"5min":1.5,"15min":1.5},"last":0.82499999999999996}]}]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1136,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1163264,"data_stored":1167204,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":1.143,"15min":1.2370000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.754,"5min":17.132000000000001,"15min":17.132000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.1439999999999999,"15min":1.234},"min":{"1min":0.47899999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":17.084,"15min":17.084},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.44700000000000001,"15min":0.46300000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56000000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.48399999999999999,"15min":0.498},"min":{"1min":0.308,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61699999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.40500000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77100000000000002,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.40799999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75600000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78100000000000003,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.747,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.82599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.36299999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.54700000000000004,"15min":0.56699999999999995},"min":{"1min":0.314,"5min":0.31,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66700000000000004,"15min":0.67300000000000004},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80100000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70499999999999996,"15min":0.71799999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.747}]},{"osd":6,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.44400000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.363,"15min":1.363},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.77400000000000002,"15min":0.80200000000000005},"min":{"1min":0.45400000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.84599999999999997,"5min":1.161,"15min":1.161},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.439}]},{"osd":8,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.74199999999999999,"15min":0.76100000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.87,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.80300000000000005},"min":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.86399999999999999,"5min":1.024,"15min":1.024},"last":0.88300000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91700000000000004,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.95899999999999996,"15min":0.89700000000000002},"min":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.88400000000000001,"5min":17.163,"15min":17.163},"last":0.91000000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81200000000000006,"15min":0.82299999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.99399999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.93899999999999995,"5min":1.113,"15min":1.113},"last":0.95399999999999996}]}]},{"osd":9,"up_from":68,"seq":292057776173,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":520198,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60999999999999999,"15min":0.622},"min":{"1min":0.33600000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":2.0979999999999999,"5min":2.0979999999999999,"15min":2.0979999999999999},"last":0.437},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.0760000000000001},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79800000000000004,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85899999999999999,"15min":0.91700000000000004},"min":{"1min":0.55500000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77800000000000002,"15min":0.80200000000000005},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.952,"5min":1.952,"15min":1.952},"last":0.54300000000000004}]},{"osd":3,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.81799999999999995},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.84499999999999997,"15min":0.89100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.1230000000000002},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76100000000000001,"15min":0.77000000000000002},"min":{"1min":0.48799999999999999,"5min":0.433,"15min":0.433},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.626},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":0.92900000000000005},"min":{"1min":0.57899999999999996,"5min":0.497,"15min":0.497},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.877,"15min":0.92500000000000004},"min":{"1min":0.47099999999999997,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.851,"5min":1.851,"15min":1.851},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.90300000000000002,"15min":0.94599999999999995},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:18:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6100000000000001,"5min":1.6100000000000001,"15min":1.6100000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6910000000000001,"5min":1.6910000000000001,"15min":1.6910000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.97499999999999998,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":1.006,"15min":1.077},"min":{"1min":0.58299999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.585,"5min":1.8859999999999999,"15min":1.8859999999999999},"last":0.754}]},{"osd":8,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.052,"15min":1.133},"min":{"1min":0.61599999999999999,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.611,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.99099999999999999,"15min":1.069},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.9019999999999999,"5min":1.99,"15min":1.99},"last":0.79700000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.90500000000000003,"15min":0.94199999999999995},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0720000000000001,"5min":2.0720000000000001,"15min":2.0720000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.94199999999999995,"15min":0.96299999999999997},"min":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.29,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.91800000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9970000000000001,"5min":1.9970000000000001,"15min":1.9970000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.0389999999999999}]}]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":1032192,"data_stored":516022,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.72499999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.63,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.59299999999999997,"15min":0.57799999999999996},"min":{"1min":0.45200000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.83999999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63900000000000001,"15min":0.61699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34599999999999997,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.94399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67800000000000005,"15min":0.67600000000000005},"min":{"1min":0.44700000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.94599999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.623,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.629},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.46200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.753,"15min":0.745},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.747},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.59999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.083,"15min":1.083},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:20:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.54600000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.627,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0229999999999999,"5min":1.111,"15min":1.111},"last":0.78600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.80200000000000005,"15min":0.77000000000000002},"min":{"1min":0.73299999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.82799999999999996,"15min":0.79600000000000004},"min":{"1min":0.78500000000000003,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.97099999999999997}]},{"osd":10,"last update":"Fri Jul 21 22:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.71999999999999997,"15min":0.68200000000000005},"min":{"1min":0.58399999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.93400000000000005,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.79700000000000004,"15min":0.75700000000000001},"min":{"1min":0.66500000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.82499999999999996,"15min":0.83099999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.90400000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.879},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89600000000000002,"15min":0.89500000000000002},"min":{"1min":0.72199999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.94799999999999995}]}]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":1112822,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56799999999999995,"15min":0.56200000000000006},"min":{"1min":0.46000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77300000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42999999999999999,"15min":0.434},"min":{"1min":0.33900000000000002,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.49099999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.36799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.57799999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79300000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61699999999999999,"15min":0.60999999999999999},"min":{"1min":0.503,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.75900000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55700000000000005}]},{"osd":3,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.74199999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63200000000000001,"15min":0.626},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88700000000000001,"5min":1.399,"15min":1.399},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.501,"15min":0.499},"min":{"1min":0.35299999999999998,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.65900000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.498,"15min":0.51200000000000001},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.441}]},{"osd":5,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.5,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77400000000000002,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68500000000000005,"15min":0.68899999999999995},"min":{"1min":0.57499999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.874,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Jul 21 22:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.52900000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91700000000000004,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.51200000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.57799999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.66100000000000003,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.83899999999999997,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72599999999999998}]},{"osd":9,"last update":"Fri Jul 21 22:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.442},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.62,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.79300000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.55000000000000004,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96599999999999997,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.60199999999999998,"5min":0.499,"15min":0.499},"max":{"1min":0.95599999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.74299999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82499999999999996,"15min":0.82099999999999995},"min":{"1min":0.69899999999999995,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78900000000000003,"15min":0.78100000000000003},"min":{"1min":0.69099999999999995,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.77000000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":511278,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54500000000000004,"15min":0.52500000000000002},"min":{"1min":0.35399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.365,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.498},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55500000000000005,"15min":0.54300000000000004},"min":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.76400000000000001,"15min":0.72699999999999998},"min":{"1min":0.63200000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.435,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.139,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.82199999999999995}]},{"osd":2,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79700000000000004,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.95599999999999996}]},{"osd":3,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73599999999999999,"15min":0.70499999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.258,"5min":1.284,"15min":1.284},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67000000000000004,"15min":0.66000000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.46300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.65300000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.98299999999999998}]},{"osd":5,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.78900000000000003,"15min":0.77100000000000002},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.228,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.746},"min":{"1min":0.55500000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.80000000000000004},"min":{"1min":0.52700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2170000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.871},{"interface":"front","average":{"1min":0.996,"5min":0.91900000000000004,"15min":0.88300000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":3.202,"5min":3.202,"15min":3.202},"last":1.0069999999999999}]},{"osd":8,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.93200000000000005,"15min":0.93500000000000005},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.1279999999999999,"5min":1.599,"15min":1.599},"last":1.026},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.88900000000000001,"15min":0.90400000000000003},"min":{"1min":0.66400000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1799999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.90600000000000003}]},{"osd":9,"last update":"Fri Jul 21 22:20:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.006,"15min":1.0109999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.091},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.93999999999999995,"15min":0.95199999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.94099999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.84299999999999997,"15min":0.85599999999999998},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.2629999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.018,"15min":1.036},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1120000000000001,"5min":1.4139999999999999,"15min":1.4139999999999999},"last":1.048}]},{"osd":11,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.92400000000000004,"15min":0.94199999999999995},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.0369999999999999,"5min":1.585,"15min":1.585},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":1.0660000000000001,"15min":1.083},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.163,"5min":1.373,"15min":1.373},"last":1.069}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":1056,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1081344,"data_stored":574775,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53500000000000003,"15min":0.50900000000000001},"min":{"1min":0.317,"5min":0.31,"15min":0.31},"max":{"1min":0.69199999999999995,"5min":0.95099999999999996,"15min":0.98799999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.51500000000000001},"min":{"1min":0.36599999999999999,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.69199999999999995,"5min":0.88300000000000001,"15min":1.1100000000000001},"last":0.62}]},{"osd":2,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.60599999999999998,"15min":0.57399999999999995},"min":{"1min":0.51000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84199999999999997,"5min":0.871,"15min":1.0289999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.58799999999999997,"15min":0.56699999999999995},"min":{"1min":0.47399999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.93000000000000005,"15min":0.93400000000000005},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jul 21 22:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.66400000000000003,"15min":0.59799999999999998},"min":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":6.0119999999999996,"5min":6.0119999999999996,"15min":6.0119999999999996},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.59999999999999998,"15min":0.57799999999999996},"min":{"1min":0.45300000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.372,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64800000000000002,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.46200000000000002},"min":{"1min":0.38600000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.64700000000000002},"min":{"1min":0.56499999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92900000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.69299999999999995,"15min":0.66400000000000003},"min":{"1min":0.64100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91500000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.66900000000000004},"min":{"1min":0.51100000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0169999999999999,"5min":1.45,"15min":1.45},"last":0.747},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70699999999999996,"15min":0.69099999999999995},"min":{"1min":0.55700000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.58999999999999997,"15min":0.55200000000000005},"min":{"1min":0.44,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85699999999999998,"5min":1.077,"15min":1.077},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.64300000000000002,"15min":0.59399999999999997},"min":{"1min":0.56999999999999995,"5min":0.313,"15min":0.313},"max":{"1min":0.93600000000000005,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.70399999999999996,"15min":0.67200000000000004},"min":{"1min":0.55300000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.99199999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.71099999999999997,"15min":0.69299999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.94499999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.94299999999999995}]},{"osd":9,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.75,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.82099999999999995,"15min":0.80800000000000005},"min":{"1min":0.69799999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.1080000000000001,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.95699999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.5,"15min":0.5},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.755},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.80300000000000005,"15min":0.78200000000000003},"min":{"1min":0.753,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.98799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":0.93600000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.877,"15min":0.879},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":0.995,"5min":0.997,"15min":0.997},"last":0.93000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":1036288,"data_stored":512666,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 22:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.67900000000000005,"15min":0.55100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35999999999999999,"15min":0.35899999999999999},"max":{"1min":3.048,"5min":3.048,"15min":3.048},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73499999999999999,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.0780000000000001,"5min":5.4909999999999997,"15min":5.4909999999999997},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.66500000000000004},"min":{"1min":0.52500000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.76300000000000001,"15min":0.65900000000000003},"min":{"1min":0.56200000000000006,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.90000000000000002}]},{"osd":3,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66400000000000003,"15min":0.66200000000000003},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.2250000000000001,"5min":1.23,"15min":1.23},"last":0.37}]},{"osd":4,"last update":"Fri Jul 21 22:20:53 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.74399999999999999,"15min":0.71299999999999997},"min":{"1min":0.55400000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.1579999999999999,"5min":3.1579999999999999,"15min":3.1579999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.72999999999999998,"15min":0.68000000000000005},"min":{"1min":0.60299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.149,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.76900000000000002,"15min":0.73599999999999999},"min":{"1min":0.55400000000000005,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77600000000000002,"15min":0.71699999999999997},"min":{"1min":0.53500000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.246,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Fri Jul 21 22:20:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.78200000000000003,"15min":0.72499999999999998},"min":{"1min":0.51600000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.68700000000000006,"15min":0.63700000000000001},"min":{"1min":0.58299999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.496}]},{"osd":7,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":3.1219999999999999,"5min":3.1219999999999999,"15min":3.1219999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.0880000000000001,"5min":3.0880000000000001,"15min":3.0880000000000001},"last":0.82899999999999996}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.625,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.86899999999999999}]},{"osd":9,"last update":"Fri Jul 21 22:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.97099999999999997,"15min":0.98899999999999999},"min":{"1min":0.65700000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.226,"5min":0.88800000000000001,"15min":0.83199999999999996},"min":{"1min":0.57599999999999996,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.0640000000000001,"5min":6.0640000000000001,"15min":6.0640000000000001},"last":0.56499999999999995}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98399999999999999,"15min":0.99399999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4199999999999999,"5min":1.526,"15min":1.526},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.871,"15min":0.84899999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Jul 21 22:20:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.86899999999999999,"15min":0.84599999999999997},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":1.9710000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.91200000000000003,"15min":0.89600000000000002},"min":{"1min":0.71399999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.88}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1105920,"data_stored":576924,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.63400000000000001},"min":{"1min":0.376,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.79300000000000004,"5min":1.9530000000000001,"15min":3.988},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.30299999999999999},"max":{"1min":0.754,"5min":1.421,"15min":1.421},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.626,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.006,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.54500000000000004},"min":{"1min":0.51600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.96599999999999997,"5min":2.044,"15min":2.044},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.41799999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.88600000000000001,"5min":1.984,"15min":1.984},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.51600000000000001}]},{"osd":4,"last update":"Fri Jul 21 22:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.70199999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50600000000000001,"15min":0.45800000000000002},"min":{"1min":0.36899999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.82799999999999996,"5min":1.542,"15min":1.542},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.52500000000000002,"15min":0.47299999999999998},"min":{"1min":0.39200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.84499999999999997,"5min":1.76,"15min":1.76},"last":0.755}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71399999999999997,"15min":0.68500000000000005},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.079,"5min":1.907,"15min":1.907},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.72399999999999998,"15min":0.69599999999999995},"min":{"1min":0.57299999999999995,"5min":0.442,"15min":0.442},"max":{"1min":1.0489999999999999,"5min":1.484,"15min":1.484},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jul 21 22:21:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83699999999999997,"15min":0.83499999999999996},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.77000000000000002}]},{"osd":8,"last update":"Fri Jul 21 22:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63800000000000001,"15min":0.61799999999999999},"min":{"1min":0.505,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.98199999999999998,"5min":1.609,"15min":1.609},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.0620000000000001,"5min":1.859,"15min":1.859},"last":0.496}]},{"osd":9,"last update":"Fri Jul 21 22:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80500000000000005,"15min":0.79800000000000004},"min":{"1min":0.58499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.117,"5min":2.016,"15min":2.016},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.69099999999999995,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.022,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.81699999999999995,"15min":0.79700000000000004},"min":{"1min":0.622,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.149,"5min":1.887,"15min":1.887},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.82099999999999995},"min":{"1min":0.71099999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.101,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.85299999999999998}]},{"osd":11,"last update":"Fri Jul 21 22:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82399999999999995,"15min":0.82299999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.88500000000000001,"15min":0.88200000000000001},"min":{"1min":0.71899999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66600000000000004}]}]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":511472,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.501,"15min":0.41299999999999998},"min":{"1min":0.39100000000000001,"5min":0.247,"15min":0.247},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.47799999999999998,"15min":0.42299999999999999},"min":{"1min":0.35999999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.63800000000000001,"15min":0.56899999999999995},"min":{"1min":0.44,"5min":0.378,"15min":0.378},"max":{"1min":3.0739999999999998,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.67200000000000004,"15min":0.57599999999999996},"min":{"1min":0.50700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":2.734,"5min":2.734,"15min":2.734},"last":0.85299999999999998}]},{"osd":2,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.67000000000000004,"15min":0.60999999999999999},"min":{"1min":0.55200000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.86499999999999999,"5min":1.139,"15min":1.139},"last":0.56299999999999994},{"interface":"front","average":{"1min":1.177,"5min":0.76900000000000002,"15min":0.65900000000000003},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":6.0199999999999996,"5min":6.0199999999999996,"15min":6.0199999999999996},"last":0.54200000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.68500000000000005,"15min":0.61199999999999999},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.746,"15min":0.64600000000000002},"min":{"1min":0.57699999999999996,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6850000000000001,"5min":2.6850000000000001,"15min":2.6850000000000001},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69999999999999996,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":1.002,"15min":1.002},"last":0.623},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77300000000000002,"15min":0.71899999999999997},"min":{"1min":0.56100000000000005,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.073,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.80900000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:20:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":0.80500000000000005,"15min":0.71899999999999997},"min":{"1min":0.61399999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.923,"5min":2.923,"15min":2.923},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.76400000000000001,"15min":0.71599999999999997},"min":{"1min":0.501,"5min":0.375,"15min":0.375},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Jul 21 22:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.78200000000000003,"15min":0.71099999999999997},"min":{"1min":0.48899999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.1739999999999999,"5min":3.1739999999999999,"15min":3.1739999999999999},"last":0.755},{"interface":"front","average":{"1min":1.1080000000000001,"5min":0.84399999999999997,"15min":0.76900000000000002},"min":{"1min":0.72299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.77,"5min":2.77,"15min":2.77},"last":0.90600000000000003}]},{"osd":8,"last update":"Fri Jul 21 22:20:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":3.2549999999999999,"5min":3.2549999999999999,"15min":3.2549999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.88100000000000001}]},{"osd":9,"last update":"Fri Jul 21 22:20:39 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":0.98799999999999999,"15min":0.97299999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.90100000000000002,"15min":0.91000000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.72899999999999998}]},{"osd":10,"last update":"Fri Jul 21 22:21:02 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":0.98299999999999998,"15min":0.93799999999999994},"min":{"1min":0.80300000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":2.8399999999999999,"5min":2.8399999999999999,"15min":2.8399999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.89100000000000001,"15min":0.88400000000000001},"min":{"1min":0.71899999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.78200000000000003}]},{"osd":11,"last update":"Fri Jul 21 22:21:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.1439999999999999,"15min":1.1639999999999999},"min":{"1min":0.85599999999999998,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.3700000000000001,"5min":2.883,"15min":2.883},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":0.93300000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5109999999999999,"5min":1.716,"15min":1.716},"last":0.82599999999999996}]}]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":516411,"data_compressed":23475,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:18:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.55500000000000005,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83999999999999997,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.83699999999999997,"5min":1.484,"15min":1.484},"last":0.69499999999999995}]},{"osd":1,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42599999999999999,"15min":0.38600000000000001},"min":{"1min":0.35799999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59699999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.44800000000000001,"15min":0.42999999999999999},"min":{"1min":0.33000000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63100000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.377}]},{"osd":2,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.83099999999999996,"5min":1.3,"15min":1.3},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63400000000000001,"15min":0.57399999999999995},"min":{"1min":0.50900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.68400000000000005,"15min":0.625},"min":{"1min":0.56599999999999995,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":5.2770000000000001,"5min":5.2770000000000001,"15min":5.2770000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.65500000000000003,"15min":0.59199999999999997},"min":{"1min":0.48599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":3.657,"5min":3.657,"15min":3.657},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jul 21 22:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65700000000000003,"15min":0.63},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.879,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66000000000000003,"15min":0.63700000000000001},"min":{"1min":0.53000000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.80700000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Jul 21 22:20:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69999999999999996,"15min":0.70199999999999996},"min":{"1min":0.54700000000000004,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82099999999999995,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.874,"5min":0.75,"15min":0.71299999999999997},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":5.1559999999999997},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Jul 21 22:20:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.61299999999999999,"15min":0.57699999999999996},"min":{"1min":0.48099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64000000000000001,"15min":0.59999999999999998},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.879,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.501}]},{"osd":8,"last update":"Fri Jul 21 22:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77100000000000002,"15min":0.77300000000000002},"min":{"1min":0.51300000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.95199999999999996,"5min":2.202,"15min":2.202},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.752,"15min":0.72799999999999998},"min":{"1min":0.65400000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52300000000000002}]},{"osd":9,"last update":"Fri Jul 21 22:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82499999999999996,"15min":0.83899999999999997},"min":{"1min":0.59799999999999998,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.038,"5min":1.526,"15min":1.526},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83399999999999996,"15min":0.84499999999999997},"min":{"1min":0.68700000000000006,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0529999999999999,"5min":1.056,"15min":1.056},"last":0.70099999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.63300000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":1.097,"15min":1.097},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92300000000000004,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.66600000000000004}]},{"osd":11,"last update":"Fri Jul 21 22:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.61199999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97399999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.68999999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.72199999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1089536,"data_stored":1107927,"data_compressed":59256,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57699999999999996,"15min":0.52800000000000002},"min":{"1min":0.38600000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.81899999999999995,"5min":2.3759999999999999,"15min":2.3759999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.379},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58799999999999997,"15min":0.53700000000000003},"min":{"1min":0.39400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90100000000000002,"5min":3.319,"15min":3.319},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.42999999999999999,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.68400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.42899999999999999,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.46000000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.4330000000000001,"5min":1.647,"15min":1.647},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.60299999999999998},"min":{"1min":0.503,"5min":0.436,"15min":0.436},"max":{"1min":1.0860000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jul 21 22:20:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.56599999999999995},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":2.4169999999999998,"15min":2.4169999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.56799999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.88200000000000001,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Jul 21 22:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71199999999999997,"15min":0.67700000000000005},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.796,"15min":1.796},"last":0.745},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66900000000000004,"15min":0.64500000000000002},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95399999999999996,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Jul 21 22:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66400000000000003,"15min":0.63200000000000001},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.96999999999999997,"5min":1.623,"15min":1.623},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.628},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.79100000000000004}]},{"osd":8,"last update":"Fri Jul 21 22:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Jul 21 22:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.82699999999999996},"min":{"1min":0.60299999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0489999999999999,"5min":1.5700000000000001,"15min":1.5700000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.98799999999999999,"5min":1.8109999999999999,"15min":1.8109999999999999},"last":0.82599999999999996}]},{"osd":10,"last update":"Fri Jul 21 22:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.57299999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.008,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.874},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.93300000000000005,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.85899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34144,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35781,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T22:21:15.681 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:21:15.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 -- 172.21.15.66:0/3300757948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac074550 msgr2=0x7f33ac0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 --2- 172.21.15.66:0/3300757948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac074550 0x7f33ac0729a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f33a0009990 tx=0x7f33a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 -- 172.21.15.66:0/3300757948 shutdown_connections 2023-07-21T22:21:15.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 --2- 172.21.15.66:0/3300757948 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac1079c0 0x7f33ac109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 --2- 172.21.15.66:0/3300757948 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac074550 0x7f33ac0729a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 --2- 172.21.15.66:0/3300757948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac072f70 0x7f33ac0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 -- 172.21.15.66:0/3300757948 >> 172.21.15.66:0/3300757948 conn(0x7f33ac0fb0e0 msgr2=0x7f33ac0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:15.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.874+0000 7f33b2fff700 1 -- 172.21.15.66:0/3300757948 shutdown_connections 2023-07-21T22:21:15.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.875+0000 7f33b2fff700 1 -- 172.21.15.66:0/3300757948 wait complete. 2023-07-21T22:21:15.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.875+0000 7f33b2fff700 1 Processor -- start 2023-07-21T22:21:15.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 -- start start 2023-07-21T22:21:15.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 0x7f33ac197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac074550 0x7f33ac197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac1079c0 0x7f33ac19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac19f5f0 con 0x7f33ac072f70 2023-07-21T22:21:15.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac19f730 con 0x7f33ac1079c0 2023-07-21T22:21:15.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b2fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac19f8c0 con 0x7f33ac074550 2023-07-21T22:21:15.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b159c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac1079c0 0x7f33ac19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b0d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 0x7f33ac197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b159c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac1079c0 0x7f33ac19f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48274/0 (socket says 172.21.15.66:48274) 2023-07-21T22:21:15.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b0d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 0x7f33ac197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56884/0 (socket says 172.21.15.66:56884) 2023-07-21T22:21:15.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33b159c700 1 -- 172.21.15.66:0/494573021 learned_addr learned my addr 172.21.15.66:0/494573021 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:15.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.876+0000 7f33abfff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac074550 0x7f33ac197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b0d9b700 1 -- 172.21.15.66:0/494573021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac074550 msgr2=0x7f33ac197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b0d9b700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac074550 0x7f33ac197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b0d9b700 1 -- 172.21.15.66:0/494573021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac1079c0 msgr2=0x7f33ac19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:15.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b0d9b700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac1079c0 0x7f33ac19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:15.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b0d9b700 1 -- 172.21.15.66:0/494573021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00095f0 con 0x7f33ac072f70 2023-07-21T22:21:15.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33abfff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac074550 0x7f33ac197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:15.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b0d9b700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 0x7f33ac197a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f33a000b680 tx=0x7f33a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:15.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a001d070 con 0x7f33ac072f70 2023-07-21T22:21:15.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33a00054e0 con 0x7f33ac072f70 2023-07-21T22:21:15.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac19fb40 con 0x7f33ac072f70 2023-07-21T22:21:15.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0004c20 con 0x7f33ac072f70 2023-07-21T22:21:15.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.877+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac19ffc0 con 0x7f33ac072f70 2023-07-21T22:21:15.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.878+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac106070 con 0x7f33ac072f70 2023-07-21T22:21:15.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.880+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f33a000fc40 con 0x7f33ac072f70 2023-07-21T22:21:15.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.882+0000 7f33a9ffb700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f339c0a1220 0x7f339c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:15.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.882+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f33a00cdf00 con 0x7f33ac072f70 2023-07-21T22:21:15.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.882+0000 7f33abfff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f339c0a1220 0x7f339c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:15.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.883+0000 7f33abfff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f339c0a1220 0x7f339c0a36d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33ac074270 tx=0x7f3398009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:15.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:15.886+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f33a0097310 con 0x7f33ac072f70 2023-07-21T22:21:15.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:15 smithi066 ceph-mon[126379]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:15.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:15 smithi066 ceph-mon[126379]: from='client.25420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:16.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:15 smithi104 ceph-mon[129021]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:16.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:15 smithi104 ceph-mon[129021]: from='client.25420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:16.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.094+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33ac02c850 con 0x7f33ac072f70 2023-07-21T22:21:16.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.096+0000 7f33a9ffb700 1 -- 172.21.15.66:0/494573021 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7f33a009a930 con 0x7f33ac072f70 2023-07-21T22:21:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:16.142 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:21:16.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f339c0a1220 msgr2=0x7f339c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f339c0a1220 0x7f339c0a36d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f33ac074270 tx=0x7f3398009260 comp rx=0 tx=0).stop 2023-07-21T22:21:16.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 msgr2=0x7f33ac197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 0x7f33ac197a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f33a000b680 tx=0x7f33a0004730 comp rx=0 tx=0).stop 2023-07-21T22:21:16.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 shutdown_connections 2023-07-21T22:21:16.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f339c0a1220 0x7f339c0a36d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33ac1079c0 0x7f33ac19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f33ac072f70 0x7f33ac197a70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 --2- 172.21.15.66:0/494573021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33ac074550 0x7f33ac197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.098+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 >> 172.21.15.66:0/494573021 conn(0x7f33ac0fb0e0 msgr2=0x7f33ac0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:16.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.099+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 shutdown_connections 2023-07-21T22:21:16.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.099+0000 7f33b2fff700 1 -- 172.21.15.66:0/494573021 wait complete. 2023-07-21T22:21:16.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:21:16.164 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:21:16.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:15 smithi149 ceph-mon[129268]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:16.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:15 smithi149 ceph-mon[129268]: from='client.25420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 -- 172.21.15.66:0/3140247184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd600ffae0 msgr2=0x7efd600ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 --2- 172.21.15.66:0/3140247184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd600ffae0 0x7efd600ffef0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efd50009990 tx=0x7efd50009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 -- 172.21.15.66:0/3140247184 shutdown_connections 2023-07-21T22:21:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 --2- 172.21.15.66:0/3140247184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd60101660 0x7efd60107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 --2- 172.21.15.66:0/3140247184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd600ffae0 0x7efd600ffef0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 --2- 172.21.15.66:0/3140247184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60100cd0 0x7efd60101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 -- 172.21.15.66:0/3140247184 >> 172.21.15.66:0/3140247184 conn(0x7efd600fb0e0 msgr2=0x7efd600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 -- 172.21.15.66:0/3140247184 shutdown_connections 2023-07-21T22:21:16.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.330+0000 7efd667ff700 1 -- 172.21.15.66:0/3140247184 wait complete. 2023-07-21T22:21:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 Processor -- start 2023-07-21T22:21:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 -- start start 2023-07-21T22:21:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd600ffae0 0x7efd60197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 0x7efd60197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60101660 0x7efd6019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6019f610 con 0x7efd60100cd0 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6019f750 con 0x7efd60101660 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.331+0000 7efd667ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6019f8e0 con 0x7efd600ffae0 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 0x7efd60197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 0x7efd60197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56896/0 (socket says 172.21.15.66:56896) 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 -- 172.21.15.66:0/1990088662 learned_addr learned my addr 172.21.15.66:0/1990088662 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd64d9c700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60101660 0x7efd6019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5ffff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd600ffae0 0x7efd60197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 -- 172.21.15.66:0/1990088662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd600ffae0 msgr2=0x7efd60197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd600ffae0 0x7efd60197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 -- 172.21.15.66:0/1990088662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60101660 msgr2=0x7efd6019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60101660 0x7efd6019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 -- 172.21.15.66:0/1990088662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500095f0 con 0x7efd60100cd0 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.332+0000 7efd5f7fe700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 0x7efd60197fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd5400e9a0 tx=0x7efd5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.333+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54009790 con 0x7efd60100cd0 2023-07-21T22:21:16.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.333+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6019fbc0 con 0x7efd60100cd0 2023-07-21T22:21:16.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.333+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd54010430 con 0x7efd60100cd0 2023-07-21T22:21:16.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.333+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54018680 con 0x7efd60100cd0 2023-07-21T22:21:16.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.333+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd601a0120 con 0x7efd60100cd0 2023-07-21T22:21:16.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.334+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd601921c0 con 0x7efd60100cd0 2023-07-21T22:21:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.336+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7efd540105a0 con 0x7efd60100cd0 2023-07-21T22:21:16.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.337+0000 7efd5d7fa700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efd480a1150 0x7efd480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.337+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efd54014070 con 0x7efd60100cd0 2023-07-21T22:21:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.338+0000 7efd5ffff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efd480a1150 0x7efd480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.338+0000 7efd5ffff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efd480a1150 0x7efd480a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd50000c00 tx=0x7efd5000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.341+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7efd54051770 con 0x7efd60100cd0 2023-07-21T22:21:16.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.551+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd60061120 con 0x7efd60100cd0 2023-07-21T22:21:16.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.551+0000 7efd5d7fa700 1 -- 172.21.15.66:0/1990088662 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd54096210 con 0x7efd60100cd0 2023-07-21T22:21:16.597 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:21:16.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efd480a1150 msgr2=0x7efd480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efd480a1150 0x7efd480a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd50000c00 tx=0x7efd5000b260 comp rx=0 tx=0).stop 2023-07-21T22:21:16.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 msgr2=0x7efd60197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 0x7efd60197fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7efd5400e9a0 tx=0x7efd5400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 shutdown_connections 2023-07-21T22:21:16.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7efd480a1150 0x7efd480a3600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60101660 0x7efd6019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd600ffae0 0x7efd60197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 --2- 172.21.15.66:0/1990088662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efd60100cd0 0x7efd60197fd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 >> 172.21.15.66:0/1990088662 conn(0x7efd600fb0e0 msgr2=0x7efd60106600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:16.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 shutdown_connections 2023-07-21T22:21:16.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.554+0000 7efd667ff700 1 -- 172.21.15.66:0/1990088662 wait complete. 2023-07-21T22:21:16.618 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 -- 172.21.15.66:0/976767225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b201057a0 msgr2=0x7f6b20107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 --2- 172.21.15.66:0/976767225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b201057a0 0x7f6b20107c00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b0a0 tx=0x7f6b1800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 -- 172.21.15.66:0/976767225 shutdown_connections 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 --2- 172.21.15.66:0/976767225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b201057a0 0x7f6b20107c00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 --2- 172.21.15.66:0/976767225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b20100560 0x7f6b20102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 --2- 172.21.15.66:0/976767225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 0x7f6b20105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 -- 172.21.15.66:0/976767225 >> 172.21.15.66:0/976767225 conn(0x7f6b200fa000 msgr2=0x7f6b200fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 -- 172.21.15.66:0/976767225 shutdown_connections 2023-07-21T22:21:16.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.787+0000 7f6b271ff700 1 -- 172.21.15.66:0/976767225 wait complete. 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 Processor -- start 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 -- start start 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 0x7f6b20197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 0x7f6b20198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b201057a0 0x7f6b2019f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019f670 con 0x7f6b20100560 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.788+0000 7f6b271ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019f7b0 con 0x7f6b20102e80 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b271ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019f940 con 0x7f6b201057a0 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 0x7f6b20197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 0x7f6b20197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56912/0 (socket says 172.21.15.66:56912) 2023-07-21T22:21:16.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 -- 172.21.15.66:0/4150665686 learned_addr learned my addr 172.21.15.66:0/4150665686 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b1ffff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 0x7f6b20198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b2579c700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b201057a0 0x7f6b2019f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 -- 172.21.15.66:0/4150665686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b201057a0 msgr2=0x7f6b2019f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b201057a0 0x7f6b2019f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 -- 172.21.15.66:0/4150665686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 msgr2=0x7f6b20198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 0x7f6b20198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 -- 172.21.15.66:0/4150665686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c009520 con 0x7f6b20100560 2023-07-21T22:21:16.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b1ffff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 0x7f6b20198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b2579c700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b201057a0 0x7f6b2019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.789+0000 7f6b24f9b700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 0x7f6b20197af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e9a0 tx=0x7f6b1400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.790+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14009790 con 0x7f6b20100560 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.790+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b1400cbd0 con 0x7f6b20100560 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.790+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b18009d00 con 0x7f6b20100560 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.790+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14018680 con 0x7f6b20100560 2023-07-21T22:21:16.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.790+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2019ffa0 con 0x7f6b20100560 2023-07-21T22:21:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.791+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20107310 con 0x7f6b20100560 2023-07-21T22:21:16.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.793+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6b1400cd40 con 0x7f6b20100560 2023-07-21T22:21:16.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.796+0000 7f6b1dffb700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b100a1150 0x7f6b100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:16.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.796+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6b14014070 con 0x7f6b20100560 2023-07-21T22:21:16.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.796+0000 7f6b1ffff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b100a1150 0x7f6b100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:16.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.796+0000 7f6b1ffff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b100a1150 0x7f6b100a3600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009e40 tx=0x7f6b0c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:16.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:16.800+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6b14051770 con 0x7f6b20100560 2023-07-21T22:21:16.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:16 smithi066 ceph-mon[126379]: from='client.25426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:16.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:16 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/494573021' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:16.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:16 smithi066 ceph-mon[126379]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-07-21T22:21:16.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:16 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1990088662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.009+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b200611c0 con 0x7f6b20100560 2023-07-21T22:21:17.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.009+0000 7f6b1dffb700 1 -- 172.21.15.66:0/4150665686 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b14099e10 con 0x7f6b20100560 2023-07-21T22:21:17.055 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.011+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b100a1150 msgr2=0x7f6b100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.011+0000 7f6b271ff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b100a1150 0x7f6b100a3600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009e40 tx=0x7f6b0c009260 comp rx=0 tx=0).stop 2023-07-21T22:21:17.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.011+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 msgr2=0x7f6b20197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.011+0000 7f6b271ff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 0x7f6b20197af0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e9a0 tx=0x7f6b1400ed60 comp rx=0 tx=0).stop 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 shutdown_connections 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f6b100a1150 0x7f6b100a3600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6b201057a0 0x7f6b2019f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6b20100560 0x7f6b20197af0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 --2- 172.21.15.66:0/4150665686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b20102e80 0x7f6b20198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 >> 172.21.15.66:0/4150665686 conn(0x7f6b200fa000 msgr2=0x7f6b200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 shutdown_connections 2023-07-21T22:21:17.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.012+0000 7f6b271ff700 1 -- 172.21.15.66:0/4150665686 wait complete. 2023-07-21T22:21:17.076 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:21:17.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:16 smithi104 ceph-mon[129021]: from='client.25426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:17.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:16 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/494573021' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:17.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:16 smithi104 ceph-mon[129021]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-07-21T22:21:17.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:16 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1990088662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:16 smithi149 ceph-mon[129268]: from='client.25426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T22:21:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:16 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/494573021' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:16 smithi149 ceph-mon[129268]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-07-21T22:21:17.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:16 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1990088662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:21:17.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 -- 172.21.15.66:0/917549748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab580ffae0 msgr2=0x7fab580ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/917549748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab580ffae0 0x7fab580ffef0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009990 tx=0x7fab4c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 -- 172.21.15.66:0/917549748 shutdown_connections 2023-07-21T22:21:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/917549748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab58101660 0x7fab58107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/917549748 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab580ffae0 0x7fab580ffef0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/917549748 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58100cd0 0x7fab58101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 -- 172.21.15.66:0/917549748 >> 172.21.15.66:0/917549748 conn(0x7fab580fb0e0 msgr2=0x7fab580fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:17.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 -- 172.21.15.66:0/917549748 shutdown_connections 2023-07-21T22:21:17.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.242+0000 7fab5f1ff700 1 -- 172.21.15.66:0/917549748 wait complete. 2023-07-21T22:21:17.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.244+0000 7fab5f1ff700 1 Processor -- start 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.244+0000 7fab5f1ff700 1 -- start start 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.244+0000 7fab5f1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab580ffae0 0x7fab580742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.244+0000 7fab5f1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 0x7fab58072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5f1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58101660 0x7fab58072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5f1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580735e0 con 0x7fab58100cd0 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5f1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58073750 con 0x7fab580ffae0 2023-07-21T22:21:17.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5f1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819f8f0 con 0x7fab58101660 2023-07-21T22:21:17.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 0x7fab58072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 0x7fab58072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56930/0 (socket says 172.21.15.66:56930) 2023-07-21T22:21:17.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 -- 172.21.15.66:0/455065161 learned_addr learned my addr 172.21.15.66:0/455065161 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:17.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5cf9b700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab580ffae0 0x7fab580742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5d79c700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58101660 0x7fab58072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 -- 172.21.15.66:0/455065161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58101660 msgr2=0x7fab58072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58101660 0x7fab58072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 -- 172.21.15.66:0/455065161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab580ffae0 msgr2=0x7fab580742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab580ffae0 0x7fab580742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab57fff700 1 -- 172.21.15.66:0/455065161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c0095f0 con 0x7fab58100cd0 2023-07-21T22:21:17.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.245+0000 7fab5d79c700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58101660 0x7fab58072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:17.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.246+0000 7fab57fff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 0x7fab58072900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d950 tx=0x7fab4400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:17.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.246+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44009940 con 0x7fab58100cd0 2023-07-21T22:21:17.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.246+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab44010460 con 0x7fab58100cd0 2023-07-21T22:21:17.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.246+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5819fbd0 con 0x7fab58100cd0 2023-07-21T22:21:17.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.246+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4400f620 con 0x7fab58100cd0 2023-07-21T22:21:17.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.246+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab581a0130 con 0x7fab58100cd0 2023-07-21T22:21:17.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.247+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58106a60 con 0x7fab58100cd0 2023-07-21T22:21:17.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.249+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fab440108d0 con 0x7fab58100cd0 2023-07-21T22:21:17.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.251+0000 7fab55ffb700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fab480a11d0 0x7fab480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.251+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fab440ccab0 con 0x7fab58100cd0 2023-07-21T22:21:17.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.251+0000 7fab5cf9b700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fab480a11d0 0x7fab480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.252+0000 7fab5cf9b700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fab480a11d0 0x7fab480a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00b2e0 tx=0x7fab4c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:17.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.254+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fab44095e30 con 0x7fab58100cd0 2023-07-21T22:21:17.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.464+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab580611c0 con 0x7fab58100cd0 2023-07-21T22:21:17.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.464+0000 7fab55ffb700 1 -- 172.21.15.66:0/455065161 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab440160a0 con 0x7fab58100cd0 2023-07-21T22:21:17.510 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:21:17.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fab480a11d0 msgr2=0x7fab480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fab480a11d0 0x7fab480a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00b2e0 tx=0x7fab4c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 msgr2=0x7fab58072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 0x7fab58072900 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab4400d950 tx=0x7fab4400dc60 comp rx=0 tx=0).stop 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 shutdown_connections 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fab480a11d0 0x7fab480a3680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab580ffae0 0x7fab580742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab58101660 0x7fab58072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 --2- 172.21.15.66:0/455065161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab58100cd0 0x7fab58072900 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 >> 172.21.15.66:0/455065161 conn(0x7fab580fb0e0 msgr2=0x7fab580fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:17.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 shutdown_connections 2023-07-21T22:21:17.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.467+0000 7fab5f1ff700 1 -- 172.21.15.66:0/455065161 wait complete. 2023-07-21T22:21:17.531 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:21:17.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 -- 172.21.15.66:0/754538046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c101660 msgr2=0x7f427c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 --2- 172.21.15.66:0/754538046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c101660 0x7f427c107f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f427400b0a0 tx=0x7f427400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 -- 172.21.15.66:0/754538046 shutdown_connections 2023-07-21T22:21:17.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 --2- 172.21.15.66:0/754538046 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c101660 0x7f427c107f20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 --2- 172.21.15.66:0/754538046 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f427c0ffae0 0x7f427c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 --2- 172.21.15.66:0/754538046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c100cd0 0x7f427c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 -- 172.21.15.66:0/754538046 >> 172.21.15.66:0/754538046 conn(0x7f427c0fb0e0 msgr2=0x7f427c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:17.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 -- 172.21.15.66:0/754538046 shutdown_connections 2023-07-21T22:21:17.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.696+0000 7f4280dff700 1 -- 172.21.15.66:0/754538046 wait complete. 2023-07-21T22:21:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.697+0000 7f4280dff700 1 Processor -- start 2023-07-21T22:21:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.697+0000 7f4280dff700 1 -- start start 2023-07-21T22:21:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.697+0000 7f4280dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c0ffae0 0x7f427c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.697+0000 7f4280dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 0x7f427c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4280dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f427c101660 0x7f427c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4280dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0735e0 con 0x7f427c100cd0 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4280dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c073750 con 0x7f427c0ffae0 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4280dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c19f9b0 con 0x7f427c101660 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4273fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 0x7f427c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c0ffae0 0x7f427c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4273fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 0x7f427c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56956/0 (socket says 172.21.15.66:56956) 2023-07-21T22:21:17.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4273fff700 1 -- 172.21.15.66:0/3962172108 learned_addr learned my addr 172.21.15.66:0/3962172108 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427ad9d700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f427c101660 0x7f427c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 -- 172.21.15.66:0/3962172108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f427c101660 msgr2=0x7f427c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f427c101660 0x7f427c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 -- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 msgr2=0x7f427c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 0x7f427c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 -- 172.21.15.66:0/3962172108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4274009d00 con 0x7f427c0ffae0 2023-07-21T22:21:17.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f4273fff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 0x7f427c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:17.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.698+0000 7f427a59c700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c0ffae0 0x7f427c0742c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f426c00e9c0 tx=0x7f426c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:17.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.699+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c004220 con 0x7f427c0ffae0 2023-07-21T22:21:17.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.699+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f426c004380 con 0x7f427c0ffae0 2023-07-21T22:21:17.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.699+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c19fc90 con 0x7f427c0ffae0 2023-07-21T22:21:17.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.699+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c003be0 con 0x7f427c0ffae0 2023-07-21T22:21:17.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.699+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c1a01c0 con 0x7f427c0ffae0 2023-07-21T22:21:17.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.700+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c107000 con 0x7f427c0ffae0 2023-07-21T22:21:17.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.702+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f426c003d40 con 0x7f427c0ffae0 2023-07-21T22:21:17.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.704+0000 7f42737fe700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42680a11d0 0x7f42680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:17.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.704+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f426c014070 con 0x7f427c0ffae0 2023-07-21T22:21:17.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.704+0000 7f4273fff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42680a11d0 0x7f42680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:17.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.704+0000 7f4273fff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42680a11d0 0x7f42680a3680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4264011550 tx=0x7f4264009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:17.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.705+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f426c0963c0 con 0x7f427c0ffae0 2023-07-21T22:21:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.918+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f427c02c850 con 0x7f427c0ffae0 2023-07-21T22:21:17.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.919+0000 7f42737fe700 1 -- 172.21.15.66:0/3962172108 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) v1 ==== 103+0+11 (secure 0 0 0) 0x7f426c01f340 con 0x7f427c0ffae0 2023-07-21T22:21:17.964 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:21:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.921+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42680a11d0 msgr2=0x7f42680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.921+0000 7f4280dff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42680a11d0 0x7f42680a3680 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4264011550 tx=0x7f4264009260 comp rx=0 tx=0).stop 2023-07-21T22:21:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.921+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c0ffae0 msgr2=0x7f427c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.921+0000 7f4280dff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c0ffae0 0x7f427c0742c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f426c00e9c0 tx=0x7f426c00ecd0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 shutdown_connections 2023-07-21T22:21:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f42680a11d0 0x7f42680a3680 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f427c0ffae0 0x7f427c0742c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f427c101660 0x7f427c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 --2- 172.21.15.66:0/3962172108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f427c100cd0 0x7f427c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 >> 172.21.15.66:0/3962172108 conn(0x7f427c0fb0e0 msgr2=0x7f427c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 shutdown_connections 2023-07-21T22:21:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:17.922+0000 7f4280dff700 1 -- 172.21.15.66:0/3962172108 wait complete. 2023-07-21T22:21:17.986 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:21:18.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:17 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4150665686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:18.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:17 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/455065161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:18.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 -- 172.21.15.66:0/1240453244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9700ffb00 msgr2=0x7fb9700fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1240453244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9700ffb00 0x7fb9700fff10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb960009990 tx=0x7fb960009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 -- 172.21.15.66:0/1240453244 shutdown_connections 2023-07-21T22:21:18.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1240453244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 0x7fb970107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1240453244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb9700ffb00 0x7fb9700fff10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1240453244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb970100cf0 0x7fb970101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.147+0000 7fb9761ff700 1 -- 172.21.15.66:0/1240453244 >> 172.21.15.66:0/1240453244 conn(0x7fb9700fb0e0 msgr2=0x7fb9700fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.148+0000 7fb9761ff700 1 -- 172.21.15.66:0/1240453244 shutdown_connections 2023-07-21T22:21:18.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.148+0000 7fb9761ff700 1 -- 172.21.15.66:0/1240453244 wait complete. 2023-07-21T22:21:18.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 Processor -- start 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 -- start start 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9700ffb00 0x7fb970197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 0x7fb970197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 0x7fb97019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97019f5f0 con 0x7fb970100cf0 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97019f730 con 0x7fb970101680 2023-07-21T22:21:18.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb9761ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97019f8c0 con 0x7fb9700ffb00 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.149+0000 7fb96effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 0x7fb970197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 0x7fb970197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56980/0 (socket says 172.21.15.66:56980) 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 -- 172.21.15.66:0/1694011604 learned_addr learned my addr 172.21.15.66:0/1694011604 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96ffff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 0x7fb97019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96f7fe700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9700ffb00 0x7fb970197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 -- 172.21.15.66:0/1694011604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9700ffb00 msgr2=0x7fb970197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9700ffb00 0x7fb970197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 -- 172.21.15.66:0/1694011604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 msgr2=0x7fb97019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.200 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:17 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4150665686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:18.200 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:17 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/455065161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:18.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 0x7fb97019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 -- 172.21.15.66:0/1694011604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9600095f0 con 0x7fb970100cf0 2023-07-21T22:21:18.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96ffff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 0x7fb97019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:18.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.150+0000 7fb96effd700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 0x7fb970197fb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e9a0 tx=0x7fb96400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.151+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964009790 con 0x7fb970100cf0 2023-07-21T22:21:18.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.151+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb964010430 con 0x7fb970100cf0 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.151+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97019fba0 con 0x7fb970100cf0 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.151+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb964018680 con 0x7fb970100cf0 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.151+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9701a0100 con 0x7fb970100cf0 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.152+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9701921e0 con 0x7fb970100cf0 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.154+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb9640105a0 con 0x7fb970100cf0 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.155+0000 7fb96cff9700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb9580a1220 0x7fb9580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.156+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb964014070 con 0x7fb970100cf0 2023-07-21T22:21:18.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.156+0000 7fb96f7fe700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb9580a1220 0x7fb9580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.156+0000 7fb96f7fe700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb9580a1220 0x7fb9580a36d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb96000b2e0 tx=0x7fb960005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.160+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb964051770 con 0x7fb970100cf0 2023-07-21T22:21:18.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:17 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4150665686' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:21:18.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:17 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/455065161' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:21:18.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.415+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb97002c850 con 0x7fb970100cf0 2023-07-21T22:21:18.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.416+0000 7fb96cff9700 1 -- 172.21.15.66:0/1694011604 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7fb9640961b0 con 0x7fb970100cf0 2023-07-21T22:21:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:18.461 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:21:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.418+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb9580a1220 msgr2=0x7fb9580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.418+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb9580a1220 0x7fb9580a36d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb96000b2e0 tx=0x7fb960005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.418+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 msgr2=0x7fb970197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.418+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 0x7fb970197fb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb96400e9a0 tx=0x7fb96400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 shutdown_connections 2023-07-21T22:21:18.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fb9580a1220 0x7fb9580a36d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb970101680 0x7fb97019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9700ffb00 0x7fb970197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 --2- 172.21.15.66:0/1694011604 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb970100cf0 0x7fb970197fb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 >> 172.21.15.66:0/1694011604 conn(0x7fb9700fb0e0 msgr2=0x7fb9701065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 shutdown_connections 2023-07-21T22:21:18.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.419+0000 7fb9761ff700 1 -- 172.21.15.66:0/1694011604 wait complete. 2023-07-21T22:21:18.466 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:21:18.483 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:21:18.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.645+0000 7fae8f7ff700 1 -- 172.21.15.66:0/2894437061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae88100cf0 msgr2=0x7fae88101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.645+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/2894437061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae88100cf0 0x7fae88101140 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fae78009990 tx=0x7fae78009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 -- 172.21.15.66:0/2894437061 shutdown_connections 2023-07-21T22:21:18.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/2894437061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae880ffb00 0x7fae880fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/2894437061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae88101680 0x7fae88107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/2894437061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae88100cf0 0x7fae88101140 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 -- 172.21.15.66:0/2894437061 >> 172.21.15.66:0/2894437061 conn(0x7fae880fb0e0 msgr2=0x7fae880fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 -- 172.21.15.66:0/2894437061 shutdown_connections 2023-07-21T22:21:18.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.646+0000 7fae8f7ff700 1 -- 172.21.15.66:0/2894437061 wait complete. 2023-07-21T22:21:18.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 Processor -- start 2023-07-21T22:21:18.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 -- start start 2023-07-21T22:21:18.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae880ffb00 0x7fae88197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 0x7fae88197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae88101680 0x7fae8819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819f5c0 con 0x7fae880ffb00 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819f700 con 0x7fae88100cf0 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.647+0000 7fae8f7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819f890 con 0x7fae88101680 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 0x7fae88197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8d59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae880ffb00 0x7fae88197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 0x7fae88197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48356/0 (socket says 172.21.15.66:48356) 2023-07-21T22:21:18.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 -- 172.21.15.66:0/3685232716 learned_addr learned my addr 172.21.15.66:0/3685232716 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8dd9c700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae88101680 0x7fae8819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 -- 172.21.15.66:0/3685232716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae88101680 msgr2=0x7fae8819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae88101680 0x7fae8819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 -- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae880ffb00 msgr2=0x7fae88197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae880ffb00 0x7fae88197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 -- 172.21.15.66:0/3685232716 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780095f0 con 0x7fae88100cf0 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8d59b700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae880ffb00 0x7fae88197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:18.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.648+0000 7fae8cd9a700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 0x7fae88197f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae7800b620 tx=0x7fae78005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.649+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7801d070 con 0x7fae88100cf0 2023-07-21T22:21:18.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.649+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8819fb10 con 0x7fae88100cf0 2023-07-21T22:21:18.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.649+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae78022470 con 0x7fae88100cf0 2023-07-21T22:21:18.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.649+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78017650 con 0x7fae88100cf0 2023-07-21T22:21:18.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.649+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8819ff30 con 0x7fae88100cf0 2023-07-21T22:21:18.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.650+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae881921e0 con 0x7fae88100cf0 2023-07-21T22:21:18.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.652+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fae780225e0 con 0x7fae88100cf0 2023-07-21T22:21:18.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.653+0000 7fae7e7fc700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fae740a11a0 0x7fae740a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:18.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.653+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fae780ce850 con 0x7fae88100cf0 2023-07-21T22:21:18.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.654+0000 7fae8d59b700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fae740a11a0 0x7fae740a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:18.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.654+0000 7fae8d59b700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fae740a11a0 0x7fae740a3650 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae88100b50 tx=0x7fae84009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:18.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.655+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fae78097d80 con 0x7fae88100cf0 2023-07-21T22:21:18.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.871+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae8810b390 con 0x7fae88100cf0 2023-07-21T22:21:18.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.873+0000 7fae7e7fc700 1 -- 172.21.15.66:0/3685232716 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+24583 (secure 0 0 0) 0x7fae7809b3a0 con 0x7fae88100cf0 2023-07-21T22:21:18.919 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:18.919 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":93,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:20:38.285621+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:21:18.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.875+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fae740a11a0 msgr2=0x7fae740a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.875+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fae740a11a0 0x7fae740a3650 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fae88100b50 tx=0x7fae84009260 comp rx=0 tx=0).stop 2023-07-21T22:21:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.875+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 msgr2=0x7fae88197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.875+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 0x7fae88197f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fae7800b620 tx=0x7fae78005740 comp rx=0 tx=0).stop 2023-07-21T22:21:18.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 shutdown_connections 2023-07-21T22:21:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7fae740a11a0 0x7fae740a3650 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fae88101680 0x7fae8819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fae880ffb00 0x7fae88197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 --2- 172.21.15.66:0/3685232716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae88100cf0 0x7fae88197f80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 >> 172.21.15.66:0/3685232716 conn(0x7fae880fb0e0 msgr2=0x7fae881065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:18.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 shutdown_connections 2023-07-21T22:21:18.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:18.876+0000 7fae8f7ff700 1 -- 172.21.15.66:0/3685232716 wait complete. 2023-07-21T22:21:18.941 DEBUG:tasks.fwd_scrub:Yielding 2023-07-21T22:21:18.941 INFO:teuthology.run_tasks:Running task exec... 2023-07-21T22:21:18.951 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T22:21:18.951 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:21:18.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-21T22:21:18.952 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-21T22:21:18.952 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:21:18.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:21:19.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:18 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3962172108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:19.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:18 smithi104 ceph-mon[129021]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-07-21T22:21:19.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:18 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1694011604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:19.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 -- 172.21.15.66:0/3212558941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 msgr2=0x7f60440fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 --2- 172.21.15.66:0/3212558941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f60440fff30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f60340098f0 tx=0x7f6034009c00 comp rx=0 tx=0).stop 2023-07-21T22:21:19.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 -- 172.21.15.66:0/3212558941 shutdown_connections 2023-07-21T22:21:19.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 --2- 172.21.15.66:0/3212558941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f60440fff30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 --2- 172.21.15.66:0/3212558941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60441016a0 0x7f6044107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 --2- 172.21.15.66:0/3212558941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6044100d10 0x7f6044101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 -- 172.21.15.66:0/3212558941 >> 172.21.15.66:0/3212558941 conn(0x7f60440fb120 msgr2=0x7f60440fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 -- 172.21.15.66:0/3212558941 shutdown_connections 2023-07-21T22:21:19.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.114+0000 7f604a7ff700 1 -- 172.21.15.66:0/3212558941 wait complete. 2023-07-21T22:21:19.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.115+0000 7f604a7ff700 1 Processor -- start 2023-07-21T22:21:19.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.115+0000 7f604a7ff700 1 -- start start 2023-07-21T22:21:19.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f604a7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f6044197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f604a7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 0x7f6044198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f604a7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60441016a0 0x7f6044198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f604a7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044199600 con 0x7f60441016a0 2023-07-21T22:21:19.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f604a7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044199770 con 0x7f60440ffb20 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f604a7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6044199930 con 0x7f6044100d10 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f60437fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 0x7f6044198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f6044197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6048d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60441016a0 0x7f6044198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f60437fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 0x7f6044198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53482/0 (socket says 172.21.15.66:53482) 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f6044197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48374/0 (socket says 172.21.15.66:48374) 2023-07-21T22:21:19.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f60437fe700 1 -- 172.21.15.66:0/228417592 learned_addr learned my addr 172.21.15.66:0/228417592 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 -- 172.21.15.66:0/228417592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 msgr2=0x7f6044198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 0x7f6044198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 -- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60441016a0 msgr2=0x7f6044198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60441016a0 0x7f6044198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6043fff700 1 -- 172.21.15.66:0/228417592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034009520 con 0x7f60440ffb20 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f60437fe700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 0x7f6044198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.116+0000 7f6048d9c700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60441016a0 0x7f6044198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.117+0000 7f6043fff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f6044197be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6034000c00 tx=0x7f603400b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.117+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603401c070 con 0x7f60440ffb20 2023-07-21T22:21:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.117+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6044199bb0 con 0x7f60440ffb20 2023-07-21T22:21:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.117+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f603400f7f0 con 0x7f60440ffb20 2023-07-21T22:21:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.117+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60340039b0 con 0x7f60440ffb20 2023-07-21T22:21:19.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.117+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60441a03d0 con 0x7f60440ffb20 2023-07-21T22:21:19.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.118+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604404e2f0 con 0x7f60440ffb20 2023-07-21T22:21:19.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.120+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6034020030 con 0x7f60440ffb20 2023-07-21T22:21:19.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.122+0000 7f60417fa700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f602c0a1150 0x7f602c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.122+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f60340cdbd0 con 0x7f60440ffb20 2023-07-21T22:21:19.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.122+0000 7f60437fe700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f602c0a1150 0x7f602c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.123+0000 7f60437fe700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f602c0a1150 0x7f602c0a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6038005950 tx=0x7f6038009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.126+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6034096fe0 con 0x7f60440ffb20 2023-07-21T22:21:19.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1026662013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78102ee0 msgr2=0x7ffb781052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1026662013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78102ee0 0x7ffb781052c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009990 tx=0x7ffb6c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1026662013 shutdown_connections 2023-07-21T22:21:19.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1026662013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb78105800 0x7ffb78107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1026662013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb781005c0 0x7ffb781029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1026662013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78102ee0 0x7ffb781052c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1026662013 >> 172.21.15.66:0/1026662013 conn(0x7ffb780fa000 msgr2=0x7ffb780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1026662013 shutdown_connections 2023-07-21T22:21:19.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.127+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1026662013 wait complete. 2023-07-21T22:21:19.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.128+0000 7ffb7e3ff700 1 Processor -- start 2023-07-21T22:21:19.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 -- start start 2023-07-21T22:21:19.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb781005c0 0x7ffb78197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 0x7ffb78197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78105800 0x7ffb7819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7819f600 con 0x7ffb78102ee0 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7819f740 con 0x7ffb78105800 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7e3ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7819f8d0 con 0x7ffb781005c0 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb777fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 0x7ffb78197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb781005c0 0x7ffb78197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78105800 0x7ffb7819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb777fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 0x7ffb78197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57040/0 (socket says 172.21.15.66:57040) 2023-07-21T22:21:19.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb777fe700 1 -- 172.21.15.66:0/1207343597 learned_addr learned my addr 172.21.15.66:0/1207343597 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.129+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb781005c0 0x7ffb78197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53502/0 (socket says 172.21.15.66:53502) 2023-07-21T22:21:19.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb777fe700 1 -- 172.21.15.66:0/1207343597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb781005c0 msgr2=0x7ffb78197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb777fe700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb781005c0 0x7ffb78197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb777fe700 1 -- 172.21.15.66:0/1207343597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78105800 msgr2=0x7ffb7819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb777fe700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78105800 0x7ffb7819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb777fe700 1 -- 172.21.15.66:0/1207343597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c0095f0 con 0x7ffb78102ee0 2023-07-21T22:21:19.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78105800 0x7ffb7819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb777fe700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 0x7ffb78197fc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005910 tx=0x7ffb6c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb757fa700 1 -- 172.21.15.66:0/1207343597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c01d070 con 0x7ffb78102ee0 2023-07-21T22:21:19.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb757fa700 1 -- 172.21.15.66:0/1207343597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb6c022470 con 0x7ffb78102ee0 2023-07-21T22:21:19.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.130+0000 7ffb757fa700 1 -- 172.21.15.66:0/1207343597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c017650 con 0x7ffb78102ee0 2023-07-21T22:21:19.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.131+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7819fb50 con 0x7ffb78102ee0 2023-07-21T22:21:19.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.131+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7819ff20 con 0x7ffb78102ee0 2023-07-21T22:21:19.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.134+0000 7ffb757fa700 1 -- 172.21.15.66:0/1207343597 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffb6c00fad0 con 0x7ffb78102ee0 2023-07-21T22:21:19.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.135+0000 7ffb91da1b80 1 Processor -- start 2023-07-21T22:21:19.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.135+0000 7ffb91da1b80 1 -- start start 2023-07-21T22:21:19.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1301e6d30 con 0x55e1302e3220 2023-07-21T22:21:19.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e130239ac0 con 0x55e1302e28b0 2023-07-21T22:21:19.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e130232f50 con 0x55e1302e1a90 2023-07-21T22:21:19.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302e14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57046/0 (socket says 172.21.15.66:57046) 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb77fff700 1 -- 172.21.15.66:0/3333067227 learned_addr learned my addr 172.21.15.66:0/3333067227 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.136+0000 7ffb777fe700 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb7c99c700 1 -- 172.21.15.66:0/3333067227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 msgr2=0x55e1302e1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302e1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb7c99c700 1 -- 172.21.15.66:0/3333067227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 msgr2=0x55e1302e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb7c99c700 1 -- 172.21.15.66:0/3333067227 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e130060e60 con 0x55e1302e28b0 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb77fff700 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302e2cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb70007600 tx=0x7ffb70007910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3eb7f4c043a7c4e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb757fa700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffb600a1150 0x7ffb600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb5e7fc700 1 -- 172.21.15.66:0/3333067227 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70015910 con 0x55e1302e28b0 2023-07-21T22:21:19.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb757fa700 1 -- 172.21.15.66:0/1207343597 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffb6c0cdff0 con 0x7ffb78102ee0 2023-07-21T22:21:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb5e7fc700 1 -- 172.21.15.66:0/3333067227 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7000f040 con 0x55e1302e28b0 2023-07-21T22:21:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb5e7fc700 1 -- 172.21.15.66:0/3333067227 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70014db0 con 0x55e1302e28b0 2023-07-21T22:21:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.137+0000 7ffb77fff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffb600a1150 0x7ffb600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3333067227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 msgr2=0x55e1302e2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb77fff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffb600a1150 0x7ffb600a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb68011500 tx=0x7ffb680073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302e2cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb70007600 tx=0x7ffb70007910 comp rx=0 tx=0).stop 2023-07-21T22:21:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3333067227 shutdown_connections 2023-07-21T22:21:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302e2cc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302e1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3333067227 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3333067227 >> 172.21.15.66:0/3333067227 conn(0x55e1302d6f70 msgr2=0x55e1302dc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3333067227 shutdown_connections 2023-07-21T22:21:19.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.138+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3333067227 wait complete. 2023-07-21T22:21:19.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.139+0000 7ffb91da1b80 1 -- start start 2023-07-21T22:21:19.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.139+0000 7ffb91da1b80 1 Processor -- start 2023-07-21T22:21:19.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1301d2b40 con 0x55e1302e3220 2023-07-21T22:21:19.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1301d2840 con 0x55e1302e28b0 2023-07-21T22:21:19.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1300b69a0 con 0x55e1302e1a90 2023-07-21T22:21:19.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb91da1b80 10 client.0 _ll_register_callbacks cb 0x55e1301d07e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:19.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57048/0 (socket says 172.21.15.66:57048) 2023-07-21T22:21:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 -- 172.21.15.66:0/243854773 learned_addr learned my addr 172.21.15.66:0/243854773 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb77fff700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb777fe700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 -- 172.21.15.66:0/243854773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 msgr2=0x55e1302eb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302eb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 -- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 msgr2=0x55e1302eb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302eb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb7c99c700 1 -- 172.21.15.66:0/243854773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb680079a0 con 0x55e1302e3220 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.140+0000 7ffb777fe700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302f2720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000f80 tx=0x7ffb70004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a06040119f59fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb577fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb577fe700 1 -- 172.21.15.66:0/243854773 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70015710 con 0x55e1302e3220 2023-07-21T22:21:19.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e13018c570 con 0x55e1302e3220 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb577fe700 1 -- 172.21.15.66:0/243854773 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffb70015870 con 0x55e1302e3220 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb577fe700 1 -- 172.21.15.66:0/243854773 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb70015a30 con 0x55e1302e3220 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb91da1b80 10 client.15441 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.141+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ffb70009d10 con 0x55e1302e3220 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb577fe700 1 -- 172.21.15.66:0/243854773 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e13018c570 con 0x55e1302e3220 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb577fe700 1 -- 172.21.15.66:0/243854773 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb70005040 con 0x55e1302e3220 2023-07-21T22:21:19.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb91da1b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb91da1b80 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb91da1b80 20 client.15441 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e1302f6ef0 0x55e1302f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb91da1b80 4 client.15441 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e13022bf30 con 0x55e1302f6ef0 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.142+0000 7ffb77fff700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e1302f6ef0 0x55e1302f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.143+0000 7ffb77fff700 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e1302f6ef0 0x55e1302f9300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6958034a7d4a4bd server_cookie=19a421016655aae3 in_seq=0 out_seq=0 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.143+0000 7ffb577fe700 10 client.15441 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb577fe700 1 -- 172.21.15.66:0/243854773 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e13022bf30 con 0x55e1302f6ef0 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb577fe700 10 client.15441 handle_command_reply: tid=0 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 client.15441 shutdown 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 msgr2=0x55e1302f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302f2720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000f80 tx=0x7ffb70004d80 comp rx=0 tx=0).stop 2023-07-21T22:21:19.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 shutdown_connections 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e1302f6ef0 0x55e1302f9300 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e3220 0x55e1302f2720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302eb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/243854773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 >> 172.21.15.66:0/243854773 conn(0x55e1302d6f70 msgr2=0x55e1302dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb577fe700 0 client.15441 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 shutdown_connections 2023-07-21T22:21:19.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 1 -- 172.21.15.66:0/243854773 wait complete. 2023-07-21T22:21:19.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.144+0000 7ffb91da1b80 20 client.15441 trim_cache size 0 max 16384 2023-07-21T22:21:19.199 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:18 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3962172108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:19.199 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:18 smithi066 ceph-mon[126379]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-07-21T22:21:19.199 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:18 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1694011604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:19.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.163+0000 7ffb91da1b80 1 Processor -- start 2023-07-21T22:21:19.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.163+0000 7ffb91da1b80 1 -- start start 2023-07-21T22:21:19.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 0x55e1302e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e3220 0x55e1302f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1302d6a30 con 0x55e1302e1a90 2023-07-21T22:21:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1302ead10 con 0x55e1302e28b0 2023-07-21T22:21:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1302ea620 con 0x55e1302e3220 2023-07-21T22:21:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 0x55e1302e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 0x55e1302e13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57056/0 (socket says 172.21.15.66:57056) 2023-07-21T22:21:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb77fff700 1 -- 172.21.15.66:0/3897699040 learned_addr learned my addr 172.21.15.66:0/3897699040 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e3220 0x55e1302f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.164+0000 7ffb777fe700 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb77fff700 1 -- 172.21.15.66:0/3897699040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e3220 msgr2=0x55e1302f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb77fff700 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e3220 0x55e1302f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb77fff700 1 -- 172.21.15.66:0/3897699040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 msgr2=0x55e1302f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb77fff700 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb77fff700 1 -- 172.21.15.66:0/3897699040 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e130049ce0 con 0x55e1302e1a90 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb77fff700 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 0x55e1302e13d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffb6801e550 tx=0x7ffb68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68783b2b3909ced5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb5effd700 1 -- 172.21.15.66:0/3897699040 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6802d070 con 0x55e1302e1a90 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb5effd700 1 -- 172.21.15.66:0/3897699040 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb68029e10 con 0x55e1302e1a90 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.165+0000 7ffb5effd700 1 -- 172.21.15.66:0/3897699040 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68024040 con 0x55e1302e1a90 2023-07-21T22:21:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3897699040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 msgr2=0x55e1302e13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 0x55e1302e13d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffb6801e550 tx=0x7ffb68016000 comp rx=0 tx=0).stop 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3897699040 shutdown_connections 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e3220 0x55e1302f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e1a90 0x55e1302e13d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/3897699040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e28b0 0x55e1302f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3897699040 >> 172.21.15.66:0/3897699040 conn(0x55e1302d6f70 msgr2=0x55e1302f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3897699040 shutdown_connections 2023-07-21T22:21:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.166+0000 7ffb91da1b80 1 -- 172.21.15.66:0/3897699040 wait complete. 2023-07-21T22:21:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.167+0000 7ffb91da1b80 1 -- start start 2023-07-21T22:21:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.167+0000 7ffb91da1b80 1 Processor -- start 2023-07-21T22:21:19.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e28b0 0x55e1302ef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 0x55e1302ef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1302efad0 con 0x55e1302e28b0 2023-07-21T22:21:19.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1302efc40 con 0x55e1302e3220 2023-07-21T22:21:19.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13034adb0 con 0x55e1302e1a90 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb91da1b80 10 client.0 _ll_register_callbacks cb 0x7ffb680077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb777fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e28b0 0x55e1302ef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 0x55e1302ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53540/0 (socket says 172.21.15.66:53540) 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb777fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e28b0 0x55e1302ef000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57070/0 (socket says 172.21.15.66:57070) 2023-07-21T22:21:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb77fff700 1 -- 172.21.15.66:0/1297921662 learned_addr learned my addr 172.21.15.66:0/1297921662 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 0x55e1302ef540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48414/0 (socket says 172.21.15.66:48414) 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 -- 172.21.15.66:0/1297921662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 msgr2=0x55e1302f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 -- 172.21.15.66:0/1297921662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e28b0 msgr2=0x55e1302ef000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e28b0 0x55e1302ef000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb7c99c700 1 -- 172.21.15.66:0/1297921662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb68011e40 con 0x55e1302e3220 2023-07-21T22:21:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.168+0000 7ffb77fff700 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb7c99c700 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 0x55e1302ef540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb70007520 tx=0x7ffb70006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d21a4dba31640dc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb577fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb577fe700 1 -- 172.21.15.66:0/1297921662 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb70014760 con 0x55e1302e3220 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb7000f040 con 0x55e1302e3220 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb577fe700 1 -- 172.21.15.66:0/1297921662 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffb700148c0 con 0x55e1302e3220 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb577fe700 1 -- 172.21.15.66:0/1297921662 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb70004ea0 con 0x55e1302e3220 2023-07-21T22:21:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb91da1b80 10 client.25508 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.169+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e1300872d0 con 0x55e1302e3220 2023-07-21T22:21:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb577fe700 1 -- 172.21.15.66:0/1297921662 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb7000f040 con 0x55e1302e3220 2023-07-21T22:21:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb577fe700 1 -- 172.21.15.66:0/1297921662 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb700256a0 con 0x55e1302e3220 2023-07-21T22:21:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb91da1b80 10 client.25508 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb91da1b80 10 client.25508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb91da1b80 20 client.25508 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e13034c4a0 0x55e13034e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb91da1b80 4 client.25508 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e1301ee0d0 con 0x55e13034c4a0 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.170+0000 7ffb77fff700 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e13034c4a0 0x55e13034e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.171+0000 7ffb77fff700 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e13034c4a0 0x55e13034e8b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60fe87a6c8ff155d server_cookie=6e1b36b7c0df70ba in_seq=0 out_seq=0 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.171+0000 7ffb577fe700 10 client.25508 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb577fe700 1 -- 172.21.15.66:0/1297921662 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e1301ee0d0 con 0x55e13034c4a0 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb577fe700 10 client.25508 handle_command_reply: tid=0 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 client.25508 shutdown 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 msgr2=0x55e1302ef540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 0x55e1302ef540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb70007520 tx=0x7ffb70006000 comp rx=0 tx=0).stop 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 shutdown_connections 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e1302e3220 0x55e1302ef540 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55e13034c4a0 0x55e13034e8b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e1302e1a90 0x55e1302f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 --2- 172.21.15.66:0/1297921662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e1302e28b0 0x55e1302ef000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb577fe700 0 client.25508 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 >> 172.21.15.66:0/1297921662 conn(0x55e1302d6f70 msgr2=0x55e1302f2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 shutdown_connections 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.172+0000 7ffb91da1b80 1 -- 172.21.15.66:0/1297921662 wait complete. 2023-07-21T22:21:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb91da1b80 20 client.25508 trim_cache size 0 max 16384 2023-07-21T22:21:19.226 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:19.226 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:21:19.226 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "362bb9e9-faa8-4d2d-86e5-56ed6ddcb8c5", 2023-07-21T22:21:19.226 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:21:19.226 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:19.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffb600a1150 msgr2=0x7ffb600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffb600a1150 0x7ffb600a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffb68011500 tx=0x7ffb680073d0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 msgr2=0x7ffb78197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 0x7ffb78197fc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c005910 tx=0x7ffb6c00f690 comp rx=0 tx=0).stop 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 shutdown_connections 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7ffb600a1150 0x7ffb600a3600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb78105800 0x7ffb7819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb781005c0 0x7ffb78197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 --2- 172.21.15.66:0/1207343597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ffb78102ee0 0x7ffb78197fc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 >> 172.21.15.66:0/1207343597 conn(0x7ffb780fa000 msgr2=0x7ffb78101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 shutdown_connections 2023-07-21T22:21:19.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.173+0000 7ffb7e3ff700 1 -- 172.21.15.66:0/1207343597 wait complete. 2023-07-21T22:21:19.237 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:21:19.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:18 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3962172108' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:21:19.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:18 smithi149 ceph-mon[129268]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-07-21T22:21:19.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:18 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1694011604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:19.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 -- 172.21.15.66:0/2359227011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc1079a0 msgr2=0x7f74bc109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 --2- 172.21.15.66:0/2359227011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc1079a0 0x7f74bc109e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b0a0 tx=0x7f74b400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 -- 172.21.15.66:0/2359227011 shutdown_connections 2023-07-21T22:21:19.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 --2- 172.21.15.66:0/2359227011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc1079a0 0x7f74bc109e20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 --2- 172.21.15.66:0/2359227011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74bc074550 0x7f74bc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 --2- 172.21.15.66:0/2359227011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc072f70 0x7f74bc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 -- 172.21.15.66:0/2359227011 >> 172.21.15.66:0/2359227011 conn(0x7f74bc0fb0e0 msgr2=0x7f74bc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 -- 172.21.15.66:0/2359227011 shutdown_connections 2023-07-21T22:21:19.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.404+0000 7f74c1dff700 1 -- 172.21.15.66:0/2359227011 wait complete. 2023-07-21T22:21:19.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.405+0000 7f74c1dff700 1 Processor -- start 2023-07-21T22:21:19.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 -- start start 2023-07-21T22:21:19.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74bc072f70 0x7f74bc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc074550 0x7f74bc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc1079a0 0x7f74bc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc19f690 con 0x7f74bc1079a0 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc19f7d0 con 0x7f74bc074550 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.406+0000 7f74c1dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc19f960 con 0x7f74bc072f70 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc1079a0 0x7f74bc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc074550 0x7f74bc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74bc072f70 0x7f74bc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc1079a0 0x7f74bc19f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57082/0 (socket says 172.21.15.66:57082) 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74bbfff700 1 -- 172.21.15.66:0/428220469 learned_addr learned my addr 172.21.15.66:0/428220469 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 -- 172.21.15.66:0/428220469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74bc072f70 msgr2=0x7f74bc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74bc072f70 0x7f74bc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 -- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc1079a0 msgr2=0x7f74bc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc1079a0 0x7f74bc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 -- 172.21.15.66:0/428220469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0009520 con 0x7f74bc074550 2023-07-21T22:21:19.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.407+0000 7f74baffd700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc074550 0x7f74bc198050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e9c0 tx=0x7f74b000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.408+0000 7f74b8ff9700 1 -- 172.21.15.66:0/428220469 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b000cb70 con 0x7f74bc074550 2023-07-21T22:21:19.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.408+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b4009d00 con 0x7f74bc074550 2023-07-21T22:21:19.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.408+0000 7f74b8ff9700 1 -- 172.21.15.66:0/428220469 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b000ee60 con 0x7f74bc074550 2023-07-21T22:21:19.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.408+0000 7f74b8ff9700 1 -- 172.21.15.66:0/428220469 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0018680 con 0x7f74bc074550 2023-07-21T22:21:19.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.408+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc19ffc0 con 0x7f74bc074550 2023-07-21T22:21:19.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.411+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f6044199d40 con 0x7f60440ffb20 2023-07-21T22:21:19.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.411+0000 7f74b8ff9700 1 -- 172.21.15.66:0/428220469 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f74b0010430 con 0x7f74bc074550 2023-07-21T22:21:19.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 Processor -- start 2023-07-21T22:21:19.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74b8ff9700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f74a40a1150 0x7f74a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 -- start start 2023-07-21T22:21:19.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74b8ff9700 1 -- 172.21.15.66:0/428220469 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f74b0014070 con 0x7f74bc074550 2023-07-21T22:21:19.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74bbfff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f74a40a1150 0x7f74a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 0x5647744c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c81a0 0x5647744c6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c6a10 0x5647744c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647743cbcb0 con 0x5647744c7830 2023-07-21T22:21:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477441ea40 con 0x5647744c6a10 2023-07-21T22:21:19.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.413+0000 7f74d562bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564774417ed0 con 0x5647744c81a0 2023-07-21T22:21:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 0x5647744c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 0x5647744c7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57094/0 (socket says 172.21.15.66:57094) 2023-07-21T22:21:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1062293640 learned_addr learned my addr 172.21.15.66:0/1062293640 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74baffd700 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c81a0 0x5647744c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bbfff700 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c6a10 0x5647744c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bbfff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f74a40a1150 0x7f74a40a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009fd0 tx=0x7f74b4006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1062293640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c81a0 msgr2=0x5647744c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c81a0 0x5647744c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1062293640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c6a10 msgr2=0x5647744c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c6a10 0x5647744c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1062293640 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564774245de0 con 0x5647744c7830 2023-07-21T22:21:19.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bbfff700 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c6a10 0x5647744c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.414+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 0x5647744c7c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009d20 tx=0x7f74ac011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a590e080fec7294 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.415+0000 7f74a1ffb700 1 -- 172.21.15.66:0/1062293640 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac019630 con 0x5647744c7830 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.415+0000 7f74a1ffb700 1 -- 172.21.15.66:0/1062293640 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74ac019c50 con 0x5647744c7830 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.415+0000 7f74a1ffb700 1 -- 172.21.15.66:0/1062293640 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00eb90 con 0x5647744c7830 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.415+0000 7f74d562bb80 1 -- 172.21.15.66:0/1062293640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 msgr2=0x5647744c7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.415+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 0x5647744c7c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009d20 tx=0x7f74ac011720 comp rx=0 tx=0).stop 2023-07-21T22:21:19.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 -- 172.21.15.66:0/1062293640 shutdown_connections 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c6a10 0x5647744c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c7830 0x5647744c7c40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1062293640 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c81a0 0x5647744c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 -- 172.21.15.66:0/1062293640 >> 172.21.15.66:0/1062293640 conn(0x5647744bbef0 msgr2=0x5647744c1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 -- 172.21.15.66:0/1062293640 shutdown_connections 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 -- 172.21.15.66:0/1062293640 wait complete. 2023-07-21T22:21:19.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.416+0000 7f74d562bb80 1 -- start start 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 Processor -- start 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 0x5647744d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c7830 0x5647744d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c81a0 0x5647744d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647743b7ac0 con 0x5647744c6a10 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647743b77c0 con 0x5647744c81a0 2023-07-21T22:21:19.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.417+0000 7f74d562bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56477429b920 con 0x5647744c7830 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74d562bb80 10 client.0 _ll_register_callbacks cb 0x5647743b5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 0x5647744d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 0x5647744d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57108/0 (socket says 172.21.15.66:57108) 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 -- 172.21.15.66:0/681797586 learned_addr learned my addr 172.21.15.66:0/681797586 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bbfff700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c81a0 0x5647744d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74baffd700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c7830 0x5647744d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 -- 172.21.15.66:0/681797586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c7830 msgr2=0x5647744d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c7830 0x5647744d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 -- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c81a0 msgr2=0x5647744d7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c81a0 0x5647744d7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 -- 172.21.15.66:0/681797586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac009980 con 0x5647744c6a10 2023-07-21T22:21:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 0x5647744d00f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0046e0 tx=0x7f74ac00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affd3c8cf5326978 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.418+0000 7f749affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f749affd700 1 -- 172.21.15.66:0/681797586 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac019840 con 0x5647744c6a10 2023-07-21T22:21:19.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f749affd700 1 -- 172.21.15.66:0/681797586 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f74ac00e460 con 0x5647744c6a10 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647743714f0 con 0x5647744c6a10 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f749affd700 1 -- 172.21.15.66:0/681797586 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74ac005ce0 con 0x5647744c6a10 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 10 client.15459 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56477422ec60 con 0x5647744c6a10 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f749affd700 1 -- 172.21.15.66:0/681797586 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647743714f0 con 0x5647744c6a10 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f749affd700 1 -- 172.21.15.66:0/681797586 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74ac029020 con 0x5647744c6a10 2023-07-21T22:21:19.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 10 client.15459 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 20 client.15459 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.419+0000 7f74d562bb80 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647744dbc60 0x5647744de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.420+0000 7f74d562bb80 4 client.15459 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.420+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564774410eb0 con 0x5647744dbc60 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.420+0000 7f74baffd700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647744dbc60 0x5647744de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.420+0000 7f74baffd700 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647744dbc60 0x5647744de070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9128b951cc9e307 server_cookie=ba33f9c76bda165b in_seq=0 out_seq=0 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.420+0000 7f749affd700 10 client.15459 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f749affd700 1 -- 172.21.15.66:0/681797586 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564774410eb0 con 0x5647744dbc60 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f749affd700 10 client.15459 handle_command_reply: tid=0 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 client.15459 shutdown 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 msgr2=0x5647744d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 0x5647744d00f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0046e0 tx=0x7f74ac00ee10 comp rx=0 tx=0).stop 2023-07-21T22:21:19.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 shutdown_connections 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c81a0 0x5647744d7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647744dbc60 0x5647744de070 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c6a10 0x5647744d00f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 --2- 172.21.15.66:0/681797586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c7830 0x5647744d0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f749affd700 0 client.15459 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 >> 172.21.15.66:0/681797586 conn(0x5647744bbef0 msgr2=0x5647744c0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 shutdown_connections 2023-07-21T22:21:19.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.421+0000 7f74d562bb80 1 -- 172.21.15.66:0/681797586 wait complete. 2023-07-21T22:21:19.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.422+0000 7f74d562bb80 20 client.15459 trim_cache size 0 max 16384 2023-07-21T22:21:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.439+0000 7f74d562bb80 1 Processor -- start 2023-07-21T22:21:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.439+0000 7f74d562bb80 1 -- start start 2023-07-21T22:21:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x5647744cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74d562bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647744b8ec0 con 0x5647744c81a0 2023-07-21T22:21:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74d562bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647744ce8a0 con 0x5647744c7830 2023-07-21T22:21:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74d562bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647744db9b0 con 0x5647744c6a10 2023-07-21T22:21:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x5647744cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x5647744cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:57112/0 (socket says 172.21.15.66:57112) 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 -- 172.21.15.66:0/183121630 learned_addr learned my addr 172.21.15.66:0/183121630 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74baffd700 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 -- 172.21.15.66:0/183121630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 msgr2=0x5647744d7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 -- 172.21.15.66:0/183121630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 msgr2=0x5647744d7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.440+0000 7f74bbfff700 1 -- 172.21.15.66:0/183121630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56477426c250 con 0x5647744c81a0 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74bbfff700 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x5647744cab30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b070 tx=0x7f74b4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909be538135e4a10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74a27fc700 1 -- 172.21.15.66:0/183121630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400e040 con 0x5647744c81a0 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74a27fc700 1 -- 172.21.15.66:0/183121630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74b4003e90 con 0x5647744c81a0 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74a27fc700 1 -- 172.21.15.66:0/183121630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b40055d0 con 0x5647744c81a0 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74d562bb80 1 -- 172.21.15.66:0/183121630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 msgr2=0x5647744cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74d562bb80 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x5647744cab30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74b400b070 tx=0x7f74b4021000 comp rx=0 tx=0).stop 2023-07-21T22:21:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74d562bb80 1 -- 172.21.15.66:0/183121630 shutdown_connections 2023-07-21T22:21:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.441+0000 7f74d562bb80 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x5647744cab30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 --2- 172.21.15.66:0/183121630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 -- 172.21.15.66:0/183121630 >> 172.21.15.66:0/183121630 conn(0x5647744bbef0 msgr2=0x5647744cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 -- 172.21.15.66:0/183121630 shutdown_connections 2023-07-21T22:21:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 -- 172.21.15.66:0/183121630 wait complete. 2023-07-21T22:21:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 -- start start 2023-07-21T22:21:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.442+0000 7f74d562bb80 1 Processor -- start 2023-07-21T22:21:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x56477452fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647744d4f70 con 0x5647744c81a0 2023-07-21T22:21:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564774530050 con 0x5647744c7830 2023-07-21T22:21:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564774530190 con 0x5647744c6a10 2023-07-21T22:21:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74d562bb80 10 client.0 _ll_register_callbacks cb 0x7f74b000cee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x56477452fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53596/0 (socket says 172.21.15.66:53596) 2023-07-21T22:21:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1033850909 learned_addr learned my addr 172.21.15.66:0/1033850909 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74baffd700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 msgr2=0x5647744d4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1033850909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 msgr2=0x56477452fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x56477452fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bb7fe700 1 -- 172.21.15.66:0/1033850909 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc10aa40 con 0x5647744c6a10 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.443+0000 7f74bbfff700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x56477452fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f74bb7fe700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d4410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74ac001eb0 tx=0x7f74ac00e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b894f95c16acf31e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f749affd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f749affd700 1 -- 172.21.15.66:0/1033850909 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00e620 con 0x5647744c6a10 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74ac019c50 con 0x5647744c6a10 2023-07-21T22:21:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f749affd700 1 -- 172.21.15.66:0/1033850909 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f74ac000e40 con 0x5647744c6a10 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f749affd700 1 -- 172.21.15.66:0/1033850909 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74ac005ce0 con 0x5647744c6a10 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f74d562bb80 10 client.25510 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.444+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5647742442b0 con 0x5647744c6a10 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f749affd700 1 -- 172.21.15.66:0/1033850909 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74ac019c50 con 0x5647744c6a10 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f749affd700 1 -- 172.21.15.66:0/1033850909 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74ac029070 con 0x5647744c6a10 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74d562bb80 10 client.25510 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74d562bb80 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74d562bb80 20 client.25510 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647745316c0 0x564774533ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74d562bb80 4 client.25510 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647743d3050 con 0x5647745316c0 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.445+0000 7f74baffd700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647745316c0 0x564774533ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74baffd700 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647745316c0 0x564774533ad0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58d2cf758d1abbe server_cookie=ef0e249256d15132 in_seq=0 out_seq=0 2023-07-21T22:21:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f749affd700 10 client.25510 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f749affd700 1 -- 172.21.15.66:0/1033850909 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5647743d3050 con 0x5647745316c0 2023-07-21T22:21:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f749affd700 10 client.25510 handle_command_reply: tid=0 2023-07-21T22:21:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 client.25510 shutdown 2023-07-21T22:21:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 msgr2=0x5647744d4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d4410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74ac001eb0 tx=0x7f74ac00e3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 shutdown_connections 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5647745316c0 0x564774533ad0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5647744c81a0 0x56477452fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5647744c6a10 0x5647744d4410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 --2- 172.21.15.66:0/1033850909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5647744c7830 0x5647744d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f749affd700 0 client.25510 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 >> 172.21.15.66:0/1033850909 conn(0x5647744bbef0 msgr2=0x5647744cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 shutdown_connections 2023-07-21T22:21:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 1 -- 172.21.15.66:0/1033850909 wait complete. 2023-07-21T22:21:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.446+0000 7f74d562bb80 20 client.25510 trim_cache size 0 max 16384 2023-07-21T22:21:19.502 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:19.502 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:21:19.502 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:21:19.503 INFO:teuthology.orchestra.run.smithi066.stdout: "362bb9e9-faa8-4d2d-86e5-56ed6ddcb8c5": { 2023-07-21T22:21:19.503 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:21:19.503 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "362bb9e9-faa8-4d2d-86e5-56ed6ddcb8c5", 2023-07-21T22:21:19.503 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:21:19.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:21:19.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:21:19.504 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:19.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f74a40a1150 msgr2=0x7f74a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f74a40a1150 0x7f74a40a3600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009fd0 tx=0x7f74b4006210 comp rx=0 tx=0).stop 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc074550 msgr2=0x7f74bc198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc074550 0x7f74bc198050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f74b000e9c0 tx=0x7f74b000ecd0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 shutdown_connections 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f74a40a1150 0x7f74a40a3600 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74bc1079a0 0x7f74bc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74bc072f70 0x7f74bc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 --2- 172.21.15.66:0/428220469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74bc074550 0x7f74bc198050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 >> 172.21.15.66:0/428220469 conn(0x7f74bc0fb0e0 msgr2=0x7f74bc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 shutdown_connections 2023-07-21T22:21:19.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.447+0000 7f74c1dff700 1 -- 172.21.15.66:0/428220469 wait complete. 2023-07-21T22:21:19.510 INFO:tasks.cephfs.filesystem:scrub status for tag:362bb9e9-faa8-4d2d-86e5-56ed6ddcb8c5 - {'path': '/', 'tag': '362bb9e9-faa8-4d2d-86e5-56ed6ddcb8c5', 'options': 'recursive,force'} 2023-07-21T22:21:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.836+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f60340973b0 con 0x7f60440ffb20 2023-07-21T22:21:19.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.836+0000 7f60417fa700 1 -- 172.21.15.66:0/228417592 <== mon.2 v2:172.21.15.104:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f603405e4e0 con 0x7f60440ffb20 2023-07-21T22:21:19.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f602c0a1150 msgr2=0x7f602c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f602c0a1150 0x7f602c0a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6038005950 tx=0x7f6038009310 comp rx=0 tx=0).stop 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 msgr2=0x7f6044197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f6044197be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6034000c00 tx=0x7f603400b730 comp rx=0 tx=0).stop 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 shutdown_connections 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f602c0a1150 0x7f602c0a3600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60440ffb20 0x7f6044197be0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f60441016a0 0x7f6044198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 --2- 172.21.15.66:0/228417592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6044100d10 0x7f6044198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:19.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 >> 172.21.15.66:0/228417592 conn(0x7f60440fb120 msgr2=0x7f60440fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:19.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 shutdown_connections 2023-07-21T22:21:19.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:19.839+0000 7f604a7ff700 1 -- 172.21.15.66:0/228417592 wait complete. 2023-07-21T22:21:19.904 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-21T22:21:20.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3685232716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:20.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:21:20.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:20.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:21:20.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:20.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/228417592' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:21:20.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:21:20.120 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: ignoring --setuser ceph since I am not root 2023-07-21T22:21:20.121 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: ignoring --setgroup ceph since I am not root 2023-07-21T22:21:20.121 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:19 smithi104 ceph-mgr[130501]: -- 172.21.15.104:0/170175 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5601fd1971e0 con 0x5601fd16cc00 2023-07-21T22:21:20.121 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.003+0000 7f7880d9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:21:20.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 -- 172.21.15.66:0/2721404975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f700ffb10 msgr2=0x7f2f700fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 --2- 172.21.15.66:0/2721404975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f700ffb10 0x7f2f700fff20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009990 tx=0x7f2f60009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 -- 172.21.15.66:0/2721404975 shutdown_connections 2023-07-21T22:21:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 --2- 172.21.15.66:0/2721404975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f70101690 0x7f2f70107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 --2- 172.21.15.66:0/2721404975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f700ffb10 0x7f2f700fff20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 --2- 172.21.15.66:0/2721404975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70100d00 0x7f2f70101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 -- 172.21.15.66:0/2721404975 >> 172.21.15.66:0/2721404975 conn(0x7f2f700fb130 msgr2=0x7f2f700fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 -- 172.21.15.66:0/2721404975 shutdown_connections 2023-07-21T22:21:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.075+0000 7f2f753ff700 1 -- 172.21.15.66:0/2721404975 wait complete. 2023-07-21T22:21:20.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 Processor -- start 2023-07-21T22:21:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 -- start start 2023-07-21T22:21:20.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 0x7f2f70197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f70100d00 0x7f2f70198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70101690 0x7f2f70198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70199540 con 0x7f2f70101690 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f701996b0 con 0x7f2f700ffb10 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.077+0000 7f2f753ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70199870 con 0x7f2f70100d00 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 0x7f2f70197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:20.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f70100d00 0x7f2f70198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 0x7f2f70197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48510/0 (socket says 172.21.15.66:48510) 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 -- 172.21.15.66:0/3631878847 learned_addr learned my addr 172.21.15.66:0/3631878847 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6f7fe700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70101690 0x7f2f70198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 -- 172.21.15.66:0/3631878847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f70100d00 msgr2=0x7f2f70198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f70100d00 0x7f2f70198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 -- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70101690 msgr2=0x7f2f70198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70101690 0x7f2f70198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 -- 172.21.15.66:0/3631878847 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f64009520 con 0x7f2f700ffb10 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6f7fe700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70101690 0x7f2f70198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.078+0000 7f2f6effd700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 0x7f2f70197b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f60004710 tx=0x7f2f60009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.079+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6001d070 con 0x7f2f700ffb10 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.079+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f60021410 con 0x7f2f700ffb10 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.079+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f600095f0 con 0x7f2f700ffb10 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.079+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60017650 con 0x7f2f700ffb10 2023-07-21T22:21:20.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.079+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f701a0130 con 0x7f2f700ffb10 2023-07-21T22:21:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.080+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f70192190 con 0x7f2f700ffb10 2023-07-21T22:21:20.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.082+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f2f60021580 con 0x7f2f700ffb10 2023-07-21T22:21:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.083+0000 7f2f57fff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 0x7f2f580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.083+0000 7f2f6e7fc700 1 -- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 msgr2=0x7f2f580a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.083+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2f600cdfe0 con 0x7f2f700ffb10 2023-07-21T22:21:20.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.083+0000 7f2f6e7fc700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 0x7f2f580a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:20.130 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: ignoring --setuser ceph since I am not root 2023-07-21T22:21:20.130 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: ignoring --setgroup ceph since I am not root 2023-07-21T22:21:20.130 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mgr[131212]: -- 172.21.15.149:0/336221391 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bece6db1e0 con 0x55bece6b1000 2023-07-21T22:21:20.130 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.005+0000 7fb39f25a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:21:20.130 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.084+0000 7fb39f25a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:20.130 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3685232716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:20.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:21:20.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:20.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:21:20.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:20.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/228417592' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:21:20.131 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:19 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:21:20.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.085+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2f60097450 con 0x7f2f700ffb10 2023-07-21T22:21:20.164 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setuser ceph since I am not root 2023-07-21T22:21:20.164 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: ignoring --setgroup ceph since I am not root 2023-07-21T22:21:20.164 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3685232716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:21:20.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:21:20.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:20.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:21:20.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:20.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/228417592' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:21:20.165 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:19 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T22:21:20.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.285+0000 7f2f6e7fc700 1 -- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 msgr2=0x7f2f580a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:20.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.285+0000 7f2f6e7fc700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 0x7f2f580a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:20.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.294+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f2f7002cad0 con 0x7f2f700ffb10 2023-07-21T22:21:20.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.299+0000 7f2f57fff700 1 -- 172.21.15.66:0/3631878847 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2f6009aa70 con 0x7f2f700ffb10 2023-07-21T22:21:20.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 msgr2=0x7f2f580a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:20.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 0x7f2f580a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 msgr2=0x7f2f70197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 0x7f2f70197b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2f60004710 tx=0x7f2f60009580 comp rx=0 tx=0).stop 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 shutdown_connections 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f2f580a1220 0x7f2f580a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f700ffb10 0x7f2f70197b20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2f70101690 0x7f2f70198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 --2- 172.21.15.66:0/3631878847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f70100d00 0x7f2f70198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 >> 172.21.15.66:0/3631878847 conn(0x7f2f700fb130 msgr2=0x7f2f701065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:20.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 shutdown_connections 2023-07-21T22:21:20.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.302+0000 7f2f753ff700 1 -- 172.21.15.66:0/3631878847 wait complete. 2023-07-21T22:21:20.368 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-21T22:21:20.431 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:20.118+0000 7f7667875200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T22:21:20.431 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:20.229+0000 7f7667875200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:20.432 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:20.353+0000 7f7667875200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:21:20.480 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.082+0000 7f7880d9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:20.481 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.174+0000 7f7880d9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:21:20.481 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.266+0000 7f7880d9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:21:20.481 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.435+0000 7f7880d9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:21:20.484 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.176+0000 7fb39f25a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T22:21:20.484 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.270+0000 7fb39f25a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:21:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 -- 172.21.15.66:0/198076226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c074550 msgr2=0x7f187c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 --2- 172.21.15.66:0/198076226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c074550 0x7f187c0729a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186c009990 tx=0x7f186c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 -- 172.21.15.66:0/198076226 shutdown_connections 2023-07-21T22:21:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 --2- 172.21.15.66:0/198076226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c074550 0x7f187c0729a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 --2- 172.21.15.66:0/198076226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c1079c0 0x7f187c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 --2- 172.21.15.66:0/198076226 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 0x7f187c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 -- 172.21.15.66:0/198076226 >> 172.21.15.66:0/198076226 conn(0x7f187c0fb0e0 msgr2=0x7f187c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:20.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 -- 172.21.15.66:0/198076226 shutdown_connections 2023-07-21T22:21:20.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.529+0000 7f18811ff700 1 -- 172.21.15.66:0/198076226 wait complete. 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.530+0000 7f18811ff700 1 Processor -- start 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.530+0000 7f18811ff700 1 -- start start 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f18811ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 0x7f187c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f18811ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c074550 0x7f187c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f18811ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c1079c0 0x7f187c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f18811ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c1995a0 con 0x7f187c072f70 2023-07-21T22:21:20.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f18811ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c199710 con 0x7f187c1079c0 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f18811ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c1998d0 con 0x7f187c074550 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f187a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c074550 0x7f187c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f187b59e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c1079c0 0x7f187c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f187a59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c074550 0x7f187c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53644/0 (socket says 172.21.15.66:53644) 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f187a59c700 1 -- 172.21.15.66:0/1740107928 learned_addr learned my addr 172.21.15.66:0/1740107928 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f187ad9d700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 0x7f187c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:20.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.531+0000 7f187b59e700 1 -- 172.21.15.66:0/1740107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c074550 msgr2=0x7f187c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f187b59e700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c074550 0x7f187c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f187b59e700 1 -- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 msgr2=0x7f187c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f187b59e700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 0x7f187c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f187b59e700 1 -- 172.21.15.66:0/1740107928 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0095f0 con 0x7f187c1079c0 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f187ad9d700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 0x7f187c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f187b59e700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c1079c0 0x7f187c198f40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1874007f10 tx=0x7f187400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187400ddd0 con 0x7f187c1079c0 2023-07-21T22:21:20.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f187400f040 con 0x7f187c1079c0 2023-07-21T22:21:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c199bb0 con 0x7f187c1079c0 2023-07-21T22:21:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1874005a90 con 0x7f187c1079c0 2023-07-21T22:21:20.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.532+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c1a0370 con 0x7f187c1079c0 2023-07-21T22:21:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.534+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c106070 con 0x7f187c1079c0 2023-07-21T22:21:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.535+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f1874004ad0 con 0x7f187c1079c0 2023-07-21T22:21:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.537+0000 7f1863fff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 0x7f18640a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.537+0000 7f187ad9d700 1 -- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 msgr2=0x7f18640a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.537+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f18740cc580 con 0x7f187c1079c0 2023-07-21T22:21:20.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.537+0000 7f187ad9d700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 0x7f18640a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:20.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.540+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f187404e770 con 0x7f187c1079c0 2023-07-21T22:21:20.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:20.482+0000 7f7667875200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T22:21:20.739 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.440+0000 7fb39f25a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:21:20.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.738+0000 7f187ad9d700 1 -- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 msgr2=0x7f18640a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:20.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.738+0000 7f187ad9d700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 0x7f18640a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:20.785 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.740+0000 7f7880d9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:21:20.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.772+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f187c02cb00 con 0x7f187c1079c0 2023-07-21T22:21:20.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.778+0000 7f1863fff700 1 -- 172.21.15.66:0/1740107928 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1874095900 con 0x7f187c1079c0 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 msgr2=0x7f18640a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 0x7f18640a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c1079c0 msgr2=0x7f187c198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c1079c0 0x7f187c198f40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1874007f10 tx=0x7f187400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 shutdown_connections 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f18640a1220 0x7f18640a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f187c1079c0 0x7f187c198f40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f187c072f70 0x7f187c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 --2- 172.21.15.66:0/1740107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f187c074550 0x7f187c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:20.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 >> 172.21.15.66:0/1740107928 conn(0x7f187c0fb0e0 msgr2=0x7f187c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:20.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 shutdown_connections 2023-07-21T22:21:20.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:20.780+0000 7f18811ff700 1 -- 172.21.15.66:0/1740107928 wait complete. 2023-07-21T22:21:20.845 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-21T22:21:21.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 -- 172.21.15.66:0/2889029195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fe410 msgr2=0x7f4fe40fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 --2- 172.21.15.66:0/2889029195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fe410 0x7f4fe40fe820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009990 tx=0x7f4fd4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 -- 172.21.15.66:0/2889029195 shutdown_connections 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 --2- 172.21.15.66:0/2889029195 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe40ff710 0x7f4fe40ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 --2- 172.21.15.66:0/2889029195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fe410 0x7f4fe40fe820 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 --2- 172.21.15.66:0/2889029195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe40fed60 0x7f4fe40ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 -- 172.21.15.66:0/2889029195 >> 172.21.15.66:0/2889029195 conn(0x7f4fe40fa000 msgr2=0x7f4fe40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:21.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 -- 172.21.15.66:0/2889029195 shutdown_connections 2023-07-21T22:21:21.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.015+0000 7f4fe9dff700 1 -- 172.21.15.66:0/2889029195 wait complete. 2023-07-21T22:21:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.016+0000 7f4fe9dff700 1 Processor -- start 2023-07-21T22:21:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.016+0000 7f4fe9dff700 1 -- start start 2023-07-21T22:21:21.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe40fe410 0x7f4fe4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fed60 0x7f4fe41980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe40ff710 0x7f4fe4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4199580 con 0x7f4fe40ff710 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41996f0 con 0x7f4fe40fe410 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41998b0 con 0x7f4fe40fed60 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fed60 0x7f4fe41980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:21.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe40fe410 0x7f4fe4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fed60 0x7f4fe41980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:53664/0 (socket says 172.21.15.66:53664) 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe40fe410 0x7f4fe4197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48556/0 (socket says 172.21.15.66:48556) 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 -- 172.21.15.66:0/826250018 learned_addr learned my addr 172.21.15.66:0/826250018 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe3fff700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe40ff710 0x7f4fe4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe40fe410 msgr2=0x7f4fe4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4fe40fe410 0x7f4fe4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:21.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe40ff710 msgr2=0x7f4fe4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4fe40ff710 0x7f4fe4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.017+0000 7f4fe2ffd700 1 -- 172.21.15.66:0/826250018 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd40095f0 con 0x7f4fe40fed60 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.018+0000 7f4fe2ffd700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fed60 0x7f4fe41980a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800e9a0 tx=0x7f4fd800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.018+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8009790 con 0x7f4fe40fed60 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.018+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4fd800cbd0 con 0x7f4fe40fed60 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.018+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe4199b90 con 0x7f4fe40fed60 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.018+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8018680 con 0x7f4fe40fed60 2023-07-21T22:21:21.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.018+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41a0350 con 0x7f4fe40fed60 2023-07-21T22:21:21.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.019+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe41091f0 con 0x7f4fe40fed60 2023-07-21T22:21:21.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.021+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f4fd800cd40 con 0x7f4fe40fed60 2023-07-21T22:21:21.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.022+0000 7f4fe0ff9700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:21.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.022+0000 7f4fe37fe700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:21.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.022+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4fd8014070 con 0x7f4fe40fed60 2023-07-21T22:21:21.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.022+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T22:21:21.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.026+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4fd8051770 con 0x7f4fe40fed60 2023-07-21T22:21:21.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T22:21:21.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-mon[129021]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-07-21T22:21:21.098 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' 2023-07-21T22:21:21.099 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' 2023-07-21T22:21:21.099 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.830+0000 7f7880d9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:21:21.099 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:20.959+0000 7f7880d9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:21:21.108 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.748+0000 7fb39f25a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:21:21.108 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.838+0000 7fb39f25a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:21:21.108 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:20.969+0000 7fb39f25a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:21:21.108 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T22:21:21.109 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-mon[129268]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-07-21T22:21:21.109 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' 2023-07-21T22:21:21.109 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:20 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' 2023-07-21T22:21:21.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:20.735+0000 7f7667875200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T22:21:21.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T22:21:21.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-mon[126379]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-07-21T22:21:21.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' 2023-07-21T22:21:21.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:20 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' 2023-07-21T22:21:21.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.224+0000 7f4fe37fe700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:21.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.224+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T22:21:21.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.266+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 --> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe4060950 con 0x7f4fcc0a1220 2023-07-21T22:21:21.489 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:21.063+0000 7fb39f25a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:21:21.500 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:21.162+0000 7f7667875200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T22:21:21.500 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:21.284+0000 7f7667875200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T22:21:21.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.625+0000 7f4fe37fe700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:21.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:21.625+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T22:21:21.869 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:21 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:21.053+0000 7f7880d9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:21:21.869 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:21 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:21.472+0000 7f7880d9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:21:21.945 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:21.454+0000 7f7667875200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T22:21:21.945 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:21.583+0000 7f7667875200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T22:21:21.989 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:21.490+0000 7fb39f25a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:21:22.368 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:22.028+0000 7f7880d9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:21:22.369 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:22.196+0000 7f7880d9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:21:22.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:22.143+0000 7f7667875200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T22:21:22.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:22.426+0000 7f4fe37fe700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:22.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:22.426+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T22:21:22.489 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:22.057+0000 7fb39f25a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:21:22.489 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:22.231+0000 7fb39f25a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:21:23.262 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:22.957+0000 7f7880d9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:23.262 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:23 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:23.048+0000 7f7880d9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:21:23.308 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:23.003+0000 7fb39f25a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:23.308 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:23.094+0000 7fb39f25a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:21:23.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:22.923+0000 7f7667875200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T22:21:23.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:23.140+0000 7f7667875200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T22:21:23.589 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:23.263+0000 7fb39f25a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:23.868 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:23 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:23.217+0000 7f7880d9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:23.869 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:23 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:23.494+0000 7f7880d9d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:21:23.869 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:23 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:23.716+0000 7f7880d9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:21:23.969 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:23.544+0000 7fb39f25a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:21:23.970 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:23.778+0000 7fb39f25a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:21:24.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:24.028+0000 7f4fe37fe700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:24.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:24.028+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T22:21:24.166 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:23 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:23.864+0000 7f7880d9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:21:24.230 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:23.924+0000 7fb39f25a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:21:24.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:24.136+0000 7f7667875200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:24.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:24.261+0000 7f7667875200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T22:21:24.489 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:24.185+0000 7fb39f25a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:21:24.619 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:24.121+0000 7f7880d9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:21:24.915 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:24.494+0000 7f7667875200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T22:21:25.118 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:24.812+0000 7f7880d9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:21:25.119 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:24.940+0000 7f7880d9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:21:25.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:24.870+0000 7f7667875200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T22:21:25.239 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:24.894+0000 7fb39f25a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:21:25.239 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:25.025+0000 7fb39f25a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:21:25.499 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:25.129+0000 7f7880d9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:21:25.499 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:25.241+0000 7f7880d9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:25.499 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:25.454+0000 7f7880d9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:21:25.589 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:25.215+0000 7fb39f25a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:21:25.590 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:25.329+0000 7fb39f25a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:25.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:25.159+0000 7f7667875200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T22:21:25.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:25.359+0000 7f7667875200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T22:21:25.989 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:25.545+0000 7fb39f25a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:21:26.195 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:25.717+0000 7f7667875200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T22:21:26.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:26.377+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mgrmap(e 20) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f4fd8096660 con 0x7f4fe40fed60 2023-07-21T22:21:26.619 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y[130477]: 2023-07-21T22:21:26.300+0000 7f7880d9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:21:26.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-mon[129021]: Standby manager daemon y restarted 2023-07-21T22:21:26.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-mon[129021]: Standby manager daemon y started 2023-07-21T22:21:26.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:21:26.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:21:26.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:21:26.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:26 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:21:26.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-mon[126379]: Standby manager daemon y restarted 2023-07-21T22:21:26.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-mon[126379]: Standby manager daemon y started 2023-07-21T22:21:26.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:21:26.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:21:26.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:21:26.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:21:26.739 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z[131126]: 2023-07-21T22:21:26.422+0000 7fb39f25a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:21:26.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-mon[129268]: Standby manager daemon y restarted 2023-07-21T22:21:26.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-mon[129268]: Standby manager daemon y started 2023-07-21T22:21:26.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T22:21:26.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:21:26.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T22:21:26.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:26 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.104:0/3440424728' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:21:27.179 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:26.692+0000 7f7667875200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T22:21:27.179 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:26 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:26.873+0000 7f7667875200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T22:21:27.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:27.231+0000 7f4fe37fe700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.66:6800/1262827329 2023-07-21T22:21:27.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:27.231+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T22:21:27.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:27.397+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 8 ==== mgrmap(e 21) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f4fd80ca6c0 con 0x7f4fe40fed60 2023-07-21T22:21:27.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:27.133+0000 7f7667875200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T22:21:27.445 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:27.288+0000 7f7667875200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T22:21:27.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-07-21T22:21:27.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: Standby manager daemon z restarted 2023-07-21T22:21:27.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: Standby manager daemon z started 2023-07-21T22:21:27.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:21:27.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:21:27.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:21:27.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-mon[126379]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:21:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-07-21T22:21:27.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: Standby manager daemon z restarted 2023-07-21T22:21:27.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: Standby manager daemon z started 2023-07-21T22:21:27.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:21:27.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:21:27.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:21:27.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:27 smithi149 ceph-mon[129268]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:21:27.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: mgrmap e20: x(active, since 8m), standbys: z, y 2023-07-21T22:21:27.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: Standby manager daemon z restarted 2023-07-21T22:21:27.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: Standby manager daemon z started 2023-07-21T22:21:27.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T22:21:27.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T22:21:27.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T22:21:27.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:27 smithi104 ceph-mon[129021]: from='mgr.? 172.21.15.149:0/4169952207' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T22:21:27.945 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:27 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:27.582+0000 7f7667875200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T22:21:28.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-mon[126379]: mgrmap e21: x(active, since 8m), standbys: z, y 2023-07-21T22:21:28.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:28 smithi149 ceph-mon[129268]: mgrmap e21: x(active, since 8m), standbys: z, y 2023-07-21T22:21:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:28.718+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 9 ==== mgrmap(e 22) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f4fd8091880 con 0x7f4fe40fed60 2023-07-21T22:21:28.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:28.718+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 msgr2=0x7f4fcc0a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:21:28.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:28.718+0000 7f4fe0ff9700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/1262827329,v1:172.21.15.66:6801/1262827329] conn(0x7f4fcc0a1220 0x7f4fcc0a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:28.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:28 smithi104 ceph-mon[129021]: mgrmap e21: x(active, since 8m), standbys: z, y 2023-07-21T22:21:29.180 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.700+0000 7f7667875200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T22:21:29.180 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.921+0000 7f75c6d95700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.181 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.921+0000 7f75c6d95700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.181 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.921+0000 7f75c6d95700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.181 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.921+0000 7f75c6d95700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.181 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.921+0000 7f75c6d95700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.181 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.922+0000 7f75c3d8f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.181 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.922+0000 7f75c3d8f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.182 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.922+0000 7f75c3d8f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.182 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.922+0000 7f75c3d8f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.182 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:21:28 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:21:28.922+0000 7f75c3d8f700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:21:29.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: Active manager daemon x restarted 2023-07-21T22:21:29.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: Activating manager daemon x 2023-07-21T22:21:29.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T22:21:29.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: mgrmap e22: x(active, starting, since 0.0125367s), standbys: z, y 2023-07-21T22:21:29.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:21:29.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:21:29.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:21:29.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:21:29.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:21:29.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:21:29.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:21:29.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:21:29.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:21:29.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:21:29.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:21:29.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:21:29.700 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:21:29.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:21:29.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:21:29.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:21:29.701 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:21:29.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:21:29.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:21:29.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:21:29.702 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:21:29.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:21:29.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:21:29.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:21:29.703 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:21:29.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:21:29.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:21:29.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:21:29.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:21:29.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:21:29.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:21:29.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:21:29.705 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:21:29.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: Manager daemon x is now available 2023-07-21T22:21:29.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:29.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:29.706 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:21:29.707 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:29 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:21:29.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: Active manager daemon x restarted 2023-07-21T22:21:29.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: Activating manager daemon x 2023-07-21T22:21:29.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T22:21:29.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: mgrmap e22: x(active, starting, since 0.0125367s), standbys: z, y 2023-07-21T22:21:29.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:21:29.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:21:29.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:21:29.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:21:29.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:21:29.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:21:29.743 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:21:29.743 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:21:29.743 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:21:29.743 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:21:29.744 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:21:29.744 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:21:29.744 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:21:29.745 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:21:29.745 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:21:29.745 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:21:29.745 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:21:29.746 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:21:29.746 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:21:29.746 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:21:29.746 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:21:29.747 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:21:29.747 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:21:29.747 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:21:29.747 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:21:29.748 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:21:29.748 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:21:29.748 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:21:29.748 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:21:29.749 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:21:29.749 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:21:29.749 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:21:29.749 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:21:29.750 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: Manager daemon x is now available 2023-07-21T22:21:29.750 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:29.750 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:29.751 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:21:29.751 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:29 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:21:29.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.725+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mon.1 v2:172.21.15.149:3300/0 10 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f4fd8096f30 con 0x7f4fe40fed60 2023-07-21T22:21:29.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.726+0000 7f4fe0ff9700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4fcc067120 0x7f4fcc0674f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:29.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.726+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 --> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe4060950 con 0x7f4fcc067120 2023-07-21T22:21:29.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.729+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4fcc067120 0x7f4fcc0674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:29.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.737+0000 7f4fe37fe700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4fcc067120 0x7f4fcc0674f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b2e0 tx=0x7f4fd4005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:29.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.805+0000 7f4fe0ff9700 1 -- 172.21.15.66:0/826250018 <== mgr.15483 v2:172.21.15.66:6800/3475780841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4fe4060950 con 0x7f4fcc067120 2023-07-21T22:21:29.851 INFO:teuthology.orchestra.run.smithi066.stdout:Schedule set for path / 2023-07-21T22:21:29.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4fcc067120 msgr2=0x7f4fcc0674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:29.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4fcc067120 0x7f4fcc0674f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b2e0 tx=0x7f4fd4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fed60 msgr2=0x7f4fe41980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 --2- 172.21.15.66:0/826250018 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe40fed60 0x7f4fe41980a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800e9a0 tx=0x7f4fd800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe3fff700 1 -- 172.21.15.66:0/826250018 reap_dead start 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 shutdown_connections 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 >> 172.21.15.66:0/826250018 conn(0x7f4fe40fa000 msgr2=0x7f4fe40fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 shutdown_connections 2023-07-21T22:21:29.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:29.808+0000 7f4fe9dff700 1 -- 172.21.15.66:0/826250018 wait complete. 2023-07-21T22:21:29.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: Active manager daemon x restarted 2023-07-21T22:21:29.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: Activating manager daemon x 2023-07-21T22:21:29.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T22:21:29.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: mgrmap e22: x(active, starting, since 0.0125367s), standbys: z, y 2023-07-21T22:21:29.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T22:21:29.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T22:21:29.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T22:21:29.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:21:29.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:21:29.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:21:29.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:21:29.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:21:29.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:21:29.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T22:21:29.873 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T22:21:29.874 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T22:21:29.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T22:21:29.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: Manager daemon x is now available 2023-07-21T22:21:29.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:29.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:29.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T22:21:29.875 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:29 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T22:21:29.876 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-21T22:21:30.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 -- 172.21.15.66:0/4191919788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac072f70 msgr2=0x7f37ac0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 --2- 172.21.15.66:0/4191919788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac072f70 0x7f37ac0733e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f37a0009990 tx=0x7f37a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 -- 172.21.15.66:0/4191919788 shutdown_connections 2023-07-21T22:21:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 --2- 172.21.15.66:0/4191919788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37ac074550 0x7f37ac0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 --2- 172.21.15.66:0/4191919788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac10bf10 0x7f37ac10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 --2- 172.21.15.66:0/4191919788 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac072f70 0x7f37ac0733e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 -- 172.21.15.66:0/4191919788 >> 172.21.15.66:0/4191919788 conn(0x7f37ac0faf80 msgr2=0x7f37ac0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 -- 172.21.15.66:0/4191919788 shutdown_connections 2023-07-21T22:21:30.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.059+0000 7f37abfff700 1 -- 172.21.15.66:0/4191919788 wait complete. 2023-07-21T22:21:30.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.060+0000 7f37abfff700 1 Processor -- start 2023-07-21T22:21:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.060+0000 7f37abfff700 1 -- start start 2023-07-21T22:21:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 0x7f37ac19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37ac074550 0x7f37ac19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac10bf10 0x7f37ac1a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37abfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1a3850 con 0x7f37ac10bf10 2023-07-21T22:21:30.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37abfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1a39c0 con 0x7f37ac074550 2023-07-21T22:21:30.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37abfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1a3b80 con 0x7f37ac072f70 2023-07-21T22:21:30.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 0x7f37ac19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 0x7f37ac19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45416/0 (socket says 172.21.15.66:45416) 2023-07-21T22:21:30.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 -- 172.21.15.66:0/10853584 learned_addr learned my addr 172.21.15.66:0/10853584 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:30.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aa7fc700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37ac074550 0x7f37ac19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37ab7fe700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac10bf10 0x7f37ac1a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 -- 172.21.15.66:0/10853584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37ac074550 msgr2=0x7f37ac19c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37ac074550 0x7f37ac19c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 -- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac10bf10 msgr2=0x7f37ac1a3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac10bf10 0x7f37ac1a3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.061+0000 7f37aaffd700 1 -- 172.21.15.66:0/10853584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009c40 con 0x7f37ac072f70 2023-07-21T22:21:30.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.062+0000 7f37aaffd700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 0x7f37ac19bd10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e970 tx=0x7f379c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.062+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c009790 con 0x7f37ac072f70 2023-07-21T22:21:30.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.062+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a00095f0 con 0x7f37ac072f70 2023-07-21T22:21:30.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.062+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f379c00c7c0 con 0x7f37ac072f70 2023-07-21T22:21:30.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.062+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c005460 con 0x7f37ac072f70 2023-07-21T22:21:30.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.062+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac1a4180 con 0x7f37ac072f70 2023-07-21T22:21:30.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.063+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac101a60 con 0x7f37ac072f70 2023-07-21T22:21:30.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.065+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f379c005600 con 0x7f37ac072f70 2023-07-21T22:21:30.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.067+0000 7f37b09fd700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f37940a0f50 0x7f37940a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.067+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f379c014070 con 0x7f37ac072f70 2023-07-21T22:21:30.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.067+0000 7f37aa7fc700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f37940a0f50 0x7f37940a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.068+0000 7f37aa7fc700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f37940a0f50 0x7f37940a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b2e0 tx=0x7f37a0005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.071+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f379c0960f0 con 0x7f37ac072f70 2023-07-21T22:21:30.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.319+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 --> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f37ac0609f0 con 0x7f37940a0f50 2023-07-21T22:21:30.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.381+0000 7f37b09fd700 1 -- 172.21.15.66:0/10853584 <== mgr.15483 v2:172.21.15.66:6800/3475780841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f37ac0609f0 con 0x7f37940a0f50 2023-07-21T22:21:30.427 INFO:teuthology.orchestra.run.smithi066.stdout:Retention added to path / 2023-07-21T22:21:30.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.385+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f37940a0f50 msgr2=0x7f37940a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.385+0000 7f37abfff700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f37940a0f50 0x7f37940a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f37a000b2e0 tx=0x7f37a0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.385+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 msgr2=0x7f37ac19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.385+0000 7f37abfff700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 0x7f37ac19bd10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e970 tx=0x7f379c00ec80 comp rx=0 tx=0).stop 2023-07-21T22:21:30.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 shutdown_connections 2023-07-21T22:21:30.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f37940a0f50 0x7f37940a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f37ac10bf10 0x7f37ac1a3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37ac072f70 0x7f37ac19bd10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 --2- 172.21.15.66:0/10853584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37ac074550 0x7f37ac19c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 >> 172.21.15.66:0/10853584 conn(0x7f37ac0faf80 msgr2=0x7f37ac0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:30.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 shutdown_connections 2023-07-21T22:21:30.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.386+0000 7f37abfff700 1 -- 172.21.15.66:0/10853584 wait complete. 2023-07-21T22:21:30.450 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T22:21:30.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 -- 172.21.15.66:0/4040057975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 msgr2=0x7f83300729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 --2- 172.21.15.66:0/4040057975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f83300729a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f832c009990 tx=0x7f832c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 -- 172.21.15.66:0/4040057975 shutdown_connections 2023-07-21T22:21:30.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 --2- 172.21.15.66:0/4040057975 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f833010c0e0 0x7f833010e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 --2- 172.21.15.66:0/4040057975 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f83300729a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 --2- 172.21.15.66:0/4040057975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 0x7f83300733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 -- 172.21.15.66:0/4040057975 >> 172.21.15.66:0/4040057975 conn(0x7f83300fb130 msgr2=0x7f83300fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:30.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 -- 172.21.15.66:0/4040057975 shutdown_connections 2023-07-21T22:21:30.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.618+0000 7f83387ff700 1 -- 172.21.15.66:0/4040057975 wait complete. 2023-07-21T22:21:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.619+0000 7f83387ff700 1 Processor -- start 2023-07-21T22:21:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 -- start start 2023-07-21T22:21:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 0x7f833019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f833019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f833010c0e0 0x7f833019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833019d9b0 con 0x7f8330074550 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833019db20 con 0x7f8330072f70 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f83387ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833019dce0 con 0x7f833010c0e0 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f8335d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f833019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f833659b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 0x7f833019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f8335d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f833019c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51762/0 (socket says 172.21.15.66:51762) 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.620+0000 7f8335d9a700 1 -- 172.21.15.66:0/1330736627 learned_addr learned my addr 172.21.15.66:0/1330736627 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f8336d9c700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f833010c0e0 0x7f833019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f833659b700 1 -- 172.21.15.66:0/1330736627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f833010c0e0 msgr2=0x7f833019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f833659b700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f833010c0e0 0x7f833019d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f833659b700 1 -- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 msgr2=0x7f833019c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f833659b700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f833019c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f833659b700 1 -- 172.21.15.66:0/1330736627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0095f0 con 0x7f8330072f70 2023-07-21T22:21:30.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f833659b700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 0x7f833019bf90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b680 tx=0x7f832c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c01d070 con 0x7f8330072f70 2023-07-21T22:21:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.621+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833019df60 con 0x7f8330072f70 2023-07-21T22:21:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.622+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83301a4780 con 0x7f8330072f70 2023-07-21T22:21:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.622+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f832c0173f0 con 0x7f8330072f70 2023-07-21T22:21:30.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.622+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c017ea0 con 0x7f8330072f70 2023-07-21T22:21:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.623+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8314005320 con 0x7f8330072f70 2023-07-21T22:21:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.625+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 23) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f832c00fad0 con 0x7f8330072f70 2023-07-21T22:21:30.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.626+0000 7f83277fe700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f831c0a0fd0 0x7f831c0a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:30.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.626+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f832c0cdb60 con 0x7f8330072f70 2023-07-21T22:21:30.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.627+0000 7f8335d9a700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f831c0a0fd0 0x7f831c0a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:30.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.627+0000 7f8335d9a700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f831c0a0fd0 0x7f831c0a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8320005950 tx=0x7f8320009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:30.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.630+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f832c097180 con 0x7f8330072f70 2023-07-21T22:21:30.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.733+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f832c097550 con 0x7f8330072f70 2023-07-21T22:21:30.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.885+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 --> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f8314000bf0 con 0x7f831c0a0fd0 2023-07-21T22:21:30.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.894+0000 7f83277fe700 1 -- 172.21.15.66:0/1330736627 <== mgr.15483 v2:172.21.15.66:6800/3475780841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f8314000bf0 con 0x7f831c0a0fd0 2023-07-21T22:21:30.940 INFO:teuthology.orchestra.run.smithi066.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T22:21:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-21T22:21:30.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f831c0a0fd0 msgr2=0x7f831c0a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f831c0a0fd0 0x7f831c0a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8320005950 tx=0x7f8320009260 comp rx=0 tx=0).stop 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 msgr2=0x7f833019bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 0x7f833019bf90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b680 tx=0x7f832c005740 comp rx=0 tx=0).stop 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 shutdown_connections 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8330072f70 0x7f833019bf90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f831c0a0fd0 0x7f831c0a3480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f833010c0e0 0x7f833019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 --2- 172.21.15.66:0/1330736627 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8330074550 0x7f833019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:30.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 >> 172.21.15.66:0/1330736627 conn(0x7f83300fb130 msgr2=0x7f83300fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:30.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 shutdown_connections 2023-07-21T22:21:30.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:30.896+0000 7f83387ff700 1 -- 172.21.15.66:0/1330736627 wait complete. 2023-07-21T22:21:30.960 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T22:21:31.029 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:30 smithi066 ceph-mon[126379]: mgrmap e23: x(active, since 1.02263s), standbys: z, y 2023-07-21T22:21:31.029 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:30 smithi066 ceph-mon[126379]: from='client.25543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:31.029 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:30 smithi066 ceph-mon[126379]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:31.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:30 smithi104 ceph-mon[129021]: mgrmap e23: x(active, since 1.02263s), standbys: z, y 2023-07-21T22:21:31.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:30 smithi104 ceph-mon[129021]: from='client.25543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:31.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:30 smithi104 ceph-mon[129021]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:31.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 -- 172.21.15.66:0/932320055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd134100d00 msgr2=0x7fd134101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 --2- 172.21.15.66:0/932320055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd134100d00 0x7fd134101150 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd124009990 tx=0x7fd124009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 -- 172.21.15.66:0/932320055 shutdown_connections 2023-07-21T22:21:31.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 --2- 172.21.15.66:0/932320055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1340ffb10 0x7fd1340fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 --2- 172.21.15.66:0/932320055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd134101690 0x7fd134107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 --2- 172.21.15.66:0/932320055 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd134100d00 0x7fd134101150 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.132+0000 7fd13bdff700 1 -- 172.21.15.66:0/932320055 >> 172.21.15.66:0/932320055 conn(0x7fd1340fb130 msgr2=0x7fd1340fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.133+0000 7fd13bdff700 1 -- 172.21.15.66:0/932320055 shutdown_connections 2023-07-21T22:21:31.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.133+0000 7fd13bdff700 1 -- 172.21.15.66:0/932320055 wait complete. 2023-07-21T22:21:31.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 Processor -- start 2023-07-21T22:21:31.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 -- start start 2023-07-21T22:21:31.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 0x7fd134195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd134100d00 0x7fd134195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 0x7fd134196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134197360 con 0x7fd1340ffb10 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1341974d0 con 0x7fd134101690 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd13bdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134197690 con 0x7fd134100d00 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.134+0000 7fd139b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 0x7fd134195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd13a39c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 0x7fd134196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 0x7fd134195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51770/0 (socket says 172.21.15.66:51770) 2023-07-21T22:21:31.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 -- 172.21.15.66:0/4227732995 learned_addr learned my addr 172.21.15.66:0/4227732995 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd13a39c700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 0x7fd134196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47242/0 (socket says 172.21.15.66:47242) 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd13939a700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd134100d00 0x7fd134195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 -- 172.21.15.66:0/4227732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd134100d00 msgr2=0x7fd134195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd134100d00 0x7fd134195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 -- 172.21.15.66:0/4227732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 msgr2=0x7fd134196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 0x7fd134196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 -- 172.21.15.66:0/4227732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240095f0 con 0x7fd1340ffb10 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd13a39c700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 0x7fd134196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd139b9b700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 0x7fd134195940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd13000d980 tx=0x7fd13000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13000f890 con 0x7fd1340ffb10 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd130010460 con 0x7fd1340ffb10 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.135+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1300195c0 con 0x7fd1340ffb10 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.136+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd134197970 con 0x7fd1340ffb10 2023-07-21T22:21:31.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.136+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd134072920 con 0x7fd1340ffb10 2023-07-21T22:21:31.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.137+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd134107050 con 0x7fd1340ffb10 2023-07-21T22:21:31.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.139+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fd1300105d0 con 0x7fd1340ffb10 2023-07-21T22:21:31.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.140+0000 7fd12affd700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd1200a13d0 0x7fd1200a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:31.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.140+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd1300ccef0 con 0x7fd1340ffb10 2023-07-21T22:21:31.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.141+0000 7fd13939a700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd1200a13d0 0x7fd1200a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:31.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.141+0000 7fd13939a700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd1200a13d0 0x7fd1200a3880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b2e0 tx=0x7fd124005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:31.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.144+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd130095de0 con 0x7fd1340ffb10 2023-07-21T22:21:31.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:30 smithi149 ceph-mon[129268]: mgrmap e23: x(active, since 1.02263s), standbys: z, y 2023-07-21T22:21:31.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:30 smithi149 ceph-mon[129268]: from='client.25543 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:31.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:30 smithi149 ceph-mon[129268]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:31.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.396+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 --> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd134060950 con 0x7fd1200a13d0 2023-07-21T22:21:31.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.404+0000 7fd12affd700 1 -- 172.21.15.66:0/4227732995 <== mgr.15483 v2:172.21.15.66:6800/3475780841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd134060950 con 0x7fd1200a13d0 2023-07-21T22:21:31.449 INFO:teuthology.orchestra.run.smithi066.stdout:/ 1M 6M3h 2023-07-21T22:21:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd1200a13d0 msgr2=0x7fd1200a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd1200a13d0 0x7fd1200a3880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b2e0 tx=0x7fd124005fb0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 msgr2=0x7fd134195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 0x7fd134195940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd13000d980 tx=0x7fd13000dc90 comp rx=0 tx=0).stop 2023-07-21T22:21:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 shutdown_connections 2023-07-21T22:21:31.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd134101690 0x7fd134196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd1200a13d0 0x7fd1200a3880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd1340ffb10 0x7fd134195940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 --2- 172.21.15.66:0/4227732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd134100d00 0x7fd134195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.406+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 >> 172.21.15.66:0/4227732995 conn(0x7fd1340fb130 msgr2=0x7fd134104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.407+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 shutdown_connections 2023-07-21T22:21:31.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:31.407+0000 7fd13bdff700 1 -- 172.21.15.66:0/4227732995 wait complete. 2023-07-21T22:21:31.471 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-21T22:21:31.554 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-21T22:21:31.567 INFO:teuthology.run_tasks:Running task workunit... 2023-07-21T22:21:31.593 INFO:tasks.workunit:Pulling workunits from ref bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:21:31.595 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-07-21T22:21:31.595 INFO:tasks.workunit:timeout=3h 2023-07-21T22:21:31.595 INFO:tasks.workunit:cleanup=True 2023-07-21T22:21:31.596 DEBUG:teuthology.orchestra.run.smithi066:> stat -- /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:31.643 INFO:teuthology.orchestra.run.smithi066.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:31.709 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:cephfs_t:s0 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2023-07-21 22:19:35.831658754 +0000 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2023-07-21 22:19:35.831658754 +0000 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2023-07-21 22:21:08.580942371 +0000 2023-07-21T22:21:31.710 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2023-07-21 22:19:35.831658754 +0000 2023-07-21T22:21:31.711 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-07-21T22:21:31.711 DEBUG:teuthology.orchestra.run.smithi066:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-07-21T22:21:31.803 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea 2023-07-21T22:21:31.851 INFO:tasks.workunit.client.0.smithi066.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-07-21T22:21:32.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: [21/Jul/2023:22:21:30] ENGINE Bus STARTING 2023-07-21T22:21:32.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: from='client.25582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: [21/Jul/2023:22:21:30] ENGINE Serving on http://172.21.15.66:8765 2023-07-21T22:21:32.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: [21/Jul/2023:22:21:30] ENGINE Serving on https://172.21.15.66:7150 2023-07-21T22:21:32.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: [21/Jul/2023:22:21:30] ENGINE Bus STARTED 2023-07-21T22:21:32.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:32.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: mgrmap e24: x(active, since 2s), standbys: z, y 2023-07-21T22:21:32.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:21:32.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: from='client.25601 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:31 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:32.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: [21/Jul/2023:22:21:30] ENGINE Bus STARTING 2023-07-21T22:21:32.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: from='client.25582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: [21/Jul/2023:22:21:30] ENGINE Serving on http://172.21.15.66:8765 2023-07-21T22:21:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: [21/Jul/2023:22:21:30] ENGINE Serving on https://172.21.15.66:7150 2023-07-21T22:21:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: [21/Jul/2023:22:21:30] ENGINE Bus STARTED 2023-07-21T22:21:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: mgrmap e24: x(active, since 2s), standbys: z, y 2023-07-21T22:21:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:21:32.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: from='client.25601 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:31 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: [21/Jul/2023:22:21:30] ENGINE Bus STARTING 2023-07-21T22:21:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: from='client.25582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: [21/Jul/2023:22:21:30] ENGINE Serving on http://172.21.15.66:8765 2023-07-21T22:21:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: [21/Jul/2023:22:21:30] ENGINE Serving on https://172.21.15.66:7150 2023-07-21T22:21:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: [21/Jul/2023:22:21:30] ENGINE Bus STARTED 2023-07-21T22:21:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:32.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: mgrmap e24: x(active, since 2s), standbys: z, y 2023-07-21T22:21:32.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:21:32.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: from='client.25601 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:32.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:31 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:33.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:32 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:32 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='client.15558 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:21:33.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:33.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:32 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:34.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi104 to 196.2M 2023-07-21T22:21:34.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: Unable to set osd_memory_target on smithi104 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-21T22:21:34.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:34.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:34.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2023-07-21T22:21:34.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:33 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi104 to 196.2M 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: Unable to set osd_memory_target on smithi104 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2023-07-21T22:21:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:33 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi104 to 196.2M 2023-07-21T22:21:34.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: Unable to set osd_memory_target on smithi104 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:33 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi066 to 220.9M 2023-07-21T22:21:35.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Unable to set osd_memory_target on smithi066 to 231653273: error parsing value: Value '231653273' is below minimum 939524096 2023-07-21T22:21:35.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:35.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:35.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:21:35.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:21:35.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:21:35.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.623 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:35.623 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:35.623 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:35.624 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:35 smithi104 ceph-mon[129021]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi066 to 220.9M 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Unable to set osd_memory_target on smithi066 to 231653273: error parsing value: Value '231653273' is below minimum 939524096 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:21:35.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:35.698 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:35 smithi066 ceph-mon[126379]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:35.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:21:35.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi066 to 220.9M 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Unable to set osd_memory_target on smithi066 to 231653273: error parsing value: Value '231653273' is below minimum 939524096 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi066:/etc/ceph/ceph.conf 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi104:/etc/ceph/ceph.conf 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi149:/etc/ceph/ceph.conf 2023-07-21T22:21:35.741 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:35.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:21:35.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:35.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:35.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:35 smithi149 ceph-mon[129268]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T22:21:36.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:36 smithi066 ceph-mon[126379]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:36.739 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.740 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:36 smithi149 ceph-mon[129268]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:36.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: Updating smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:36.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: Updating smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:36.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:36.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:36 smithi104 ceph-mon[129021]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2023-07-21T22:21:37.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:37.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 50 op/s 2023-07-21T22:21:37.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:37.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:37.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:37.870 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:37.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 50 op/s 2023-07-21T22:21:37.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:37.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:37.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:37.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: Updating smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T22:21:37.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:37.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 132 KiB/s wr, 50 op/s 2023-07-21T22:21:37.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:21:37.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:21:37.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:21:37.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:21:40.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:39 smithi104 ceph-mon[129021]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-21T22:21:40.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:39 smithi066 ceph-mon[126379]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-21T22:21:40.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:39 smithi149 ceph-mon[129268]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-21T22:21:42.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:41 smithi104 ceph-mon[129021]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-21T22:21:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:41 smithi066 ceph-mon[126379]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-21T22:21:42.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:41 smithi149 ceph-mon[129268]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-07-21T22:21:44.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:43 smithi104 ceph-mon[129021]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-21T22:21:44.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:43 smithi066 ceph-mon[126379]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-21T22:21:44.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:43 smithi149 ceph-mon[129268]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-21T22:21:46.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:45 smithi104 ceph-mon[129021]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-21T22:21:46.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:45 smithi066 ceph-mon[126379]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-21T22:21:46.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:45 smithi149 ceph-mon[129268]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-07-21T22:21:48.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:47 smithi104 ceph-mon[129021]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-21T22:21:48.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:47 smithi066 ceph-mon[126379]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-21T22:21:48.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:47 smithi149 ceph-mon[129268]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 77 KiB/s wr, 34 op/s 2023-07-21T22:21:49.511 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:21:49.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 -- 172.21.15.66:0/325416213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4105ae0 msgr2=0x7ff1a4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/325416213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4105eb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff194009990 tx=0x7ff194009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 -- 172.21.15.66:0/325416213 shutdown_connections 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/325416213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a4100480 0x7ff1a410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/325416213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4105eb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/325416213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a40ffad0 0x7ff1a40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 -- 172.21.15.66:0/325416213 >> 172.21.15.66:0/325416213 conn(0x7ff1a40fb5f0 msgr2=0x7ff1a40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:49.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 -- 172.21.15.66:0/325416213 shutdown_connections 2023-07-21T22:21:49.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.689+0000 7ff1a9bff700 1 -- 172.21.15.66:0/325416213 wait complete. 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.690+0000 7ff1a9bff700 1 Processor -- start 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.690+0000 7ff1a9bff700 1 -- start start 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a9bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a40ffad0 0x7ff1a419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a9bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4100480 0x7ff1a419bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a9bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a9bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4196100 con 0x7ff1a4100480 2023-07-21T22:21:49.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a9bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4196270 con 0x7ff1a4105ae0 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a9bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4196430 con 0x7ff1a40ffad0 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4100480 0x7ff1a419bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.691+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a40ffad0 0x7ff1a419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a40ffad0 0x7ff1a419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52640/0 (socket says 172.21.15.66:52640) 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34380/0 (socket says 172.21.15.66:34380) 2023-07-21T22:21:49.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a37fe700 1 -- 172.21.15.66:0/2037318645 learned_addr learned my addr 172.21.15.66:0/2037318645 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:49.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2037318645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a40ffad0 msgr2=0x7ff1a419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a40ffad0 0x7ff1a419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4100480 msgr2=0x7ff1a419bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4100480 0x7ff1a419bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2037318645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940095f0 con 0x7ff1a4105ae0 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4100480 0x7ff1a419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4195a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff19c007f10 tx=0x7ff19c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a0ff9700 1 -- 172.21.15.66:0/2037318645 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c009ea0 con 0x7ff1a4105ae0 2023-07-21T22:21:49.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a0ff9700 1 -- 172.21.15.66:0/2037318645 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c00f040 con 0x7ff1a4105ae0 2023-07-21T22:21:49.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.692+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a4196710 con 0x7ff1a4105ae0 2023-07-21T22:21:49.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.693+0000 7ff1a0ff9700 1 -- 172.21.15.66:0/2037318645 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c004350 con 0x7ff1a4105ae0 2023-07-21T22:21:49.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.693+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a4072920 con 0x7ff1a4105ae0 2023-07-21T22:21:49.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.696+0000 7ff1a0ff9700 1 -- 172.21.15.66:0/2037318645 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7ff19c00db20 con 0x7ff1a4105ae0 2023-07-21T22:21:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.697+0000 7ff1bd5fcb80 1 Processor -- start 2023-07-21T22:21:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.697+0000 7ff1a0ff9700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff18c0a1350 0x7ff18c0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1a0ff9700 1 -- 172.21.15.66:0/2037318645 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff19c0cca30 con 0x7ff1a4105ae0 2023-07-21T22:21:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 -- start start 2023-07-21T22:21:49.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff18c0a1350 0x7ff18c0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fde6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fde1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fde1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fce6cb0 con 0x55f88fde6ac0 2023-07-21T22:21:49.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fd39a40 con 0x55f88fde1930 2023-07-21T22:21:49.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.698+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fd32ed0 con 0x55f88fde61b0 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff18c0a1350 0x7ff18c0a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff19800ffd0 tx=0x7ff19800f820 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fde6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fde1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fde1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fde1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34396/0 (socket says 172.21.15.66:34396) 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fde6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52650/0 (socket says 172.21.15.66:52650) 2023-07-21T22:21:49.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 -- 172.21.15.66:0/3151425171 learned_addr learned my addr 172.21.15.66:0/3151425171 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 -- 172.21.15.66:0/3151425171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 msgr2=0x55f88fde6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fde6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 -- 172.21.15.66:0/3151425171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 msgr2=0x55f88fde1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fde1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 -- 172.21.15.66:0/3151425171 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f88fb60de0 con 0x55f88fde1930 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a37fe700 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fde6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.699+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fde1da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff19c099570 tx=0x7ff19c0993b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac950bf528369209 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:49.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff189ffb700 1 -- 172.21.15.66:0/3151425171 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0b90d0 con 0x55f88fde1930 2023-07-21T22:21:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff189ffb700 1 -- 172.21.15.66:0/3151425171 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0995e0 con 0x55f88fde1930 2023-07-21T22:21:49.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff189ffb700 1 -- 172.21.15.66:0/3151425171 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0c9fc0 con 0x55f88fde1930 2023-07-21T22:21:49.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3151425171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 msgr2=0x55f88fde1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fde1da0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff19c099570 tx=0x7ff19c0993b0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3151425171 shutdown_connections 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fde1da0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fde6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3151425171 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fde1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3151425171 >> 172.21.15.66:0/3151425171 conn(0x55f88fdd6ef0 msgr2=0x55f88fddc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3151425171 shutdown_connections 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.700+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3151425171 wait complete. 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.701+0000 7ff1bd5fcb80 1 -- start start 2023-07-21T22:21:49.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.701+0000 7ff1bd5fcb80 1 Processor -- start 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdeb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fcd2ac0 con 0x55f88fde61b0 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fcd27c0 con 0x55f88fde6ac0 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fbb6920 con 0x55f88fde1930 2023-07-21T22:21:49.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1bd5fcb80 10 client.0 _ll_register_callbacks cb 0x55f88fcd0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdeb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdeb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49984/0 (socket says 172.21.15.66:49984) 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/2423474327 learned_addr learned my addr 172.21.15.66:0/2423474327 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/2423474327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 msgr2=0x55f88fdeae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 msgr2=0x55f88fdeb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.702+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/2423474327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a410d210 con 0x55f88fde61b0 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdeb3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff198005950 tx=0x7ff1980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9009ee0c37b152c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff182ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff182ffd700 1 -- 172.21.15.66:0/2423474327 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19802b070 con 0x55f88fde61b0 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f88fc8c4f0 con 0x55f88fde61b0 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff182ffd700 1 -- 172.21.15.66:0/2423474327 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff19801be30 con 0x55f88fde61b0 2023-07-21T22:21:49.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff182ffd700 1 -- 172.21.15.66:0/2423474327 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff198019d40 con 0x55f88fde61b0 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff1bd5fcb80 10 client.15570 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.703+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff1a4072920 con 0x55f88fde61b0 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff182ffd700 1 -- 172.21.15.66:0/2423474327 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f88fc8c4f0 con 0x55f88fde61b0 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff182ffd700 1 -- 172.21.15.66:0/2423474327 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff198026070 con 0x55f88fde61b0 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1bd5fcb80 10 client.15570 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1bd5fcb80 10 client.15570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1bd5fcb80 20 client.15570 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:49.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fdfb630 0x55f88fdfda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1bd5fcb80 4 client.15570 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f88fd2beb0 con 0x55f88fdfb630 2023-07-21T22:21:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.704+0000 7ff1a37fe700 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fdfb630 0x55f88fdfda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.705+0000 7ff1a37fe700 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fdfb630 0x55f88fdfda40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ec52e910bd5b4d7 server_cookie=e84b58c8b43a03de in_seq=0 out_seq=0 2023-07-21T22:21:49.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.705+0000 7ff182ffd700 10 client.15570 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff182ffd700 1 -- 172.21.15.66:0/2423474327 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f88fd2beb0 con 0x55f88fdfb630 2023-07-21T22:21:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff182ffd700 10 client.15570 handle_command_reply: tid=0 2023-07-21T22:21:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 client.15570 shutdown 2023-07-21T22:21:49.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 msgr2=0x55f88fdeb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdeb3c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff198005950 tx=0x7ff1980047c0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 shutdown_connections 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fdfb630 0x55f88fdfda40 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2423474327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdeb3c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff182ffd700 0 client.15570 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:49.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 >> 172.21.15.66:0/2423474327 conn(0x55f88fdd6ef0 msgr2=0x55f88fddc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:49.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 shutdown_connections 2023-07-21T22:21:49.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2423474327 wait complete. 2023-07-21T22:21:49.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.706+0000 7ff1bd5fcb80 20 client.15570 trim_cache size 0 max 16384 2023-07-21T22:21:49.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 Processor -- start 2023-07-21T22:21:49.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 -- start start 2023-07-21T22:21:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fdf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fddf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fddff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fde2300 con 0x55f88fde6ac0 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fdedf30 con 0x55f88fde1930 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fdd69b0 con 0x55f88fde61b0 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.724+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fddff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fddff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50000/0 (socket says 172.21.15.66:50000) 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fdf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2470882629 learned_addr learned my addr 172.21.15.66:0/2470882629 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fddf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2470882629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 msgr2=0x55f88fddf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fddf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2470882629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 msgr2=0x55f88fdf2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fdf2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 -- 172.21.15.66:0/2470882629 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f88fddb230 con 0x55f88fde6ac0 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a37fe700 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fdf2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff1a3fff700 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fddff00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff19c096050 tx=0x7ff19c020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3079c880557c08 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff18a7fc700 1 -- 172.21.15.66:0/2470882629 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c0b90d0 con 0x55f88fde6ac0 2023-07-21T22:21:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff18a7fc700 1 -- 172.21.15.66:0/2470882629 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0a1a30 con 0x55f88fde6ac0 2023-07-21T22:21:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.725+0000 7ff18a7fc700 1 -- 172.21.15.66:0/2470882629 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c084290 con 0x55f88fde6ac0 2023-07-21T22:21:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2470882629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 msgr2=0x55f88fddff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fddff00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff19c096050 tx=0x7ff19c020040 comp rx=0 tx=0).stop 2023-07-21T22:21:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2470882629 shutdown_connections 2023-07-21T22:21:49.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde1930 0x55f88fdf2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde6ac0 0x55f88fddff00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/2470882629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde61b0 0x55f88fddf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2470882629 >> 172.21.15.66:0/2470882629 conn(0x55f88fdd6ef0 msgr2=0x55f88fddc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2470882629 shutdown_connections 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/2470882629 wait complete. 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.726+0000 7ff1bd5fcb80 1 -- start start 2023-07-21T22:21:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 Processor -- start 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fdebe40 con 0x55f88fde61b0 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fdebfb0 con 0x55f88fde6ac0 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f88fdeb1d0 con 0x55f88fde1930 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1bd5fcb80 10 client.0 _ll_register_callbacks cb 0x7ff19801b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.727+0000 7ff1a37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdea720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50006/0 (socket says 172.21.15.66:50006) 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/3034783614 learned_addr learned my addr 172.21.15.66:0/3034783614 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/3034783614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 msgr2=0x55f88fdeb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 msgr2=0x55f88fdeac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 -- 172.21.15.66:0/3034783614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19400b850 con 0x55f88fde61b0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1a2ffd700 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdea720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55f88fdec530 tx=0x7ff19801b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167dbcb4d7e9840b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff182ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff182ffd700 1 -- 172.21.15.66:0/3034783614 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19803b070 con 0x55f88fde61b0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff19c0995e0 con 0x55f88fde61b0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.728+0000 7ff182ffd700 1 -- 172.21.15.66:0/3034783614 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff198009e70 con 0x55f88fde61b0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff182ffd700 1 -- 172.21.15.66:0/3034783614 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff19802b040 con 0x55f88fde61b0 2023-07-21T22:21:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 10 client.15576 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f88fb60de0 con 0x55f88fde61b0 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff182ffd700 1 -- 172.21.15.66:0/3034783614 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff19c0995e0 con 0x55f88fde61b0 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff182ffd700 1 -- 172.21.15.66:0/3034783614 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff19803e020 con 0x55f88fde61b0 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 10 client.15576 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 10 client.15576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 20 client.15576 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:49.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fe50680 0x55f88fe52a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:49.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 4 client.15576 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:21:49.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f88fcee050 con 0x55f88fe50680 2023-07-21T22:21:49.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.729+0000 7ff1a37fe700 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fe50680 0x55f88fe52a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1a37fe700 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fe50680 0x55f88fe52a90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c1b12607a764cb server_cookie=5ee9ba9d0a1033b in_seq=0 out_seq=0 2023-07-21T22:21:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff182ffd700 10 client.15576 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff182ffd700 1 -- 172.21.15.66:0/3034783614 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f88fcee050 con 0x55f88fe50680 2023-07-21T22:21:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff182ffd700 10 client.15576 handle_command_reply: tid=0 2023-07-21T22:21:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 client.15576 shutdown 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 msgr2=0x55f88fdea720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdea720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55f88fdec530 tx=0x7ff19801b410 comp rx=0 tx=0).stop 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 shutdown_connections 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f88fde6ac0 0x55f88fdeac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f88fe50680 0x55f88fe52a90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f88fde1930 0x55f88fdeb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 --2- 172.21.15.66:0/3034783614 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f88fde61b0 0x55f88fdea720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff182ffd700 0 client.15576 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 >> 172.21.15.66:0/3034783614 conn(0x55f88fdd6ef0 msgr2=0x55f88fddf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.730+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 shutdown_connections 2023-07-21T22:21:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1bd5fcb80 1 -- 172.21.15.66:0/3034783614 wait complete. 2023-07-21T22:21:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1bd5fcb80 20 client.15576 trim_cache size 0 max 16384 2023-07-21T22:21:49.784 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:49.785 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:21:49.785 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:21:49.785 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff18c0a1350 msgr2=0x7ff18c0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff18c0a1350 0x7ff18c0a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff19800ffd0 tx=0x7ff19800f820 comp rx=0 tx=0).stop 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 msgr2=0x7ff1a4195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4195a90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff19c007f10 tx=0x7ff19c00d3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 shutdown_connections 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1a4105ae0 0x7ff1a4195a90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff18c0a1350 0x7ff18c0a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1a40ffad0 0x7ff1a419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 --2- 172.21.15.66:0/2037318645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff1a4100480 0x7ff1a419bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 >> 172.21.15.66:0/2037318645 conn(0x7ff1a40fb5f0 msgr2=0x7ff1a40fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.731+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 shutdown_connections 2023-07-21T22:21:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.732+0000 7ff1a9bff700 1 -- 172.21.15.66:0/2037318645 wait complete. 2023-07-21T22:21:49.796 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:21:49.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:21:50.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 -- 172.21.15.66:0/3097822001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac10c310 msgr2=0x7f5bac10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/3097822001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac10c310 0x7f5bac10e6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009f90 tx=0x7f5ba400b330 comp rx=0 tx=0).stop 2023-07-21T22:21:50.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 -- 172.21.15.66:0/3097822001 shutdown_connections 2023-07-21T22:21:50.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/3097822001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac10c310 0x7f5bac10e6f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/3097822001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bac0728b0 0x7f5bac072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/3097822001 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0731c0 0x7f5bac10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 -- 172.21.15.66:0/3097822001 >> 172.21.15.66:0/3097822001 conn(0x7f5bac0fb590 msgr2=0x7f5bac0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 -- 172.21.15.66:0/3097822001 shutdown_connections 2023-07-21T22:21:50.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.961+0000 7f5bb35ff700 1 -- 172.21.15.66:0/3097822001 wait complete. 2023-07-21T22:21:50.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.962+0000 7f5bb35ff700 1 Processor -- start 2023-07-21T22:21:50.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.962+0000 7f5bb35ff700 1 -- start start 2023-07-21T22:21:50.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb35ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 0x7f5bac197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb35ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac0731c0 0x7f5bac198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb35ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bac10c310 0x7f5bac19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb35ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac19cee0 con 0x7f5bac0728b0 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb35ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac19d050 con 0x7f5bac0731c0 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb35ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac19d210 con 0x7f5bac10c310 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 0x7f5bac197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 0x7f5bac197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50030/0 (socket says 172.21.15.66:50030) 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb139b700 1 -- 172.21.15.66:0/2412966256 learned_addr learned my addr 172.21.15.66:0/2412966256 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb1b9c700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bac10c310 0x7f5bac19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.963+0000 7f5bb0b9a700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac0731c0 0x7f5bac198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb139b700 1 -- 172.21.15.66:0/2412966256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bac10c310 msgr2=0x7f5bac19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb139b700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bac10c310 0x7f5bac19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb139b700 1 -- 172.21.15.66:0/2412966256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac0731c0 msgr2=0x7f5bac198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb139b700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac0731c0 0x7f5bac198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb139b700 1 -- 172.21.15.66:0/2412966256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0095f0 con 0x7f5bac0728b0 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb139b700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 0x7f5bac197f00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800e9a0 tx=0x7f5ba800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8009790 con 0x7f5bac0728b0 2023-07-21T22:21:50.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ba8010430 con 0x7f5bac0728b0 2023-07-21T22:21:50.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba4009c40 con 0x7f5bac0728b0 2023-07-21T22:21:50.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8018680 con 0x7f5bac0728b0 2023-07-21T22:21:50.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.964+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac1a4b50 con 0x7f5bac0728b0 2023-07-21T22:21:50.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.965+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac106c30 con 0x7f5bac0728b0 2023-07-21T22:21:50.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.967+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f5ba80105a0 con 0x7f5bac0728b0 2023-07-21T22:21:50.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.969+0000 7f5ba27fc700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b980a1390 0x7f5b980a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.969+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ba8014070 con 0x7f5bac0728b0 2023-07-21T22:21:50.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.969+0000 7f5bb0b9a700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b980a1390 0x7f5b980a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.970+0000 7f5bb0b9a700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b980a1390 0x7f5b980a3840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bac199510 tx=0x7f5b9c009310 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:49.973+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5ba8051770 con 0x7f5bac0728b0 2023-07-21T22:21:50.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:49 smithi104 ceph-mon[129021]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-21T22:21:50.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:49 smithi066 ceph-mon[126379]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-21T22:21:50.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:49 smithi149 ceph-mon[129268]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2023-07-21T22:21:50.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.231+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bac067300 con 0x7f5bac0728b0 2023-07-21T22:21:50.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.232+0000 7f5ba27fc700 1 -- 172.21.15.66:0/2412966256 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f5ba8096310 con 0x7f5bac0728b0 2023-07-21T22:21:50.278 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:21:50.278 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:21:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b980a1390 msgr2=0x7f5b980a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b980a1390 0x7f5b980a3840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5bac199510 tx=0x7f5b9c009310 comp rx=0 tx=0).stop 2023-07-21T22:21:50.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 msgr2=0x7f5bac197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 0x7f5bac197f00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800e9a0 tx=0x7f5ba800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 shutdown_connections 2023-07-21T22:21:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b980a1390 0x7f5b980a3840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bac10c310 0x7f5bac19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5bac0728b0 0x7f5bac197f00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 --2- 172.21.15.66:0/2412966256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5bac0731c0 0x7f5bac198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.234+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 >> 172.21.15.66:0/2412966256 conn(0x7f5bac0fb590 msgr2=0x7f5bac103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.235+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 shutdown_connections 2023-07-21T22:21:50.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.235+0000 7f5bb35ff700 1 -- 172.21.15.66:0/2412966256 wait complete. 2023-07-21T22:21:50.283 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:21:50.299 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:21:50.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.467+0000 7f8da15ff700 1 -- 172.21.15.66:0/3417393265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c100480 msgr2=0x7f8d9c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.467+0000 7f8da15ff700 1 --2- 172.21.15.66:0/3417393265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c100480 0x7f8d9c10c910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400b0a0 tx=0x7f8d9400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 -- 172.21.15.66:0/3417393265 shutdown_connections 2023-07-21T22:21:50.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 --2- 172.21.15.66:0/3417393265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c100480 0x7f8d9c10c910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 --2- 172.21.15.66:0/3417393265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 --2- 172.21.15.66:0/3417393265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c0ffad0 0x7f8d9c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 -- 172.21.15.66:0/3417393265 >> 172.21.15.66:0/3417393265 conn(0x7f8d9c0fb5f0 msgr2=0x7f8d9c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 -- 172.21.15.66:0/3417393265 shutdown_connections 2023-07-21T22:21:50.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.468+0000 7f8da15ff700 1 -- 172.21.15.66:0/3417393265 wait complete. 2023-07-21T22:21:50.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.469+0000 7f8da15ff700 1 Processor -- start 2023-07-21T22:21:50.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.469+0000 7f8da15ff700 1 -- start start 2023-07-21T22:21:50.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8da15ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c0ffad0 0x7f8d9c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8da15ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c100480 0x7f8d9c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8da15ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8da15ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c199bb0 con 0x7f8d9c0ffad0 2023-07-21T22:21:50.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8da15ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c199d20 con 0x7f8d9c100480 2023-07-21T22:21:50.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8da15ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c199ee0 con 0x7f8d9c105ae0 2023-07-21T22:21:50.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c100480 0x7f8d9c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c0ffad0 0x7f8d9c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52728/0 (socket says 172.21.15.66:52728) 2023-07-21T22:21:50.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 -- 172.21.15.66:0/1671223985 learned_addr learned my addr 172.21.15.66:0/1671223985 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 -- 172.21.15.66:0/1671223985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c100480 msgr2=0x7f8d9c196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c100480 0x7f8d9c196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 -- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c0ffad0 msgr2=0x7f8d9c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c0ffad0 0x7f8d9c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9b7fe700 1 -- 172.21.15.66:0/1671223985 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90009520 con 0x7f8d9c105ae0 2023-07-21T22:21:50.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.470+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c100480 0x7f8d9c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:50.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.471+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c1995b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009fd0 tx=0x7f8d9400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.471+0000 7f8d83fff700 1 -- 172.21.15.66:0/1671223985 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400e040 con 0x7f8d9c105ae0 2023-07-21T22:21:50.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.471+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d94009d00 con 0x7f8d9c105ae0 2023-07-21T22:21:50.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.471+0000 7f8d83fff700 1 -- 172.21.15.66:0/1671223985 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d940094c0 con 0x7f8d9c105ae0 2023-07-21T22:21:50.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.471+0000 7f8d83fff700 1 -- 172.21.15.66:0/1671223985 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d940052a0 con 0x7f8d9c105ae0 2023-07-21T22:21:50.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.471+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c1a2940 con 0x7f8d9c105ae0 2023-07-21T22:21:50.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.474+0000 7f8d83fff700 1 -- 172.21.15.66:0/1671223985 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f8d9401d470 con 0x7f8d9c105ae0 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.476+0000 7f8d83fff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8d840a13d0 0x7f8d840a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.476+0000 7f8db4fffb80 1 Processor -- start 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.476+0000 7f8d83fff700 1 -- 172.21.15.66:0/1671223985 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8d940ce390 con 0x7f8d9c105ae0 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.476+0000 7f8db4fffb80 1 -- start start 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.476+0000 7f8d9affd700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8d840a13d0 0x7f8d840a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce5565c0 0x55adce556990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 0x55adce54d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce54dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce452cb0 con 0x55adce556ed0 2023-07-21T22:21:50.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce4a5a40 con 0x55adce54d930 2023-07-21T22:21:50.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce49eed0 con 0x55adce5565c0 2023-07-21T22:21:50.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 0x55adce54d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 0x55adce54d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50042/0 (socket says 172.21.15.66:50042) 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/3774260167 learned_addr learned my addr 172.21.15.66:0/3774260167 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8d9affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce5565c0 0x55adce556990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce54dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.477+0000 7f8d9affd700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8d840a13d0 0x7f8d840a3880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009c30 tx=0x7f8d8c009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/3774260167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce5565c0 msgr2=0x55adce556990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce5565c0 0x55adce556990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/3774260167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 msgr2=0x55adce54dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce54dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/3774260167 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adce2ccde0 con 0x55adce556ed0 2023-07-21T22:21:50.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 0x55adce54d350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011810 tx=0x7f8d90011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ca77b09bc696a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d817fa700 1 -- 172.21.15.66:0/3774260167 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90004830 con 0x55adce556ed0 2023-07-21T22:21:50.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d817fa700 1 -- 172.21.15.66:0/3774260167 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d90004e50 con 0x55adce556ed0 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.478+0000 7f8d817fa700 1 -- 172.21.15.66:0/3774260167 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9000f7a0 con 0x55adce556ed0 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3774260167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 msgr2=0x55adce54d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 0x55adce54d350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8d90011810 tx=0x7f8d90011b20 comp rx=0 tx=0).stop 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3774260167 shutdown_connections 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce54dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce5565c0 0x55adce556990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3774260167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce556ed0 0x55adce54d350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3774260167 >> 172.21.15.66:0/3774260167 conn(0x55adce542ef0 msgr2=0x55adce548830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3774260167 shutdown_connections 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.479+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3774260167 wait complete. 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.480+0000 7f8db4fffb80 1 -- start start 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.480+0000 7f8db4fffb80 1 Processor -- start 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce43eac0 con 0x55adce5565c0 2023-07-21T22:21:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce43e7c0 con 0x55adce556ed0 2023-07-21T22:21:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce322920 con 0x55adce54d930 2023-07-21T22:21:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8db4fffb80 10 client.0 _ll_register_callbacks cb 0x55adce43c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:50.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52744/0 (socket says 172.21.15.66:52744) 2023-07-21T22:21:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 -- 172.21.15.66:0/3126465380 learned_addr learned my addr 172.21.15.66:0/3126465380 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 -- 172.21.15.66:0/3126465380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 msgr2=0x55adce55b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 -- 172.21.15.66:0/3126465380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 msgr2=0x55adce55b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.481+0000 7f8d9affd700 1 -- 172.21.15.66:0/3126465380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d90011440 con 0x55adce54d930 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.482+0000 7f8d9affd700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55e010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c01c570 tx=0x7f8d8c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6efcf96c08c4174f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.482+0000 7f8d7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.482+0000 7f8d7eff5700 1 -- 172.21.15.66:0/3126465380 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c029070 con 0x55adce54d930 2023-07-21T22:21:50.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.482+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adce3f84f0 con 0x55adce54d930 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.482+0000 7f8d7eff5700 1 -- 172.21.15.66:0/3126465380 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8d8c004830 con 0x55adce54d930 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.482+0000 7f8d7eff5700 1 -- 172.21.15.66:0/3126465380 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d8c01ce80 con 0x55adce54d930 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 10 client.25630 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55adce2b5c60 con 0x55adce54d930 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8d7eff5700 1 -- 172.21.15.66:0/3126465380 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55adce3f84f0 con 0x55adce54d930 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8d7eff5700 1 -- 172.21.15.66:0/3126465380 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d8c02c020 con 0x55adce54d930 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 10 client.25630 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:50.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 10 client.25630 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:21:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 20 client.25630 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce56b8e0 0x55adce56dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 4 client.25630 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.483+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adce497eb0 con 0x55adce56b8e0 2023-07-21T22:21:50.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.484+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce56b8e0 0x55adce56dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.484+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce56b8e0 0x55adce56dcf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe2a6d592e4d2d server_cookie=be88f2e127e26a51 in_seq=0 out_seq=0 2023-07-21T22:21:50.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.484+0000 7f8d7eff5700 10 client.25630 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:50.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8d7eff5700 1 -- 172.21.15.66:0/3126465380 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55adce497eb0 con 0x55adce56b8e0 2023-07-21T22:21:50.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8d7eff5700 10 client.25630 handle_command_reply: tid=0 2023-07-21T22:21:50.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 client.25630 shutdown 2023-07-21T22:21:50.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 msgr2=0x55adce55e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55e010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c01c570 tx=0x7f8d8c0047c0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 shutdown_connections 2023-07-21T22:21:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce56b8e0 0x55adce56dcf0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55e010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/3126465380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 >> 172.21.15.66:0/3126465380 conn(0x55adce542ef0 msgr2=0x55adce548420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8d7eff5700 0 client.25630 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:50.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.485+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 shutdown_connections 2023-07-21T22:21:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.486+0000 7f8db4fffb80 1 -- 172.21.15.66:0/3126465380 wait complete. 2023-07-21T22:21:50.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.486+0000 7f8db4fffb80 20 client.25630 trim_cache size 0 max 16384 2023-07-21T22:21:50.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 Processor -- start 2023-07-21T22:21:50.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 -- start start 2023-07-21T22:21:50.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce545030 con 0x55adce5565c0 2023-07-21T22:21:50.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce5429b0 con 0x55adce556ed0 2023-07-21T22:21:50.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.500+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce5682b0 con 0x55adce54d930 2023-07-21T22:21:50.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50064/0 (socket says 172.21.15.66:50064) 2023-07-21T22:21:50.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/2435971104 learned_addr learned my addr 172.21.15.66:0/2435971104 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 -- 172.21.15.66:0/2435971104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 msgr2=0x55adce55aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 -- 172.21.15.66:0/2435971104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 msgr2=0x55adce55e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 -- 172.21.15.66:0/2435971104 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adce2f3250 con 0x55adce54d930 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.501+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55db40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0058a0 tx=0x7f8d8c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ef3f7342a0dd87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.502+0000 7f8d81ffb700 1 -- 172.21.15.66:0/2435971104 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c029070 con 0x55adce54d930 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.502+0000 7f8d81ffb700 1 -- 172.21.15.66:0/2435971104 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d8c01cc10 con 0x55adce54d930 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.502+0000 7f8d81ffb700 1 -- 172.21.15.66:0/2435971104 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0367b0 con 0x55adce54d930 2023-07-21T22:21:50.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2435971104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 msgr2=0x55adce55db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55db40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0058a0 tx=0x7f8d8c004750 comp rx=0 tx=0).stop 2023-07-21T22:21:50.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2435971104 shutdown_connections 2023-07-21T22:21:50.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce556ed0 0x55adce55aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce54d930 0x55adce55db40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2435971104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce55e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2435971104 >> 172.21.15.66:0/2435971104 conn(0x55adce542ef0 msgr2=0x55adce54baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2435971104 shutdown_connections 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2435971104 wait complete. 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 -- start start 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.503+0000 7f8db4fffb80 1 Processor -- start 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce562740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce5609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce556ed0 0x55adce560f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce561450 con 0x55adce5565c0 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce5615c0 con 0x55adce54d930 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adce561780 con 0x55adce556ed0 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8db4fffb80 10 client.0 _ll_register_callbacks cb 0x7f8d8c011f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce5609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8d9a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce5609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50070/0 (socket says 172.21.15.66:50070) 2023-07-21T22:21:50.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/2925042090 learned_addr learned my addr 172.21.15.66:0/2925042090 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce562740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.504+0000 7f8d9b7fe700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce556ed0 0x55adce560f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce556ed0 msgr2=0x55adce560f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce556ed0 0x55adce560f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/2925042090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 msgr2=0x55adce562740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce562740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d9a7fc700 1 -- 172.21.15.66:0/2925042090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adce2ccde0 con 0x55adce5565c0 2023-07-21T22:21:50.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d9a7fc700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce5609d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8d90000c00 tx=0x7f8d90005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbec5c84c5aea1fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d7eff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d7eff5700 1 -- 172.21.15.66:0/2925042090 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9002a040 con 0x55adce5565c0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d90004e50 con 0x55adce5565c0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.505+0000 7f8d7eff5700 1 -- 172.21.15.66:0/2925042090 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8d90005220 con 0x55adce5565c0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8d7eff5700 1 -- 172.21.15.66:0/2925042090 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d90003680 con 0x55adce5565c0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 10 client.15600 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55adce2b5c60 con 0x55adce5565c0 2023-07-21T22:21:50.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8d7eff5700 1 -- 172.21.15.66:0/2925042090 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d90004e50 con 0x55adce5565c0 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8d7eff5700 1 -- 172.21.15.66:0/2925042090 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8d90011840 con 0x55adce5565c0 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 10 client.15600 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 20 client.15600 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce5c09b0 0x55adce5c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 4 client.15600 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:21:50.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.506+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55adce45a050 con 0x55adce5c09b0 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.507+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce5c09b0 0x55adce5c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.507+0000 7f8d9affd700 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce5c09b0 0x55adce5c2dc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aca0e1aab9fe390c server_cookie=874eb5c863c87bcb in_seq=0 out_seq=0 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.507+0000 7f8d7eff5700 10 client.15600 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8d7eff5700 1 -- 172.21.15.66:0/2925042090 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55adce45a050 con 0x55adce5c09b0 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8d7eff5700 10 client.15600 handle_command_reply: tid=0 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 client.15600 shutdown 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 msgr2=0x55adce5609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce5609d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8d90000c00 tx=0x7f8d90005de0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 shutdown_connections 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55adce5c09b0 0x55adce5c2dc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55adce54d930 0x55adce562740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55adce556ed0 0x55adce560f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8d7eff5700 0 client.15600 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 --2- 172.21.15.66:0/2925042090 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55adce5565c0 0x55adce5609d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 >> 172.21.15.66:0/2925042090 conn(0x55adce542ef0 msgr2=0x55adce54c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 shutdown_connections 2023-07-21T22:21:50.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 1 -- 172.21.15.66:0/2925042090 wait complete. 2023-07-21T22:21:50.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.508+0000 7f8db4fffb80 20 client.15600 trim_cache size 0 max 16384 2023-07-21T22:21:50.567 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:21:50.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8d840a13d0 msgr2=0x7f8d840a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8d840a13d0 0x7f8d840a3880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009c30 tx=0x7f8d8c009260 comp rx=0 tx=0).stop 2023-07-21T22:21:50.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 msgr2=0x7f8d9c1995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c1995b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009fd0 tx=0x7f8d9400bce0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 shutdown_connections 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8d840a13d0 0x7f8d840a3880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d9c105ae0 0x7f8d9c1995b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8d9c0ffad0 0x7f8d9c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 --2- 172.21.15.66:0/1671223985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d9c100480 0x7f8d9c196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 >> 172.21.15.66:0/1671223985 conn(0x7f8d9c0fb5f0 msgr2=0x7f8d9c0fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 shutdown_connections 2023-07-21T22:21:50.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.509+0000 7f8da15ff700 1 -- 172.21.15.66:0/1671223985 wait complete. 2023-07-21T22:21:50.573 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:21:50.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 -- 172.21.15.66:0/3521460906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc073160 msgr2=0x7fb6fc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 --2- 172.21.15.66:0/3521460906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc073160 0x7fb6fc10be60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0098f0 tx=0x7fb6ec009c00 comp rx=0 tx=0).stop 2023-07-21T22:21:50.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 -- 172.21.15.66:0/3521460906 shutdown_connections 2023-07-21T22:21:50.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 --2- 172.21.15.66:0/3521460906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc072850 0x7fb6fc072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 --2- 172.21.15.66:0/3521460906 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 0x7fb6fc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 --2- 172.21.15.66:0/3521460906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc073160 0x7fb6fc10be60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 -- 172.21.15.66:0/3521460906 >> 172.21.15.66:0/3521460906 conn(0x7fb6fc0fb600 msgr2=0x7fb6fc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 -- 172.21.15.66:0/3521460906 shutdown_connections 2023-07-21T22:21:50.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.750+0000 7fb7041ff700 1 -- 172.21.15.66:0/3521460906 wait complete. 2023-07-21T22:21:50.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.752+0000 7fb7041ff700 1 Processor -- start 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.752+0000 7fb7041ff700 1 -- start start 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.752+0000 7fb7041ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc072850 0x7fb6fc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.752+0000 7fb7041ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc073160 0x7fb6fc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.752+0000 7fb7041ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 0x7fb6fc19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.752+0000 7fb7041ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc19ce90 con 0x7fb6fc10c3a0 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb7041ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc19d000 con 0x7fb6fc073160 2023-07-21T22:21:50.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb7041ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6fc19d1c0 con 0x7fb6fc072850 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70279c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 0x7fb6fc19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb701f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc072850 0x7fb6fc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70179a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc073160 0x7fb6fc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70279c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 0x7fb6fc19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50092/0 (socket says 172.21.15.66:50092) 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70279c700 1 -- 172.21.15.66:0/2547144029 learned_addr learned my addr 172.21.15.66:0/2547144029 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70179a700 1 -- 172.21.15.66:0/2547144029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc072850 msgr2=0x7fb6fc197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70179a700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc072850 0x7fb6fc197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70179a700 1 -- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 msgr2=0x7fb6fc19c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70179a700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 0x7fb6fc19c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.753+0000 7fb70179a700 1 -- 172.21.15.66:0/2547144029 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec009520 con 0x7fb6fc073160 2023-07-21T22:21:50.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.754+0000 7fb70179a700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc073160 0x7fb6fc1983f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009810 tx=0x7fb6ec00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.754+0000 7fb6f2ffd700 1 -- 172.21.15.66:0/2547144029 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec01c070 con 0x7fb6fc073160 2023-07-21T22:21:50.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.754+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6fc19d440 con 0x7fb6fc073160 2023-07-21T22:21:50.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.754+0000 7fb6f2ffd700 1 -- 172.21.15.66:0/2547144029 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6ec00bba0 con 0x7fb6fc073160 2023-07-21T22:21:50.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.754+0000 7fb6f2ffd700 1 -- 172.21.15.66:0/2547144029 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec003970 con 0x7fb6fc073160 2023-07-21T22:21:50.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.754+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6fc1a4ce0 con 0x7fb6fc073160 2023-07-21T22:21:50.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.758+0000 7fb6f2ffd700 1 -- 172.21.15.66:0/2547144029 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fb6ec020030 con 0x7fb6fc073160 2023-07-21T22:21:50.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.758+0000 7fb717b31b80 1 Processor -- start 2023-07-21T22:21:50.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 -- start start 2023-07-21T22:21:50.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x55992865c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 0x559928657350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559928657930 0x559928657da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992855ccb0 con 0x55992865c1b0 2023-07-21T22:21:50.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599285afa40 con 0x559928657930 2023-07-21T22:21:50.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.759+0000 7fb717b31b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599285a8ed0 con 0x55992865cac0 2023-07-21T22:21:50.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 0x559928657350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb70179a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559928657930 0x559928657da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb70279c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x55992865c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 0x559928657350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52788/0 (socket says 172.21.15.66:52788) 2023-07-21T22:21:50.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb70179a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559928657930 0x559928657da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34538/0 (socket says 172.21.15.66:34538) 2023-07-21T22:21:50.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 -- 172.21.15.66:0/2128748359 learned_addr learned my addr 172.21.15.66:0/2128748359 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 -- 172.21.15.66:0/2128748359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559928657930 msgr2=0x559928657da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559928657930 0x559928657da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb6f2ffd700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb6e80a13d0 0x7fb6e80a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 -- 172.21.15.66:0/2128748359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 msgr2=0x55992865c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x55992865c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 -- 172.21.15.66:0/2128748359 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599283d6de0 con 0x55992865cac0 2023-07-21T22:21:50.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb6f2ffd700 1 -- 172.21.15.66:0/2547144029 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb6ec0cdfa0 con 0x7fb6fc073160 2023-07-21T22:21:50.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb70279c700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb6e80a13d0 0x7fb6e80a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.760+0000 7fb701f9b700 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 0x559928657350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009d80 tx=0x7fb6f8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb98ef2ebcd0ec1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.761+0000 7fb6e7fff700 1 -- 172.21.15.66:0/2128748359 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8004230 con 0x55992865cac0 2023-07-21T22:21:50.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.761+0000 7fb6e7fff700 1 -- 172.21.15.66:0/2128748359 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f8004390 con 0x55992865cac0 2023-07-21T22:21:50.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.761+0000 7fb6e7fff700 1 -- 172.21.15.66:0/2128748359 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8019860 con 0x55992865cac0 2023-07-21T22:21:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 -- 172.21.15.66:0/2128748359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 msgr2=0x559928657350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 0x559928657350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009d80 tx=0x7fb6f8011720 comp rx=0 tx=0).stop 2023-07-21T22:21:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb70279c700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb6e80a13d0 0x7fb6e80a3880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40062a0 tx=0x7fb6f4004ad0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 -- 172.21.15.66:0/2128748359 shutdown_connections 2023-07-21T22:21:50.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559928657930 0x559928657da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x55992865c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2128748359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865cac0 0x559928657350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 -- 172.21.15.66:0/2128748359 >> 172.21.15.66:0/2128748359 conn(0x55992864cef0 msgr2=0x559928652830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 -- 172.21.15.66:0/2128748359 shutdown_connections 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.762+0000 7fb717b31b80 1 -- 172.21.15.66:0/2128748359 wait complete. 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 -- start start 2023-07-21T22:21:50.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 Processor -- start 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928663c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928660e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928661580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559928548ac0 con 0x559928657930 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599285487c0 con 0x55992865cac0 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992842c920 con 0x55992865c1b0 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.763+0000 7fb717b31b80 10 client.0 _ll_register_callbacks cb 0x559928546760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:50.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70179a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928660e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb701f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928663c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70179a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928660e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52804/0 (socket says 172.21.15.66:52804) 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70179a700 1 -- 172.21.15.66:0/827008854 learned_addr learned my addr 172.21.15.66:0/827008854 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70279c700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928661580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70279c700 1 -- 172.21.15.66:0/827008854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 msgr2=0x559928660e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70279c700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928660e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70279c700 1 -- 172.21.15.66:0/827008854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 msgr2=0x559928663c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70279c700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928663c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70279c700 1 -- 172.21.15.66:0/827008854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f80099e0 con 0x55992865cac0 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.764+0000 7fb70179a700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928660e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb70279c700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928661580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40060b0 tx=0x7fb6f400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe13bb918adb013 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb6e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb6e57fa700 1 -- 172.21.15.66:0/827008854 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4027070 con 0x55992865cac0 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599285024f0 con 0x55992865cac0 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb6e57fa700 1 -- 172.21.15.66:0/827008854 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb6f4017e10 con 0x55992865cac0 2023-07-21T22:21:50.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb6e57fa700 1 -- 172.21.15.66:0/827008854 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6f400e040 con 0x55992865cac0 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb717b31b80 10 client.25661 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.765+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5599283bfc60 con 0x55992865cac0 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb6e57fa700 1 -- 172.21.15.66:0/827008854 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599285024f0 con 0x55992865cac0 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb6e57fa700 1 -- 172.21.15.66:0/827008854 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6f402a020 con 0x55992865cac0 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb717b31b80 10 client.25661 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb717b31b80 10 client.25661 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:21:50.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb717b31b80 20 client.25661 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb717b31b80 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286714c0 0x5599286738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb717b31b80 4 client.25661 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599285a1eb0 con 0x5599286714c0 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.766+0000 7fb701f9b700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286714c0 0x5599286738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.767+0000 7fb6e57fa700 10 client.25661 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.767+0000 7fb701f9b700 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286714c0 0x5599286738d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23927d59be7db028 server_cookie=f4c7e08109acb582 in_seq=0 out_seq=0 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb6e57fa700 1 -- 172.21.15.66:0/827008854 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5599285a1eb0 con 0x5599286714c0 2023-07-21T22:21:50.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb6e57fa700 10 client.25661 handle_command_reply: tid=0 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 client.25661 shutdown 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 msgr2=0x559928661580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928661580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40060b0 tx=0x7fb6f400b040 comp rx=0 tx=0).stop 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 shutdown_connections 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286714c0 0x5599286738d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928661580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928663c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb6e57fa700 0 client.25661 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 --2- 172.21.15.66:0/827008854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928660e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 >> 172.21.15.66:0/827008854 conn(0x55992864cef0 msgr2=0x559928652420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 shutdown_connections 2023-07-21T22:21:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 1 -- 172.21.15.66:0/827008854 wait complete. 2023-07-21T22:21:50.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.768+0000 7fb717b31b80 20 client.25661 trim_cache size 0 max 16384 2023-07-21T22:21:50.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.782+0000 7fb717b31b80 1 Processor -- start 2023-07-21T22:21:50.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.782+0000 7fb717b31b80 1 -- start start 2023-07-21T22:21:50.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928668570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928655a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928655f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb717b31b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599286586d0 con 0x559928657930 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb717b31b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559928661b80 con 0x55992865cac0 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb717b31b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55992865be00 con 0x55992865c1b0 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928668570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928668570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50118/0 (socket says 172.21.15.66:50118) 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 -- 172.21.15.66:0/2715236597 learned_addr learned my addr 172.21.15.66:0/2715236597 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb70279c700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928655f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb70179a700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928655a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 -- 172.21.15.66:0/2715236597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 msgr2=0x559928655a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928655a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 -- 172.21.15.66:0/2715236597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 msgr2=0x559928655f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928655f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb701f9b700 1 -- 172.21.15.66:0/2715236597 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599283fd250 con 0x559928657930 2023-07-21T22:21:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb70179a700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928655a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:50.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.783+0000 7fb70279c700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928655f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.784+0000 7fb701f9b700 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928668570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8000be0 tx=0x7fb6f8003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a719f2c594bdf62f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.784+0000 7fb6f0ff9700 1 -- 172.21.15.66:0/2715236597 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8029070 con 0x559928657930 2023-07-21T22:21:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.784+0000 7fb6f0ff9700 1 -- 172.21.15.66:0/2715236597 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f8009ed0 con 0x559928657930 2023-07-21T22:21:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.784+0000 7fb6f0ff9700 1 -- 172.21.15.66:0/2715236597 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8003a00 con 0x559928657930 2023-07-21T22:21:50.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 -- 172.21.15.66:0/2715236597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 msgr2=0x559928668570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928668570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8000be0 tx=0x7fb6f8003e00 comp rx=0 tx=0).stop 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 -- 172.21.15.66:0/2715236597 shutdown_connections 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928655f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559928657930 0x559928668570 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 --2- 172.21.15.66:0/2715236597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55992865c1b0 0x559928655a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 -- 172.21.15.66:0/2715236597 >> 172.21.15.66:0/2715236597 conn(0x55992864cef0 msgr2=0x55992866cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 -- 172.21.15.66:0/2715236597 shutdown_connections 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.785+0000 7fb717b31b80 1 -- 172.21.15.66:0/2715236597 wait complete. 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.786+0000 7fb717b31b80 1 -- start start 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.786+0000 7fb717b31b80 1 Processor -- start 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.786+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559928657930 0x559928668580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.786+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x559928662730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.786+0000 7fb717b31b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928662c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb717b31b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559928663270 con 0x55992865c1b0 2023-07-21T22:21:50.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb717b31b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599286633e0 con 0x55992865cac0 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb717b31b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599286635a0 con 0x559928657930 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb717b31b80 10 client.0 _ll_register_callbacks cb 0x7fb6f40177e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x559928662730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x559928662730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50134/0 (socket says 172.21.15.66:50134) 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 -- 172.21.15.66:0/1065706985 learned_addr learned my addr 172.21.15.66:0/1065706985 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb701f9b700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559928657930 0x559928668580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70279c700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928662c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 -- 172.21.15.66:0/1065706985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559928657930 msgr2=0x559928668580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559928657930 0x559928668580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 -- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 msgr2=0x559928662c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928662c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70179a700 1 -- 172.21.15.66:0/1065706985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec09ee70 con 0x55992865c1b0 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.787+0000 7fb70279c700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928662c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb70179a700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x559928662730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00fab0 tx=0x7fb6ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebdf7f498cd739a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:50.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb6e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:50.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb6e57fa700 1 -- 172.21.15.66:0/1065706985 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec005470 con 0x55992865c1b0 2023-07-21T22:21:50.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb6e57fa700 1 -- 172.21.15.66:0/1065706985 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb6ec09abe0 con 0x55992865c1b0 2023-07-21T22:21:50.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6f8004390 con 0x55992865c1b0 2023-07-21T22:21:50.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb6e57fa700 1 -- 172.21.15.66:0/1065706985 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6ec09a540 con 0x55992865c1b0 2023-07-21T22:21:50.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb717b31b80 10 client.15618 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:50.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5599283bfc60 con 0x55992865c1b0 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb6e57fa700 1 -- 172.21.15.66:0/1065706985 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f8004390 con 0x55992865c1b0 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.788+0000 7fb6e57fa700 1 -- 172.21.15.66:0/1065706985 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb6ec09a160 con 0x55992865c1b0 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb717b31b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb717b31b80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb717b31b80 20 client.15618 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb717b31b80 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286c6650 0x5599286c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:50.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb717b31b80 4 client.15618 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559928564050 con 0x5599286c6650 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.789+0000 7fb701f9b700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286c6650 0x5599286c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb6e57fa700 10 client.15618 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb701f9b700 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286c6650 0x5599286c8a60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85f7ea00966a3a9b server_cookie=f16258b019d37c9c in_seq=0 out_seq=0 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb6e57fa700 1 -- 172.21.15.66:0/1065706985 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559928564050 con 0x5599286c6650 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb6e57fa700 10 client.15618 handle_command_reply: tid=0 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 client.15618 shutdown 2023-07-21T22:21:50.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 msgr2=0x559928662730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x559928662730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00fab0 tx=0x7fb6ec000bc0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 shutdown_connections 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5599286c6650 0x5599286c8a60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55992865cac0 0x559928662c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559928657930 0x559928668580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 --2- 172.21.15.66:0/1065706985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55992865c1b0 0x559928662730 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb6e57fa700 0 client.15618 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:21:50.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 >> 172.21.15.66:0/1065706985 conn(0x55992864cef0 msgr2=0x559928655a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 shutdown_connections 2023-07-21T22:21:50.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.790+0000 7fb717b31b80 1 -- 172.21.15.66:0/1065706985 wait complete. 2023-07-21T22:21:50.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb717b31b80 20 client.15618 trim_cache size 0 max 16384 2023-07-21T22:21:50.843 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:21:50.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb6e80a13d0 msgr2=0x7fb6e80a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb6e80a13d0 0x7fb6e80a3880 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb6f40062a0 tx=0x7fb6f4004ad0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc073160 msgr2=0x7fb6fc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:50.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc073160 0x7fb6fc1983f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009810 tx=0x7fb6ec00fa00 comp rx=0 tx=0).stop 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 shutdown_connections 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb6e80a13d0 0x7fb6e80a3880 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb6fc10c3a0 0x7fb6fc19c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6fc072850 0x7fb6fc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 --2- 172.21.15.66:0/2547144029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6fc073160 0x7fb6fc1983f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.791+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 >> 172.21.15.66:0/2547144029 conn(0x7fb6fc0fb600 msgr2=0x7fb6fc10e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.792+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 shutdown_connections 2023-07-21T22:21:50.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:50.792+0000 7fb7041ff700 1 -- 172.21.15.66:0/2547144029 wait complete. 2023-07-21T22:21:50.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:21:51.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 -- 172.21.15.66:0/3829568556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 msgr2=0x7f05c4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 --2- 172.21.15.66:0/3829568556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c4072c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009990 tx=0x7f05b4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 -- 172.21.15.66:0/3829568556 shutdown_connections 2023-07-21T22:21:51.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 --2- 172.21.15.66:0/3829568556 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c4110ae0 0x7f05c4112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 --2- 172.21.15.66:0/3829568556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c4072c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 --2- 172.21.15.66:0/3829568556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05c40731c0 0x7f05c41105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 -- 172.21.15.66:0/3829568556 >> 172.21.15.66:0/3829568556 conn(0x7f05c40fb5f0 msgr2=0x7f05c40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 -- 172.21.15.66:0/3829568556 shutdown_connections 2023-07-21T22:21:51.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.030+0000 7f05c8bff700 1 -- 172.21.15.66:0/3829568556 wait complete. 2023-07-21T22:21:51.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.031+0000 7f05c8bff700 1 Processor -- start 2023-07-21T22:21:51.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.031+0000 7f05c8bff700 1 -- start start 2023-07-21T22:21:51.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.031+0000 7f05c8bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c419c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c8bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05c40731c0 0x7f05c419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c8bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c4110ae0 0x7f05c41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c8bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41a11b0 con 0x7f05c4110ae0 2023-07-21T22:21:51.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c8bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41a1320 con 0x7f05c40731c0 2023-07-21T22:21:51.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c8bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41a14e0 con 0x7f05c40728b0 2023-07-21T22:21:51.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c419c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c4110ae0 0x7f05c41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c419c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52842/0 (socket says 172.21.15.66:52842) 2023-07-21T22:21:51.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 -- 172.21.15.66:0/2915278158 learned_addr learned my addr 172.21.15.66:0/2915278158 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:51.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05c40731c0 0x7f05c419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 -- 172.21.15.66:0/2915278158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05c40731c0 msgr2=0x7f05c419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05c40731c0 0x7f05c419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 -- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c4110ae0 msgr2=0x7f05c41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c4110ae0 0x7f05c41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.032+0000 7f05c259c700 1 -- 172.21.15.66:0/2915278158 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009520 con 0x7f05c40728b0 2023-07-21T22:21:51.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.033+0000 7f05c259c700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c419c1d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f05b4000c00 tx=0x7f05b4004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.033+0000 7f05b37fe700 1 -- 172.21.15.66:0/2915278158 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b401d070 con 0x7f05c40728b0 2023-07-21T22:21:51.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.033+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b40095f0 con 0x7f05c40728b0 2023-07-21T22:21:51.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.033+0000 7f05b37fe700 1 -- 172.21.15.66:0/2915278158 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b4021410 con 0x7f05c40728b0 2023-07-21T22:21:51.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.033+0000 7f05b37fe700 1 -- 172.21.15.66:0/2915278158 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4017650 con 0x7f05c40728b0 2023-07-21T22:21:51.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.033+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c41a8e20 con 0x7f05c40728b0 2023-07-21T22:21:51.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.037+0000 7f05b37fe700 1 -- 172.21.15.66:0/2915278158 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f05b4021580 con 0x7f05c40728b0 2023-07-21T22:21:51.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.038+0000 7f05b37fe700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05ac0a1350 0x7f05ac0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.038+0000 7f05b37fe700 1 -- 172.21.15.66:0/2915278158 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f05b40ce4f0 con 0x7f05c40728b0 2023-07-21T22:21:51.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.038+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05ac0a1350 0x7f05ac0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.039+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05ac0a1350 0x7f05ac0a3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05c419d7e0 tx=0x7f05b8009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.040+0000 7f05dc4a5b80 1 Processor -- start 2023-07-21T22:21:51.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.040+0000 7f05dc4a5b80 1 -- start start 2023-07-21T22:21:51.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629fac48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac48ac0 0x5629fac43350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac43da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fab48cb0 con 0x5629fac481b0 2023-07-21T22:21:51.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fab9ba40 con 0x5629fac48ac0 2023-07-21T22:21:51.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fab94ed0 con 0x5629fac43930 2023-07-21T22:21:51.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac43da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac43da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52850/0 (socket says 172.21.15.66:52850) 2023-07-21T22:21:51.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05c2d9d700 1 -- 172.21.15.66:0/2528036379 learned_addr learned my addr 172.21.15.66:0/2528036379 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:51.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac48ac0 0x5629fac43350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05c259c700 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629fac48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.041+0000 7f05c2d9d700 1 -- 172.21.15.66:0/2528036379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac48ac0 msgr2=0x5629fac43350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05c2d9d700 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac48ac0 0x5629fac43350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05c2d9d700 1 -- 172.21.15.66:0/2528036379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 msgr2=0x5629fac48580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05c2d9d700 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629fac48580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05c2d9d700 1 -- 172.21.15.66:0/2528036379 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629fa9c2de0 con 0x5629fac43930 2023-07-21T22:21:51.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05c2d9d700 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac43da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05bc011600 tx=0x7f05bc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffacc61a5154ab8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05b0ff9700 1 -- 172.21.15.66:0/2528036379 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc00e040 con 0x5629fac43930 2023-07-21T22:21:51.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05b0ff9700 1 -- 172.21.15.66:0/2528036379 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05bc00f040 con 0x5629fac43930 2023-07-21T22:21:51.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.042+0000 7f05b0ff9700 1 -- 172.21.15.66:0/2528036379 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004760 con 0x5629fac43930 2023-07-21T22:21:51.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2528036379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 msgr2=0x5629fac43da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac43da0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05bc011600 tx=0x7f05bc009920 comp rx=0 tx=0).stop 2023-07-21T22:21:51.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2528036379 shutdown_connections 2023-07-21T22:21:51.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac43da0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629fac48580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2528036379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac48ac0 0x5629fac43350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2528036379 >> 172.21.15.66:0/2528036379 conn(0x5629fac38ef0 msgr2=0x5629fac3e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2528036379 shutdown_connections 2023-07-21T22:21:51.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.043+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2528036379 wait complete. 2023-07-21T22:21:51.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.044+0000 7f05dc4a5b80 1 -- start start 2023-07-21T22:21:51.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 Processor -- start 2023-07-21T22:21:51.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 0x5629fac4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac48ac0 0x5629fac4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fab34ac0 con 0x5629fac43930 2023-07-21T22:21:51.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fab347c0 con 0x5629fac481b0 2023-07-21T22:21:51.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629faa18920 con 0x5629fac48ac0 2023-07-21T22:21:51.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.045+0000 7f05dc4a5b80 10 client.0 _ll_register_callbacks cb 0x5629fab32760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:51.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 0x5629fac4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34598/0 (socket says 172.21.15.66:34598) 2023-07-21T22:21:51.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 -- 172.21.15.66:0/2879733268 learned_addr learned my addr 172.21.15.66:0/2879733268 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:51.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c259c700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 0x5629fac4fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50176/0 (socket says 172.21.15.66:50176) 2023-07-21T22:21:51.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c2d9d700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac48ac0 0x5629fac4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 -- 172.21.15.66:0/2879733268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac48ac0 msgr2=0x5629fac4d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac48ac0 0x5629fac4d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 -- 172.21.15.66:0/2879733268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 msgr2=0x5629fac4fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 0x5629fac4fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c1d9b700 1 -- 172.21.15.66:0/2879733268 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0112b0 con 0x5629fac481b0 2023-07-21T22:21:51.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.046+0000 7f05c259c700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 0x5629fac4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:51.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4ced0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f05b800ed30 tx=0x7f05b80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b41f3cea0ea973 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:21:51.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05a9ffb700 1 -- 172.21.15.66:0/2879733268 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8022070 con 0x5629fac481b0 2023-07-21T22:21:51.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629faaee4f0 con 0x5629fac481b0 2023-07-21T22:21:51.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05a9ffb700 1 -- 172.21.15.66:0/2879733268 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f05b800ee20 con 0x5629fac481b0 2023-07-21T22:21:51.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05a9ffb700 1 -- 172.21.15.66:0/2879733268 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b801dd10 con 0x5629fac481b0 2023-07-21T22:21:51.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05dc4a5b80 10 client.25679 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:51.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.047+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5629fa9abc60 con 0x5629fac481b0 2023-07-21T22:21:51.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05a9ffb700 1 -- 172.21.15.66:0/2879733268 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629faaee4f0 con 0x5629fac481b0 2023-07-21T22:21:51.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05a9ffb700 1 -- 172.21.15.66:0/2879733268 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05b8003c50 con 0x5629fac481b0 2023-07-21T22:21:51.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05dc4a5b80 10 client.25679 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:51.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05dc4a5b80 10 client.25679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:51.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05dc4a5b80 20 client.25679 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:51.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac5d450 0x5629fac5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05dc4a5b80 4 client.25679 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:51.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629fab8deb0 con 0x5629fac5d450 2023-07-21T22:21:51.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.048+0000 7f05c259c700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac5d450 0x5629fac5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.049+0000 7f05c259c700 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac5d450 0x5629fac5f860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4a07ec65760629 server_cookie=a2d9a2f9f1517cda in_seq=0 out_seq=0 2023-07-21T22:21:51.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.049+0000 7f05a9ffb700 10 client.25679 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:51.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05a9ffb700 1 -- 172.21.15.66:0/2879733268 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629fab8deb0 con 0x5629fac5d450 2023-07-21T22:21:51.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05a9ffb700 10 client.25679 handle_command_reply: tid=0 2023-07-21T22:21:51.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 client.25679 shutdown 2023-07-21T22:21:51.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 msgr2=0x5629fac4ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4ced0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f05b800ed30 tx=0x7f05b80047c0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 shutdown_connections 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac5d450 0x5629fac5f860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac48ac0 0x5629fac4d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac43930 0x5629fac4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2879733268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4ced0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 >> 172.21.15.66:0/2879733268 conn(0x5629fac38ef0 msgr2=0x5629fac3e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05a9ffb700 0 client.25679 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 shutdown_connections 2023-07-21T22:21:51.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.050+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2879733268 wait complete. 2023-07-21T22:21:51.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.051+0000 7f05dc4a5b80 20 client.25679 trim_cache size 0 max 16384 2023-07-21T22:21:51.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:50 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2412966256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:51.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.077+0000 7f05dc4a5b80 1 Processor -- start 2023-07-21T22:21:51.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 -- start start 2023-07-21T22:21:51.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 0x5629fac5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac54580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fac4e440 con 0x5629fac38ef0 2023-07-21T22:21:51.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fac47e00 con 0x5629fac481b0 2023-07-21T22:21:51.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fac59430 con 0x5629fac43930 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05c259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 0x5629fac5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05c259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 0x5629fac5f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50184/0 (socket says 172.21.15.66:50184) 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05c259c700 1 -- 172.21.15.66:0/2725723762 learned_addr learned my addr 172.21.15.66:0/2725723762 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05c2d9d700 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.078+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac54580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05c259c700 1 -- 172.21.15.66:0/2725723762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 msgr2=0x5629fac54580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05c259c700 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac54580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05c259c700 1 -- 172.21.15.66:0/2725723762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 msgr2=0x5629fac4f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05c259c700 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05c259c700 1 -- 172.21.15.66:0/2725723762 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629fa9e9250 con 0x5629fac38ef0 2023-07-21T22:21:51.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05c259c700 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 0x5629fac5f830 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f05b4097630 tx=0x7f05b4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87afa94a1c6a241d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05b17fa700 1 -- 172.21.15.66:0/2725723762 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b409aa90 con 0x5629fac38ef0 2023-07-21T22:21:51.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05b17fa700 1 -- 172.21.15.66:0/2725723762 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05b4021860 con 0x5629fac38ef0 2023-07-21T22:21:51.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.079+0000 7f05b17fa700 1 -- 172.21.15.66:0/2725723762 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b409b5f0 con 0x5629fac38ef0 2023-07-21T22:21:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2725723762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 msgr2=0x5629fac5f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 0x5629fac5f830 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f05b4097630 tx=0x7f05b4009310 comp rx=0 tx=0).stop 2023-07-21T22:21:51.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2725723762 shutdown_connections 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac481b0 0x5629fac4f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac38ef0 0x5629fac5f830 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/2725723762 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac43930 0x5629fac54580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2725723762 >> 172.21.15.66:0/2725723762 conn(0x5629fac44060 msgr2=0x5629fac58d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2725723762 shutdown_connections 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.080+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/2725723762 wait complete. 2023-07-21T22:21:51.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.081+0000 7f05dc4a5b80 1 -- start start 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.081+0000 7f05dc4a5b80 1 Processor -- start 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 0x5629fac52750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac43930 0x5629fac52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629facb0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629fac53410 con 0x5629fac481b0 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629facb0880 con 0x5629fac43930 2023-07-21T22:21:51.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629facb0a40 con 0x5629fac38ef0 2023-07-21T22:21:51.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05dc4a5b80 10 client.0 _ll_register_callbacks cb 0x7f05b800e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:51.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 0x5629fac52750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05c2d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629facb0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05c259c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 0x5629fac52750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52870/0 (socket says 172.21.15.66:52870) 2023-07-21T22:21:51.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05c259c700 1 -- 172.21.15.66:0/4057578921 learned_addr learned my addr 172.21.15.66:0/4057578921 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:51.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.082+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac43930 0x5629fac52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05c259c700 1 -- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac43930 msgr2=0x5629fac52c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05c259c700 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac43930 0x5629fac52c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05c259c700 1 -- 172.21.15.66:0/4057578921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 msgr2=0x5629facb0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05c259c700 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629facb0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05c259c700 1 -- 172.21.15.66:0/4057578921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c41a8e20 con 0x5629fac38ef0 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05c259c700 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 0x5629fac52750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05b4005cd0 tx=0x7f05b409ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89744f5240383c3d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:21:51.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05a9ffb700 1 -- 172.21.15.66:0/4057578921 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b409a820 con 0x5629fac38ef0 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05a9ffb700 1 -- 172.21.15.66:0/4057578921 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f05b409f160 con 0x5629fac38ef0 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05bc00f040 con 0x5629fac38ef0 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05a9ffb700 1 -- 172.21.15.66:0/4057578921 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05b40a34f0 con 0x5629fac38ef0 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05dc4a5b80 10 client.25672 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.083+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5629fa9c12b0 con 0x5629fac38ef0 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05a9ffb700 1 -- 172.21.15.66:0/4057578921 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05bc00f040 con 0x5629fac38ef0 2023-07-21T22:21:51.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05a9ffb700 1 -- 172.21.15.66:0/4057578921 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f05b40b91b0 con 0x5629fac38ef0 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05dc4a5b80 10 client.25672 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05dc4a5b80 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05dc4a5b80 20 client.25672 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac48ac0 0x5629facb3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05dc4a5b80 4 client.25672 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5629fab50050 con 0x5629fac48ac0 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.084+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac48ac0 0x5629facb3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:51.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05a9ffb700 10 client.25672 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05c1d9b700 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac48ac0 0x5629facb3d00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90c637cd4808423 server_cookie=1240e42f7b86989b in_seq=0 out_seq=0 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05a9ffb700 1 -- 172.21.15.66:0/4057578921 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5629fab50050 con 0x5629fac48ac0 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05a9ffb700 10 client.25672 handle_command_reply: tid=0 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 client.25672 shutdown 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 msgr2=0x5629fac52750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 0x5629fac52750 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05b4005cd0 tx=0x7f05b409ab10 comp rx=0 tx=0).stop 2023-07-21T22:21:51.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 shutdown_connections 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5629fac48ac0 0x5629facb3d00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5629fac481b0 0x5629facb0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5629fac38ef0 0x5629fac52750 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05a9ffb700 0 client.25672 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 --2- 172.21.15.66:0/4057578921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5629fac43930 0x5629fac52c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 >> 172.21.15.66:0/4057578921 conn(0x5629fac44060 msgr2=0x5629fac3e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.085+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 shutdown_connections 2023-07-21T22:21:51.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05dc4a5b80 1 -- 172.21.15.66:0/4057578921 wait complete. 2023-07-21T22:21:51.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05dc4a5b80 20 client.25672 trim_cache size 0 max 16384 2023-07-21T22:21:51.144 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:21:51.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05ac0a1350 msgr2=0x7f05ac0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05ac0a1350 0x7f05ac0a3800 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f05c419d7e0 tx=0x7f05b8009260 comp rx=0 tx=0).stop 2023-07-21T22:21:51.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 msgr2=0x7f05c419c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c419c1d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f05b4000c00 tx=0x7f05b4004790 comp rx=0 tx=0).stop 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 shutdown_connections 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05ac0a1350 0x7f05ac0a3800 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f05c4110ae0 0x7f05c41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05c40728b0 0x7f05c419c1d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 --2- 172.21.15.66:0/2915278158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f05c40731c0 0x7f05c419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.086+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 >> 172.21.15.66:0/2915278158 conn(0x7f05c40fb5f0 msgr2=0x7f05c40fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:51.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.087+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 shutdown_connections 2023-07-21T22:21:51.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:51.087+0000 7f05c8bff700 1 -- 172.21.15.66:0/2915278158 wait complete. 2023-07-21T22:21:51.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:50 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2412966256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:51.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:50 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2412966256' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:21:52.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:51 smithi104 ceph-mon[129021]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-21T22:21:52.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:21:52.191 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:21:52.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:51 smithi066 ceph-mon[126379]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-21T22:21:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:51 smithi149 ceph-mon[129268]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 7 op/s 2023-07-21T22:21:52.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 -- 172.21.15.66:0/2636229513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9501004b0 msgr2=0x7fd95010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2636229513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9501004b0 0x7fd95010c940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd948009f90 tx=0x7fd94800b330 comp rx=0 tx=0).stop 2023-07-21T22:21:52.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 -- 172.21.15.66:0/2636229513 shutdown_connections 2023-07-21T22:21:52.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2636229513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9501004b0 0x7fd95010c940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2636229513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd950105b10 0x7fd950105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2636229513 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9500ffb00 0x7fd9500fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 -- 172.21.15.66:0/2636229513 >> 172.21.15.66:0/2636229513 conn(0x7fd9500fb600 msgr2=0x7fd9500fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 -- 172.21.15.66:0/2636229513 shutdown_connections 2023-07-21T22:21:52.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.362+0000 7fd9559ff700 1 -- 172.21.15.66:0/2636229513 wait complete. 2023-07-21T22:21:52.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.363+0000 7fd9559ff700 1 Processor -- start 2023-07-21T22:21:52.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 -- start start 2023-07-21T22:21:52.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9500ffb00 0x7fd950072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 0x7fd950072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd950105b10 0x7fd9501a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd950073580 con 0x7fd9501004b0 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501a43f0 con 0x7fd950105b10 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd9559ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501a4530 con 0x7fd9500ffb00 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd94e7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 0x7fd950072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd94e7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 0x7fd950072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41446/0 (socket says 172.21.15.66:41446) 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd94e7fc700 1 -- 172.21.15.66:0/2756519832 learned_addr learned my addr 172.21.15.66:0/2756519832 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd94effd700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9500ffb00 0x7fd950072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.364+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd950105b10 0x7fd9501a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94e7fc700 1 -- 172.21.15.66:0/2756519832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9500ffb00 msgr2=0x7fd950072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9500ffb00 0x7fd950072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94e7fc700 1 -- 172.21.15.66:0/2756519832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd950105b10 msgr2=0x7fd9501a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd950105b10 0x7fd9501a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94e7fc700 1 -- 172.21.15.66:0/2756519832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9440095f0 con 0x7fd9501004b0 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94effd700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9500ffb00 0x7fd950072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 0x7fd950072d50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd94400ec10 tx=0x7fd94400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd9549fd700 1 -- 172.21.15.66:0/2756519832 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94400cc90 con 0x7fd9501004b0 2023-07-21T22:21:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd940009520 con 0x7fd9501004b0 2023-07-21T22:21:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd9549fd700 1 -- 172.21.15.66:0/2756519832 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9440044e0 con 0x7fd9501004b0 2023-07-21T22:21:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.365+0000 7fd9549fd700 1 -- 172.21.15.66:0/2756519832 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944010430 con 0x7fd9501004b0 2023-07-21T22:21:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.366+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd948009c40 con 0x7fd9501004b0 2023-07-21T22:21:52.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.369+0000 7fd9549fd700 1 -- 172.21.15.66:0/2756519832 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7fd944003fb0 con 0x7fd9501004b0 2023-07-21T22:21:52.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.370+0000 7fd96938eb80 1 Processor -- start 2023-07-21T22:21:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.370+0000 7fd96938eb80 1 -- start start 2023-07-21T22:21:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.370+0000 7fd9549fd700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd9380a13d0 0x7fd9380a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aa5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5b40 0x55fb31aa03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd9549fd700 1 -- 172.21.15.66:0/2756519832 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd944014070 con 0x7fd9501004b0 2023-07-21T22:21:52.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aa0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd96938eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb319a5d30 con 0x55fb31aa5b40 2023-07-21T22:21:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd96938eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb319f8ac0 con 0x55fb31aa5230 2023-07-21T22:21:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd96938eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb319f1f50 con 0x55fb31aa09b0 2023-07-21T22:21:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd94f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aa0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd94effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aa5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd94e7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5b40 0x55fb31aa03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd94f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aa0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45914/0 (socket says 172.21.15.66:45914) 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd94f7fe700 1 -- 172.21.15.66:0/3446943403 learned_addr learned my addr 172.21.15.66:0/3446943403 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.371+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd9380a13d0 0x7fd9380a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94e7fc700 1 -- 172.21.15.66:0/3446943403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 msgr2=0x55fb31aa0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aa0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94e7fc700 1 -- 172.21.15.66:0/3446943403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 msgr2=0x55fb31aa5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aa5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94e7fc700 1 -- 172.21.15.66:0/3446943403 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb3181fe60 con 0x55fb31aa5b40 2023-07-21T22:21:52.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd9380a13d0 0x7fd9380a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9480060d0 tx=0x7fd9480096c0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5b40 0x55fb31aa03d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9440caa30 tx=0x7fd94409a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e95ce0280051e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd935ffb700 1 -- 172.21.15.66:0/3446943403 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94409e170 con 0x55fb31aa5b40 2023-07-21T22:21:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd935ffb700 1 -- 172.21.15.66:0/3446943403 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94409a300 con 0x55fb31aa5b40 2023-07-21T22:21:52.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.372+0000 7fd935ffb700 1 -- 172.21.15.66:0/3446943403 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9440cd560 con 0x55fb31aa5b40 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 -- 172.21.15.66:0/3446943403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5b40 msgr2=0x55fb31aa03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5b40 0x55fb31aa03d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9440caa30 tx=0x7fd94409a0d0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 -- 172.21.15.66:0/3446943403 shutdown_connections 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aa5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aa0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3446943403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5b40 0x55fb31aa03d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 -- 172.21.15.66:0/3446943403 >> 172.21.15.66:0/3446943403 conn(0x55fb31a95f70 msgr2=0x55fb31a9b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 -- 172.21.15.66:0/3446943403 shutdown_connections 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.373+0000 7fd96938eb80 1 -- 172.21.15.66:0/3446943403 wait complete. 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.374+0000 7fd96938eb80 1 -- start start 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.374+0000 7fd96938eb80 1 Processor -- start 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.374+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa09b0 0x55fb31aacca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.374+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31aa9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.374+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa5b40 0x55fb31aaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd96938eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31991b40 con 0x55fb31aa5230 2023-07-21T22:21:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd96938eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31991840 con 0x55fb31aa09b0 2023-07-21T22:21:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd96938eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb318759a0 con 0x55fb31aa5b40 2023-07-21T22:21:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd96938eb80 10 client.0 _ll_register_callbacks cb 0x55fb3198f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94e7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31aa9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94e7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31aa9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41468/0 (socket says 172.21.15.66:41468) 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94e7fc700 1 -- 172.21.15.66:0/740974849 learned_addr learned my addr 172.21.15.66:0/740974849 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa5b40 0x55fb31aaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94effd700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa09b0 0x55fb31aacca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94f7fe700 1 -- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa09b0 msgr2=0x55fb31aacca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa09b0 0x55fb31aacca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94f7fe700 1 -- 172.21.15.66:0/740974849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 msgr2=0x55fb31aa9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31aa9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94f7fe700 1 -- 172.21.15.66:0/740974849 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd948009c40 con 0x55fb31aa5b40 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.375+0000 7fd94effd700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa09b0 0x55fb31aacca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa5b40 0x55fb31aaa5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd948009c10 tx=0x7fd94800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f01360314fd6eb34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd92effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd92effd700 1 -- 172.21.15.66:0/740974849 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94800e040 con 0x55fb31aa5b40 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb3194b570 con 0x55fb31aa5b40 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd92effd700 1 -- 172.21.15.66:0/740974849 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd94800be00 con 0x55fb31aa5b40 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd92effd700 1 -- 172.21.15.66:0/740974849 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd948005110 con 0x55fb31aa5b40 2023-07-21T22:21:52.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd96938eb80 10 client.25684 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.376+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fb31808ce0 con 0x55fb31aa5b40 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd92effd700 1 -- 172.21.15.66:0/740974849 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb3194b570 con 0x55fb31aa5b40 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd92effd700 1 -- 172.21.15.66:0/740974849 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd948019030 con 0x55fb31aa5b40 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd96938eb80 10 client.25684 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd96938eb80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd96938eb80 20 client.25684 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:52.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd96938eb80 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aba4e0 0x55fb31abc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd96938eb80 4 client.25684 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:52.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb319eaf30 con 0x55fb31aba4e0 2023-07-21T22:21:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.377+0000 7fd94effd700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aba4e0 0x55fb31abc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.378+0000 7fd94effd700 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aba4e0 0x55fb31abc8f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c037034fb6b871 server_cookie=be6c893a959bac93 in_seq=0 out_seq=0 2023-07-21T22:21:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.378+0000 7fd92effd700 10 client.25684 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd92effd700 1 -- 172.21.15.66:0/740974849 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fb319eaf30 con 0x55fb31aba4e0 2023-07-21T22:21:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd92effd700 10 client.25684 handle_command_reply: tid=0 2023-07-21T22:21:52.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 client.25684 shutdown 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa5b40 msgr2=0x55fb31aaa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa5b40 0x55fb31aaa5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd948009c10 tx=0x7fd94800b330 comp rx=0 tx=0).stop 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 shutdown_connections 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa09b0 0x55fb31aacca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aba4e0 0x55fb31abc8f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa5b40 0x55fb31aaa5a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 --2- 172.21.15.66:0/740974849 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31aa9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd92effd700 0 client.25684 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 >> 172.21.15.66:0/740974849 conn(0x55fb31a95f70 msgr2=0x55fb31a9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 shutdown_connections 2023-07-21T22:21:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 1 -- 172.21.15.66:0/740974849 wait complete. 2023-07-21T22:21:52.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.379+0000 7fd96938eb80 20 client.25684 trim_cache size 0 max 16384 2023-07-21T22:21:52.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.397+0000 7fd96938eb80 1 Processor -- start 2023-07-21T22:21:52.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.397+0000 7fd96938eb80 1 -- start start 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 0x55fb31ab15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aacd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd96938eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31aaa950 con 0x55fb31a95f70 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd96938eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31a95a30 con 0x55fb31aa5230 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd96938eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31aa1750 con 0x55fb31aa09b0 2023-07-21T22:21:52.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 0x55fb31ab15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 0x55fb31ab15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41484/0 (socket says 172.21.15.66:41484) 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 -- 172.21.15.66:0/2540432924 learned_addr learned my addr 172.21.15.66:0/2540432924 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aacd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 -- 172.21.15.66:0/2540432924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 msgr2=0x55fb31aac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 -- 172.21.15.66:0/2540432924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 msgr2=0x55fb31aacd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aacd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.398+0000 7fd94effd700 1 -- 172.21.15.66:0/2540432924 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb318462d0 con 0x55fb31a95f70 2023-07-21T22:21:52.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.399+0000 7fd94effd700 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 0x55fb31ab15f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55fb31abd1d0 tx=0x7fd940009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62935e4e67dcd9e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.399+0000 7fd9367fc700 1 -- 172.21.15.66:0/2540432924 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940016030 con 0x55fb31a95f70 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.399+0000 7fd9367fc700 1 -- 172.21.15.66:0/2540432924 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd940017040 con 0x55fb31a95f70 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.399+0000 7fd9367fc700 1 -- 172.21.15.66:0/2540432924 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940004b00 con 0x55fb31a95f70 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 -- 172.21.15.66:0/2540432924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 msgr2=0x55fb31ab15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 0x55fb31ab15f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55fb31abd1d0 tx=0x7fd940009260 comp rx=0 tx=0).stop 2023-07-21T22:21:52.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 -- 172.21.15.66:0/2540432924 shutdown_connections 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31aa5230 0x55fb31aacd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31a95f70 0x55fb31ab15f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 --2- 172.21.15.66:0/2540432924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 -- 172.21.15.66:0/2540432924 >> 172.21.15.66:0/2540432924 conn(0x55fb31aa10e0 msgr2=0x55fb31a9eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 -- 172.21.15.66:0/2540432924 shutdown_connections 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 -- 172.21.15.66:0/2540432924 wait complete. 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 -- start start 2023-07-21T22:21:52.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.400+0000 7fd96938eb80 1 Processor -- start 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 0x55fb31aaf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aafb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31ab0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31b0f590 con 0x55fb31aa5230 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31b0f700 con 0x55fb31a95f70 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb31b0f8c0 con 0x55fb31aa09b0 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd96938eb80 10 client.0 _ll_register_callbacks cb 0x7fd948007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:52.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd94effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 0x55fb31aaf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd94effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 0x55fb31aaf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33382/0 (socket says 172.21.15.66:33382) 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd94effd700 1 -- 172.21.15.66:0/3741414336 learned_addr learned my addr 172.21.15.66:0/3741414336 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd94f7fe700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31ab0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.401+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aafb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd94effd700 1 -- 172.21.15.66:0/3741414336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 msgr2=0x55fb31aafb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd94effd700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aafb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd94effd700 1 -- 172.21.15.66:0/3741414336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 msgr2=0x55fb31ab0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd94effd700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31ab0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd94effd700 1 -- 172.21.15.66:0/3741414336 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940011c70 con 0x55fb31a95f70 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd94effd700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 0x55fb31aaf5e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9400205d0 tx=0x7fd940020070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aeabfea6810d68c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd92effd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd92effd700 1 -- 172.21.15.66:0/3741414336 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd940016040 con 0x55fb31a95f70 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd94409a300 con 0x55fb31a95f70 2023-07-21T22:21:52.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.402+0000 7fd92effd700 1 -- 172.21.15.66:0/3741414336 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd940004040 con 0x55fb31a95f70 2023-07-21T22:21:52.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd92effd700 1 -- 172.21.15.66:0/3741414336 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd940004220 con 0x55fb31a95f70 2023-07-21T22:21:52.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 10 client.25703 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:52.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55fb3181e330 con 0x55fb31a95f70 2023-07-21T22:21:52.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd92effd700 1 -- 172.21.15.66:0/3741414336 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94409a300 con 0x55fb31a95f70 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd92effd700 1 -- 172.21.15.66:0/3741414336 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd9400048b0 con 0x55fb31a95f70 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 10 client.25703 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 20 client.25703 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aa5b40 0x55fb31b12fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 4 client.25703 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fb319ad0d0 con 0x55fb31aa5b40 2023-07-21T22:21:52.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.403+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aa5b40 0x55fb31b12fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.404+0000 7fd92effd700 10 client.25703 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.404+0000 7fd94e7fc700 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aa5b40 0x55fb31b12fb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=932f0d2b9f597397 server_cookie=a22251a36db4b59d in_seq=0 out_seq=0 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.405+0000 7fd92effd700 1 -- 172.21.15.66:0/3741414336 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fb319ad0d0 con 0x55fb31aa5b40 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.405+0000 7fd92effd700 10 client.25703 handle_command_reply: tid=0 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.405+0000 7fd96938eb80 1 client.25703 shutdown 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 msgr2=0x55fb31aaf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 0x55fb31aaf5e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9400205d0 tx=0x7fd940020070 comp rx=0 tx=0).stop 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 shutdown_connections 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb31a95f70 0x55fb31aaf5e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fb31aa5b40 0x55fb31b12fb0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fb31aa5230 0x55fb31ab0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 --2- 172.21.15.66:0/3741414336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb31aa09b0 0x55fb31aafb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 >> 172.21.15.66:0/3741414336 conn(0x55fb31aa10e0 msgr2=0x55fb31a9fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd92effd700 0 client.25703 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 shutdown_connections 2023-07-21T22:21:52.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 1 -- 172.21.15.66:0/3741414336 wait complete. 2023-07-21T22:21:52.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.406+0000 7fd96938eb80 20 client.25703 trim_cache size 0 max 16384 2023-07-21T22:21:52.458 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:52.458 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:21:52.458 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "5ad5a683-d541-4e6d-bf80-0993b63d2522", 2023-07-21T22:21:52.459 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:21:52.459 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:52.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd9380a13d0 msgr2=0x7fd9380a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd9380a13d0 0x7fd9380a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd9480060d0 tx=0x7fd9480096c0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 msgr2=0x7fd950072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 0x7fd950072d50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd94400ec10 tx=0x7fd94400ef20 comp rx=0 tx=0).stop 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 shutdown_connections 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd950105b10 0x7fd9501a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd9380a13d0 0x7fd9380a3880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9500ffb00 0x7fd950072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 --2- 172.21.15.66:0/2756519832 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd9501004b0 0x7fd950072d50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 >> 172.21.15.66:0/2756519832 conn(0x7fd9500fb600 msgr2=0x7fd950106980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 shutdown_connections 2023-07-21T22:21:52.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.407+0000 7fd9559ff700 1 -- 172.21.15.66:0/2756519832 wait complete. 2023-07-21T22:21:52.473 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:21:52.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.640+0000 7ff97ed9d700 1 -- 172.21.15.66:0/397220483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 msgr2=0x7ff978105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.640+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/397220483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978105db0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff974009990 tx=0x7ff974009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 -- 172.21.15.66:0/397220483 shutdown_connections 2023-07-21T22:21:52.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/397220483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978105db0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/397220483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff978100380 0x7ff97810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/397220483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff9780ff9d0 0x7ff9780ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 -- 172.21.15.66:0/397220483 >> 172.21.15.66:0/397220483 conn(0x7ff9780fb4b0 msgr2=0x7ff9780fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 -- 172.21.15.66:0/397220483 shutdown_connections 2023-07-21T22:21:52.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.641+0000 7ff97ed9d700 1 -- 172.21.15.66:0/397220483 wait complete. 2023-07-21T22:21:52.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 Processor -- start 2023-07-21T22:21:52.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 -- start start 2023-07-21T22:21:52.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 0x7ff978072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff978100380 0x7ff978072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781a82d0 con 0x7ff978100380 2023-07-21T22:21:52.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781a8410 con 0x7ff9781059e0 2023-07-21T22:21:52.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.642+0000 7ff97ed9d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781a8550 con 0x7ff9780ff9d0 2023-07-21T22:21:52.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97d59a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff978100380 0x7ff978072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 0x7ff978072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 0x7ff978072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45960/0 (socket says 172.21.15.66:45960) 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97dd9b700 1 -- 172.21.15.66:0/4107483824 learned_addr learned my addr 172.21.15.66:0/4107483824 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 -- 172.21.15.66:0/4107483824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 msgr2=0x7ff978072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 0x7ff978072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 -- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff978100380 msgr2=0x7ff978072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff978100380 0x7ff978072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 -- 172.21.15.66:0/4107483824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740095f0 con 0x7ff9781059e0 2023-07-21T22:21:52.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 0x7ff978072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:52.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97d59a700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff978100380 0x7ff978072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:52.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.643+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978073290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff97000efc0 tx=0x7ff97000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.644+0000 7ff96effd700 1 -- 172.21.15.66:0/4107483824 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970018070 con 0x7ff9781059e0 2023-07-21T22:21:52.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.644+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9781a87d0 con 0x7ff9781059e0 2023-07-21T22:21:52.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.644+0000 7ff96effd700 1 -- 172.21.15.66:0/4107483824 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97000f040 con 0x7ff9781059e0 2023-07-21T22:21:52.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.644+0000 7ff96effd700 1 -- 172.21.15.66:0/4107483824 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000cd40 con 0x7ff9781059e0 2023-07-21T22:21:52.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.644+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9781a8ce0 con 0x7ff9781059e0 2023-07-21T22:21:52.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.648+0000 7ff96effd700 1 -- 172.21.15.66:0/4107483824 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) v1 ==== 135891+0+0 (secure 0 0 0) 0x7ff970003a00 con 0x7ff9781059e0 2023-07-21T22:21:52.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.649+0000 7ff96effd700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9640a13f0 0x7ff9640a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.649+0000 7ff96effd700 1 -- 172.21.15.66:0/4107483824 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff9700ccd60 con 0x7ff9781059e0 2023-07-21T22:21:52.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.649+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9640a13f0 0x7ff9640a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.650+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9640a13f0 0x7ff9640a38a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff97400b2e0 tx=0x7ff974005fd0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 Processor -- start 2023-07-21T22:21:52.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 -- start start 2023-07-21T22:21:52.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c647317860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317da0 0x55c647322700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647322c40 0x55c647325020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64721dcb0 con 0x55c647322c40 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647270a40 con 0x55c647317490 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.651+0000 7ff993870b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647269ed0 con 0x55c647317da0 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c647317860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97d59a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317da0 0x55c647322700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647322c40 0x55c647325020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c647317860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33414/0 (socket says 172.21.15.66:33414) 2023-07-21T22:21:52.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97dd9b700 1 -- 172.21.15.66:0/2245912782 learned_addr learned my addr 172.21.15.66:0/2245912782 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 -- 172.21.15.66:0/2245912782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317da0 msgr2=0x55c647322700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317da0 0x55c647322700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 -- 172.21.15.66:0/2245912782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 msgr2=0x55c647317860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c647317860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 -- 172.21.15.66:0/2245912782 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c647097de0 con 0x55c647322c40 2023-07-21T22:21:52.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.652+0000 7ff97e59c700 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647322c40 0x55c647325020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff9700ca040 tx=0x7ff9700956e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bff8edafdfa0cbc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff963fff700 1 -- 172.21.15.66:0/2245912782 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970099300 con 0x55c647322c40 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff963fff700 1 -- 172.21.15.66:0/2245912782 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff970004ad0 con 0x55c647322c40 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff963fff700 1 -- 172.21.15.66:0/2245912782 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9700ca550 con 0x55c647322c40 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 -- 172.21.15.66:0/2245912782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647322c40 msgr2=0x55c647325020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647322c40 0x55c647325020 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff9700ca040 tx=0x7ff9700956e0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 -- 172.21.15.66:0/2245912782 shutdown_connections 2023-07-21T22:21:52.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c647317860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647322c40 0x55c647325020 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 --2- 172.21.15.66:0/2245912782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317da0 0x55c647322700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.653+0000 7ff993870b80 1 -- 172.21.15.66:0/2245912782 >> 172.21.15.66:0/2245912782 conn(0x55c64730def0 msgr2=0x55c647313860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.654+0000 7ff993870b80 1 -- 172.21.15.66:0/2245912782 shutdown_connections 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.654+0000 7ff993870b80 1 -- 172.21.15.66:0/2245912782 wait complete. 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.654+0000 7ff993870b80 1 -- start start 2023-07-21T22:21:52.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.654+0000 7ff993870b80 1 Processor -- start 2023-07-21T22:21:52.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 0x55c647322650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317da0 0x55c64731f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647209ac0 con 0x55c647317490 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6472097c0 con 0x55c647317da0 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6470ed920 con 0x55c647322c40 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff993870b80 10 client.0 _ll_register_callbacks cb 0x55c647207760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 0x55c647322650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 0x55c647322650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41532/0 (socket says 172.21.15.66:41532) 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff97dd9b700 1 -- 172.21.15.66:0/3766348700 learned_addr learned my addr 172.21.15.66:0/3766348700 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff97e59c700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.655+0000 7ff97d59a700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317da0 0x55c64731f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff97dd9b700 1 -- 172.21.15.66:0/3766348700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 msgr2=0x55c64731ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff97dd9b700 1 -- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317da0 msgr2=0x55c64731f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317da0 0x55c64731f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff97dd9b700 1 -- 172.21.15.66:0/3766348700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9781a8ce0 con 0x55c647317490 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 0x55c647322650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff97400b6f0 tx=0x7ff974009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689ad15575e0f6c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff9617fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff9617fa700 1 -- 172.21.15.66:0/3766348700 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97400f670 con 0x55c647317490 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6471c34f0 con 0x55c647317490 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.656+0000 7ff9617fa700 1 -- 172.21.15.66:0/3766348700 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff97400f7d0 con 0x55c647317490 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff9617fa700 1 -- 172.21.15.66:0/3766348700 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97400f990 con 0x55c647317490 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 10 client.15666 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c647080c60 con 0x55c647317490 2023-07-21T22:21:52.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff9617fa700 1 -- 172.21.15.66:0/3766348700 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6471c34f0 con 0x55c647317490 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff9617fa700 1 -- 172.21.15.66:0/3766348700 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff97402f070 con 0x55c647317490 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 10 client.15666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 20 client.15666 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647332660 0x55c647334a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 4 client.15666 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.657+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c647262eb0 con 0x55c647332660 2023-07-21T22:21:52.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.658+0000 7ff97d59a700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647332660 0x55c647334a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.658+0000 7ff97d59a700 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647332660 0x55c647334a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83d58858e08105da server_cookie=f6d6cfc7466ef580 in_seq=0 out_seq=0 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.658+0000 7ff9617fa700 10 client.15666 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff9617fa700 1 -- 172.21.15.66:0/3766348700 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c647262eb0 con 0x55c647332660 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff9617fa700 10 client.15666 handle_command_reply: tid=0 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 client.15666 shutdown 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 msgr2=0x55c647322650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 0x55c647322650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff97400b6f0 tx=0x7ff974009ca0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 shutdown_connections 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647332660 0x55c647334a70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317490 0x55c647322650 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 --2- 172.21.15.66:0/3766348700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317da0 0x55c64731f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 >> 172.21.15.66:0/3766348700 conn(0x55c64730def0 msgr2=0x55c6473133f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.659+0000 7ff9617fa700 0 client.15666 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.660+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 shutdown_connections 2023-07-21T22:21:52.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.660+0000 7ff993870b80 1 -- 172.21.15.66:0/3766348700 wait complete. 2023-07-21T22:21:52.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.660+0000 7ff993870b80 20 client.15666 trim_cache size 0 max 16384 2023-07-21T22:21:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.688+0000 7ff993870b80 1 Processor -- start 2023-07-21T22:21:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 -- start start 2023-07-21T22:21:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c6473221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c64731aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64732f020 con 0x55c647317da0 2023-07-21T22:21:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647320eb0 con 0x55c647317490 2023-07-21T22:21:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff993870b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64731a7e0 con 0x55c647322c40 2023-07-21T22:21:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff97d59a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c64731aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff97d59a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c64731aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41542/0 (socket says 172.21.15.66:41542) 2023-07-21T22:21:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff97d59a700 1 -- 172.21.15.66:0/4021505412 learned_addr learned my addr 172.21.15.66:0/4021505412 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.689+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c6473221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97e59c700 1 -- 172.21.15.66:0/4021505412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 msgr2=0x55c6473221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c6473221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97e59c700 1 -- 172.21.15.66:0/4021505412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 msgr2=0x55c64731aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c64731aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97e59c700 1 -- 172.21.15.66:0/4021505412 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6470be250 con 0x55c647322c40 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97d59a700 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c64731aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff97e59c700 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731f090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55c647318210 tx=0x7ff970000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0570ba9c424c356 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff96cff9700 1 -- 172.21.15.66:0/4021505412 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9700999a0 con 0x55c647322c40 2023-07-21T22:21:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff96cff9700 1 -- 172.21.15.66:0/4021505412 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff970099300 con 0x55c647322c40 2023-07-21T22:21:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.690+0000 7ff96cff9700 1 -- 172.21.15.66:0/4021505412 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9700a1d60 con 0x55c647322c40 2023-07-21T22:21:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 -- 172.21.15.66:0/4021505412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 msgr2=0x55c64731f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731f090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55c647318210 tx=0x7ff970000f40 comp rx=0 tx=0).stop 2023-07-21T22:21:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 -- 172.21.15.66:0/4021505412 shutdown_connections 2023-07-21T22:21:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647317490 0x55c6473221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647322c40 0x55c64731f090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 --2- 172.21.15.66:0/4021505412 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c64731aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 -- 172.21.15.66:0/4021505412 >> 172.21.15.66:0/4021505412 conn(0x55c64730def0 msgr2=0x55c64732e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 -- 172.21.15.66:0/4021505412 shutdown_connections 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.691+0000 7ff993870b80 1 -- 172.21.15.66:0/4021505412 wait complete. 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 -- start start 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 Processor -- start 2023-07-21T22:21:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317490 0x55c647329370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c6473298b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647322c40 0x55c6473277f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647327ec0 con 0x55c647317da0 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.692+0000 7ff993870b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647328030 con 0x55c647322c40 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff993870b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6473281f0 con 0x55c647317490 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff993870b80 10 client.0 _ll_register_callbacks cb 0x55c647207760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:21:52.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c6473298b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317490 0x55c647329370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c6473298b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41550/0 (socket says 172.21.15.66:41550) 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 -- 172.21.15.66:0/2741292684 learned_addr learned my addr 172.21.15.66:0/2741292684 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97e59c700 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647322c40 0x55c6473277f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 -- 172.21.15.66:0/2741292684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317490 msgr2=0x55c647329370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317490 0x55c647329370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 -- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647322c40 msgr2=0x55c6473277f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:21:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647322c40 0x55c6473277f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 -- 172.21.15.66:0/2741292684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968010430 con 0x55c647317da0 2023-07-21T22:21:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff97d59a700 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c6473298b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55c64731a560 tx=0x7ff96801fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cd83bf97d3bdee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:21:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.693+0000 7ff9617fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:21:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff9617fa700 1 -- 172.21.15.66:0/2741292684 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968004830 con 0x55c647317da0 2023-07-21T22:21:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff970004ad0 con 0x55c647317da0 2023-07-21T22:21:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff9617fa700 1 -- 172.21.15.66:0/2741292684 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff968004990 con 0x55c647317da0 2023-07-21T22:21:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff9617fa700 1 -- 172.21.15.66:0/2741292684 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff968004b50 con 0x55c647317da0 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff993870b80 10 client.15672 fetch_fsmap learned FSMap version 23 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c647097de0 con 0x55c647317da0 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.694+0000 7ff9617fa700 1 -- 172.21.15.66:0/2741292684 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff970004ad0 con 0x55c647317da0 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff9617fa700 1 -- 172.21.15.66:0/2741292684 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff9680274b0 con 0x55c647317da0 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff993870b80 10 client.15672 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff993870b80 10 client.15672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:21:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff993870b80 20 client.15672 populate_metadata read hostname 'smithi066' 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff993870b80 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647387810 0x55c647389c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff993870b80 4 client.15672 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c647225050 con 0x55c647387810 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647387810 0x55c647389c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff9617fa700 10 client.15672 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.695+0000 7ff97dd9b700 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647387810 0x55c647389c20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e647b38b41dc17 server_cookie=f5c23108e93d95d3 in_seq=0 out_seq=0 2023-07-21T22:21:52.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff9617fa700 1 -- 172.21.15.66:0/2741292684 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c647225050 con 0x55c647387810 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff9617fa700 10 client.15672 handle_command_reply: tid=0 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 client.15672 shutdown 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 msgr2=0x55c6473298b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c6473298b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55c64731a560 tx=0x7ff96801fdd0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 shutdown_connections 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c647322c40 0x55c6473277f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c647387810 0x55c647389c20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c647317490 0x55c647329370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 --2- 172.21.15.66:0/2741292684 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c647317da0 0x55c6473298b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 >> 172.21.15.66:0/2741292684 conn(0x55c64730def0 msgr2=0x55c64731aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff9617fa700 0 client.15672 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:21:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 shutdown_connections 2023-07-21T22:21:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 1 -- 172.21.15.66:0/2741292684 wait complete. 2023-07-21T22:21:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.696+0000 7ff993870b80 20 client.15672 trim_cache size 0 max 16384 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout: "5ad5a683-d541-4e6d-bf80-0993b63d2522": { 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "5ad5a683-d541-4e6d-bf80-0993b63d2522", 2023-07-21T22:21:52.751 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:21:52.752 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:21:52.752 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:21:52.752 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:21:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9640a13f0 msgr2=0x7ff9640a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9640a13f0 0x7ff9640a38a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff97400b2e0 tx=0x7ff974005fd0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 msgr2=0x7ff978073290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:21:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978073290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff97000efc0 tx=0x7ff97000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 shutdown_connections 2023-07-21T22:21:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9781059e0 0x7ff978073290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9640a13f0 0x7ff9640a38a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff9780ff9d0 0x7ff978072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 --2- 172.21.15.66:0/4107483824 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff978100380 0x7ff978072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:21:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 >> 172.21.15.66:0/4107483824 conn(0x7ff9780fb4b0 msgr2=0x7ff9780fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:21:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 shutdown_connections 2023-07-21T22:21:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:21:52.697+0000 7ff97ed9d700 1 -- 172.21.15.66:0/4107483824 wait complete. 2023-07-21T22:21:52.762 INFO:tasks.cephfs.filesystem:scrub status for tag:5ad5a683-d541-4e6d-bf80-0993b63d2522 - {'path': '/', 'tag': '5ad5a683-d541-4e6d-bf80-0993b63d2522', 'options': 'recursive,force'} 2023-07-21T22:21:54.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:53 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:21:54.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:53 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:54.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:53 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:21:54.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:53 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:54.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:53 smithi104 ceph-mon[129021]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:21:54.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:53 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:21:54.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:53 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:54.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:53 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:21:54.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:53 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:54.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:53 smithi066 ceph-mon[126379]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:21:54.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:53 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:21:54.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:53 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:53 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:21:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:53 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:21:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:53 smithi149 ceph-mon[129268]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:21:56.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:55 smithi104 ceph-mon[129021]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:21:56.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:55 smithi066 ceph-mon[126379]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:21:56.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:55 smithi149 ceph-mon[129268]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:21:57.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:56 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:21:57.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:56 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:21:57.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:56 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:21:58.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:57 smithi104 ceph-mon[129021]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:21:58.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:57 smithi066 ceph-mon[126379]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:21:58.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:57 smithi149 ceph-mon[129268]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:00.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:21:59 smithi104 ceph-mon[129021]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:00.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:21:59 smithi066 ceph-mon[126379]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:21:59 smithi149 ceph-mon[129268]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:00.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:22:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:22:00.393+0000 7f75a995b700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:22:00.696 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:22:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:22:00.393+0000 7f75a995b700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:22:00.696 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:22:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:22:00.393+0000 7f75a995b700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:22:00.696 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:22:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:22:00.393+0000 7f75a995b700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:22:00.696 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:22:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:22:00.393+0000 7f75a995b700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T22:22:02.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:01 smithi104 ceph-mon[129021]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:02.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:01 smithi104 ceph-mon[129021]: mgrmap e25: x(active, since 32s), standbys: z, y 2023-07-21T22:22:02.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:01 smithi066 ceph-mon[126379]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:02.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:01 smithi066 ceph-mon[126379]: mgrmap e25: x(active, since 32s), standbys: z, y 2023-07-21T22:22:02.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:01 smithi149 ceph-mon[129268]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:02.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:01 smithi149 ceph-mon[129268]: mgrmap e25: x(active, since 32s), standbys: z, y 2023-07-21T22:22:04.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:03 smithi104 ceph-mon[129021]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:04.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:03 smithi066 ceph-mon[126379]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:04.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:03 smithi149 ceph-mon[129268]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:06.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:05 smithi104 ceph-mon[129021]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:06.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:05 smithi066 ceph-mon[126379]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:06.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:05 smithi149 ceph-mon[129268]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:08.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:07 smithi104 ceph-mon[129021]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:08.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:07 smithi066 ceph-mon[126379]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:08.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:07 smithi149 ceph-mon[129268]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:10.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:09 smithi104 ceph-mon[129021]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:10.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:09 smithi066 ceph-mon[126379]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:10.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:09 smithi149 ceph-mon[129268]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:12.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:11 smithi066 ceph-mon[126379]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:12.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:11 smithi149 ceph-mon[129268]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:12.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:11 smithi104 ceph-mon[129021]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:14.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:13 smithi066 ceph-mon[126379]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:14.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:13 smithi149 ceph-mon[129268]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:14.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:13 smithi104 ceph-mon[129021]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:16.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:15 smithi066 ceph-mon[126379]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:16.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:15 smithi149 ceph-mon[129268]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:16.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:15 smithi104 ceph-mon[129021]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 17 op/s 2023-07-21T22:22:18.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:17 smithi066 ceph-mon[126379]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:18.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:17 smithi149 ceph-mon[129268]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:18.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:17 smithi104 ceph-mon[129021]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2023-07-21T22:22:20.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:19 smithi066 ceph-mon[126379]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-21T22:22:20.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:19 smithi149 ceph-mon[129268]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-21T22:22:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:19 smithi104 ceph-mon[129021]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-07-21T22:22:22.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:21 smithi066 ceph-mon[126379]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-21T22:22:22.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:21 smithi149 ceph-mon[129268]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-21T22:22:22.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:21 smithi104 ceph-mon[129021]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-07-21T22:22:22.763 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:22:23.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.076+0000 7f23a59ff700 1 -- 172.21.15.66:0/1117379899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 msgr2=0x7f23a00fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.076+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1117379899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a00fee40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2394009990 tx=0x7f2394009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 -- 172.21.15.66:0/1117379899 shutdown_connections 2023-07-21T22:22:23.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1117379899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a00ff380 0x7f23a010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1117379899 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 0x7f23a0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1117379899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a00fee40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 -- 172.21.15.66:0/1117379899 >> 172.21.15.66:0/1117379899 conn(0x7f23a00fa4b0 msgr2=0x7f23a00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 -- 172.21.15.66:0/1117379899 shutdown_connections 2023-07-21T22:22:23.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.077+0000 7f23a59ff700 1 -- 172.21.15.66:0/1117379899 wait complete. 2023-07-21T22:22:23.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.078+0000 7f23a59ff700 1 Processor -- start 2023-07-21T22:22:23.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 -- start start 2023-07-21T22:22:23.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a010c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a00ff380 0x7f23a010c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 0x7f23a00708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a010ce50 con 0x7f23a01049e0 2023-07-21T22:22:23.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0070df0 con 0x7f23a00fe9d0 2023-07-21T22:22:23.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.079+0000 7f23a59ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0070fb0 con 0x7f23a00ff380 2023-07-21T22:22:23.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a010c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a010c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47954/0 (socket says 172.21.15.66:47954) 2023-07-21T22:22:23.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 -- 172.21.15.66:0/1293743561 learned_addr learned my addr 172.21.15.66:0/1293743561 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239e7fc700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a00ff380 0x7f23a010c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239f7fe700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 0x7f23a00708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 -- 172.21.15.66:0/1293743561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a00ff380 msgr2=0x7f23a010c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a00ff380 0x7f23a010c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 -- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 msgr2=0x7f23a00708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 0x7f23a00708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239effd700 1 -- 172.21.15.66:0/1293743561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23940095f0 con 0x7f23a00fe9d0 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.080+0000 7f239f7fe700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 0x7f23a00708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.081+0000 7f239effd700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a010c0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f239000d980 tx=0x7f239000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.081+0000 7f23a49fd700 1 -- 172.21.15.66:0/1293743561 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390004020 con 0x7f23a00fe9d0 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.081+0000 7f23a49fd700 1 -- 172.21.15.66:0/1293743561 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2390009730 con 0x7f23a00fe9d0 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.081+0000 7f23a49fd700 1 -- 172.21.15.66:0/1293743561 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390003a80 con 0x7f23a00fe9d0 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.081+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a0071290 con 0x7f23a00fe9d0 2023-07-21T22:22:23.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.081+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a0071750 con 0x7f23a00fe9d0 2023-07-21T22:22:23.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.087+0000 7f23a49fd700 1 -- 172.21.15.66:0/1293743561 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f23900098a0 con 0x7f23a00fe9d0 2023-07-21T22:22:23.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.088+0000 7f23b92cdb80 1 Processor -- start 2023-07-21T22:22:23.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.088+0000 7f23b92cdb80 1 -- start start 2023-07-21T22:22:23.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 0x55712e554580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e54f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e54fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e454cb0 con 0x55712e5541b0 2023-07-21T22:22:23.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e4a7a40 con 0x55712e554ac0 2023-07-21T22:22:23.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e4a0ed0 con 0x55712e54f930 2023-07-21T22:22:23.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f239effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e54f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f239e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e54fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 0x55712e554580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 0x55712e554580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55384/0 (socket says 172.21.15.66:55384) 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.089+0000 7f239f7fe700 1 -- 172.21.15.66:0/1260851251 learned_addr learned my addr 172.21.15.66:0/1260851251 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f239e7fc700 1 -- 172.21.15.66:0/1260851251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 msgr2=0x55712e54f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f239e7fc700 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e54f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f239e7fc700 1 -- 172.21.15.66:0/1260851251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 msgr2=0x55712e554580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f239e7fc700 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 0x55712e554580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f239e7fc700 1 -- 172.21.15.66:0/1260851251 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712e2cede0 con 0x55712e54f930 2023-07-21T22:22:23.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f239f7fe700 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 0x55712e554580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.091+0000 7f239effd700 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e54f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:23.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.091+0000 7f239e7fc700 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e54fda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2394000c00 tx=0x7f2394009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704607a5ca29e365 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.090+0000 7f23a49fd700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f23880a1460 0x7f23880a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.091+0000 7f2385ffb700 1 -- 172.21.15.66:0/1260851251 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239401d070 con 0x55712e54f930 2023-07-21T22:22:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.091+0000 7f23a49fd700 1 -- 172.21.15.66:0/1293743561 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f23900ccf90 con 0x7f23a00fe9d0 2023-07-21T22:22:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.091+0000 7f2385ffb700 1 -- 172.21.15.66:0/1260851251 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f239401e040 con 0x55712e54f930 2023-07-21T22:22:23.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.091+0000 7f2385ffb700 1 -- 172.21.15.66:0/1260851251 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2394004900 con 0x55712e54f930 2023-07-21T22:22:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f239effd700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f23880a1460 0x7f23880a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 -- 172.21.15.66:0/1260851251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 msgr2=0x55712e54fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e54fda0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2394000c00 tx=0x7f2394009570 comp rx=0 tx=0).stop 2023-07-21T22:22:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 -- 172.21.15.66:0/1260851251 shutdown_connections 2023-07-21T22:22:23.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e54fda0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e5541b0 0x55712e554580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/1260851251 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e54f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.092+0000 7f23b92cdb80 1 -- 172.21.15.66:0/1260851251 >> 172.21.15.66:0/1260851251 conn(0x55712e544ef0 msgr2=0x55712e54a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.093+0000 7f239effd700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f23880a1460 0x7f23880a3910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23900ca210 tx=0x7f23900d09e0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.093+0000 7f23b92cdb80 1 -- 172.21.15.66:0/1260851251 shutdown_connections 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.093+0000 7f23b92cdb80 1 -- 172.21.15.66:0/1260851251 wait complete. 2023-07-21T22:22:23.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.094+0000 7f23b92cdb80 1 -- start start 2023-07-21T22:22:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.094+0000 7f23b92cdb80 1 Processor -- start 2023-07-21T22:22:23.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e55bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e558e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e5595b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e440ac0 con 0x55712e554ac0 2023-07-21T22:22:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e4407c0 con 0x55712e5541b0 2023-07-21T22:22:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e324920 con 0x55712e54f930 2023-07-21T22:22:23.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f23b92cdb80 10 client.0 _ll_register_callbacks cb 0x55712e43e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e55bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e55bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49462/0 (socket says 172.21.15.66:49462) 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f239f7fe700 1 -- 172.21.15.66:0/516802826 learned_addr learned my addr 172.21.15.66:0/516802826 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.095+0000 7f239f7fe700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e5595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.096+0000 7f239e7fc700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e558e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.096+0000 7f239f7fe700 1 -- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 msgr2=0x55712e558e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.096+0000 7f239f7fe700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e558e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.096+0000 7f239f7fe700 1 -- 172.21.15.66:0/516802826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 msgr2=0x55712e5595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.096+0000 7f239f7fe700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e5595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.096+0000 7f239f7fe700 1 -- 172.21.15.66:0/516802826 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23980090b0 con 0x55712e54f930 2023-07-21T22:22:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.097+0000 7f239f7fe700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e55bc60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f239801b670 tx=0x7f2398009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81aefec988f91350 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.097+0000 7f237effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:23.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.097+0000 7f237effd700 1 -- 172.21.15.66:0/516802826 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239801bea0 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.097+0000 7f237effd700 1 -- 172.21.15.66:0/516802826 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f23980175f0 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.097+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55712e3fa4f0 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.098+0000 7f237effd700 1 -- 172.21.15.66:0/516802826 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23980177b0 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.098+0000 7f23b92cdb80 10 client.25726 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.098+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f23a0071750 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.098+0000 7f237effd700 1 -- 172.21.15.66:0/516802826 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55712e3fa4f0 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f237effd700 1 -- 172.21.15.66:0/516802826 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2398019850 con 0x55712e54f930 2023-07-21T22:22:23.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f23b92cdb80 10 client.25726 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:23.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f23b92cdb80 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:23.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f23b92cdb80 20 client.25726 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:23.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e569470 0x55712e56b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f23b92cdb80 4 client.25726 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:23.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55712e499eb0 con 0x55712e569470 2023-07-21T22:22:23.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.099+0000 7f239e7fc700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e569470 0x55712e56b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.100+0000 7f239e7fc700 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e569470 0x55712e56b880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3cd7facc1f520 server_cookie=9d92e95939e2ad8c in_seq=0 out_seq=0 2023-07-21T22:22:23.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.100+0000 7f237effd700 10 client.25726 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f237effd700 1 -- 172.21.15.66:0/516802826 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55712e499eb0 con 0x55712e569470 2023-07-21T22:22:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f237effd700 10 client.25726 handle_command_reply: tid=0 2023-07-21T22:22:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 client.25726 shutdown 2023-07-21T22:22:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 msgr2=0x55712e55bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e55bc60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f239801b670 tx=0x7f2398009920 comp rx=0 tx=0).stop 2023-07-21T22:22:23.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 shutdown_connections 2023-07-21T22:22:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e569470 0x55712e56b880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e5595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e55bc60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/516802826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e558e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f237effd700 0 client.25726 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:23.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.101+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 >> 172.21.15.66:0/516802826 conn(0x55712e544ef0 msgr2=0x55712e54a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.102+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 shutdown_connections 2023-07-21T22:22:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.102+0000 7f23b92cdb80 1 -- 172.21.15.66:0/516802826 wait complete. 2023-07-21T22:22:23.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.102+0000 7f23b92cdb80 20 client.25726 trim_cache size 0 max 16384 2023-07-21T22:22:23.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.136+0000 7f23b92cdb80 1 Processor -- start 2023-07-21T22:22:23.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.136+0000 7f23b92cdb80 1 -- start start 2023-07-21T22:22:23.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e560520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e54da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e54dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e566bc0 con 0x55712e554ac0 2023-07-21T22:22:23.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e5449b0 con 0x55712e5541b0 2023-07-21T22:22:23.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e547010 con 0x55712e54f930 2023-07-21T22:22:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e560520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.137+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e54dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e54dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55404/0 (socket says 172.21.15.66:55404) 2023-07-21T22:22:23.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 -- 172.21.15.66:0/355304014 learned_addr learned my addr 172.21.15.66:0/355304014 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 -- 172.21.15.66:0/355304014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 msgr2=0x55712e560520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e560520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 -- 172.21.15.66:0/355304014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 msgr2=0x55712e54da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239e7fc700 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e54da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e54da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239f7fe700 1 -- 172.21.15.66:0/355304014 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0067830 con 0x55712e554ac0 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.138+0000 7f239e7fc700 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e54da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.139+0000 7f239f7fe700 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e54dfa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2398009260 tx=0x7f2398000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d9f9096f74408d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.139+0000 7f23867fc700 1 -- 172.21.15.66:0/355304014 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239801bea0 con 0x55712e554ac0 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.139+0000 7f23867fc700 1 -- 172.21.15.66:0/355304014 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f239800f030 con 0x55712e554ac0 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.139+0000 7f23867fc700 1 -- 172.21.15.66:0/355304014 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239802ae20 con 0x55712e554ac0 2023-07-21T22:22:23.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 -- 172.21.15.66:0/355304014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 msgr2=0x55712e54dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e54dfa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2398009260 tx=0x7f2398000f40 comp rx=0 tx=0).stop 2023-07-21T22:22:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 -- 172.21.15.66:0/355304014 shutdown_connections 2023-07-21T22:22:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e554ac0 0x55712e54dfa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e54f930 0x55712e560520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/355304014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e5541b0 0x55712e54da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 -- 172.21.15.66:0/355304014 >> 172.21.15.66:0/355304014 conn(0x55712e544ef0 msgr2=0x55712e54fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 -- 172.21.15.66:0/355304014 shutdown_connections 2023-07-21T22:22:23.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.140+0000 7f23b92cdb80 1 -- 172.21.15.66:0/355304014 wait complete. 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.141+0000 7f23b92cdb80 1 -- start start 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.141+0000 7f23b92cdb80 1 Processor -- start 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e54f930 0x55712e560580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e5541b0 0x55712e55e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e55eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e55f5c0 con 0x55712e54f930 2023-07-21T22:22:23.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e55f730 con 0x55712e554ac0 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712e5bd340 con 0x55712e5541b0 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.142+0000 7f23b92cdb80 10 client.0 _ll_register_callbacks cb 0x7f2398007d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e54f930 0x55712e560580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e55eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e55eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47986/0 (socket says 172.21.15.66:47986) 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 -- 172.21.15.66:0/3000429991 learned_addr learned my addr 172.21.15.66:0/3000429991 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:23.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 -- 172.21.15.66:0/3000429991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e5541b0 msgr2=0x55712e55e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e5541b0 0x55712e55e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 -- 172.21.15.66:0/3000429991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e54f930 msgr2=0x55712e560580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:23.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e54f930 0x55712e560580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 -- 172.21.15.66:0/3000429991 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239801de70 con 0x55712e554ac0 2023-07-21T22:22:23.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f239f7fe700 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e55eeb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2398007440 tx=0x7f2398007510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af438497b6609b42 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.143+0000 7f237effd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:23.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.144+0000 7f237effd700 1 -- 172.21.15.66:0/3000429991 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239801bea0 con 0x55712e554ac0 2023-07-21T22:22:23.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.144+0000 7f237effd700 1 -- 172.21.15.66:0/3000429991 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2398019a00 con 0x55712e554ac0 2023-07-21T22:22:23.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.144+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f239401e040 con 0x55712e554ac0 2023-07-21T22:22:23.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.144+0000 7f237effd700 1 -- 172.21.15.66:0/3000429991 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2398019400 con 0x55712e554ac0 2023-07-21T22:22:23.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.144+0000 7f23b92cdb80 10 client.25742 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:23.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.144+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55712e2b7c60 con 0x55712e554ac0 2023-07-21T22:22:23.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f237effd700 1 -- 172.21.15.66:0/3000429991 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2398044080 con 0x55712e554ac0 2023-07-21T22:22:23.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f237effd700 1 -- 172.21.15.66:0/3000429991 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2398038040 con 0x55712e554ac0 2023-07-21T22:22:23.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f23b92cdb80 10 client.25742 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:23.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f23b92cdb80 10 client.25742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:23.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f23b92cdb80 20 client.25742 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:23.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e5be4c0 0x55712e5c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f23b92cdb80 4 client.25742 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:22:23.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55712e45c050 con 0x55712e5be4c0 2023-07-21T22:22:23.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.145+0000 7f239e7fc700 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e5be4c0 0x55712e5c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f239e7fc700 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e5be4c0 0x55712e5c08a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f1cc20db5ca00d server_cookie=6edd97aa70abd0c0 in_seq=0 out_seq=0 2023-07-21T22:22:23.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f237effd700 10 client.25742 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f237effd700 1 -- 172.21.15.66:0/3000429991 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55712e45c050 con 0x55712e5be4c0 2023-07-21T22:22:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f237effd700 10 client.25742 handle_command_reply: tid=0 2023-07-21T22:22:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f23b92cdb80 1 client.25742 shutdown 2023-07-21T22:22:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 msgr2=0x55712e55eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.146+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e55eeb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2398007440 tx=0x7f2398007510 comp rx=0 tx=0).stop 2023-07-21T22:22:23.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 shutdown_connections 2023-07-21T22:22:23.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55712e554ac0 0x55712e55eeb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55712e5be4c0 0x55712e5c08a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55712e54f930 0x55712e560580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 --2- 172.21.15.66:0/3000429991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55712e5541b0 0x55712e55e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 >> 172.21.15.66:0/3000429991 conn(0x55712e544ef0 msgr2=0x55712e54da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f237effd700 0 client.25742 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:23.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 shutdown_connections 2023-07-21T22:22:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 1 -- 172.21.15.66:0/3000429991 wait complete. 2023-07-21T22:22:23.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.147+0000 7f23b92cdb80 20 client.25742 trim_cache size 0 max 16384 2023-07-21T22:22:23.202 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:22:23.203 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:22:23.203 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:22:23.203 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:22:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f23880a1460 msgr2=0x7f23880a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f23880a1460 0x7f23880a3910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f23900ca210 tx=0x7f23900d09e0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 msgr2=0x7f23a010c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a010c0c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f239000d980 tx=0x7f239000dc90 comp rx=0 tx=0).stop 2023-07-21T22:22:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 shutdown_connections 2023-07-21T22:22:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a00fe9d0 0x7f23a010c0c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f23880a1460 0x7f23880a3910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f23a01049e0 0x7f23a00708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 --2- 172.21.15.66:0/1293743561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23a00ff380 0x7f23a010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 >> 172.21.15.66:0/1293743561 conn(0x7f23a00fa4b0 msgr2=0x7f23a00fb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 shutdown_connections 2023-07-21T22:22:23.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.148+0000 7f23a59ff700 1 -- 172.21.15.66:0/1293743561 wait complete. 2023-07-21T22:22:23.222 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:22:23.222 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:22:23.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.525+0000 7f742f7ff700 1 -- 172.21.15.66:0/3738949939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7428105ae0 msgr2=0x7f7428105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.525+0000 7f742f7ff700 1 --2- 172.21.15.66:0/3738949939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7428105ae0 0x7f7428105eb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7424009990 tx=0x7f7424009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 -- 172.21.15.66:0/3738949939 shutdown_connections 2023-07-21T22:22:23.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 --2- 172.21.15.66:0/3738949939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428100480 0x7f742810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 --2- 172.21.15.66:0/3738949939 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7428105ae0 0x7f7428105eb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 --2- 172.21.15.66:0/3738949939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74280ffad0 0x7f74280fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 -- 172.21.15.66:0/3738949939 >> 172.21.15.66:0/3738949939 conn(0x7f74280fb590 msgr2=0x7f74280fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:23.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 -- 172.21.15.66:0/3738949939 shutdown_connections 2023-07-21T22:22:23.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.526+0000 7f742f7ff700 1 -- 172.21.15.66:0/3738949939 wait complete. 2023-07-21T22:22:23.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.527+0000 7f742f7ff700 1 Processor -- start 2023-07-21T22:22:23.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 -- start start 2023-07-21T22:22:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74280ffad0 0x7f742819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 0x7f742819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428105ae0 0x7f7428195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74281960c0 con 0x7f74280ffad0 2023-07-21T22:22:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7428196230 con 0x7f7428100480 2023-07-21T22:22:23.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.528+0000 7f742f7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74281963f0 con 0x7f7428105ae0 2023-07-21T22:22:23.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742d59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74280ffad0 0x7f742819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 0x7f742819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 0x7f742819bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48010/0 (socket says 172.21.15.66:48010) 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742d59b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74280ffad0 0x7f742819b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55428/0 (socket says 172.21.15.66:55428) 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 -- 172.21.15.66:0/2806274822 learned_addr learned my addr 172.21.15.66:0/2806274822 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742dd9c700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428105ae0 0x7f7428195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 -- 172.21.15.66:0/2806274822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428105ae0 msgr2=0x7f7428195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428105ae0 0x7f7428195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 -- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74280ffad0 msgr2=0x7f742819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74280ffad0 0x7f742819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 -- 172.21.15.66:0/2806274822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009520 con 0x7f7428100480 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.529+0000 7f742cd9a700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 0x7f742819bf20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7418010e40 tx=0x7f741800b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.530+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418015070 con 0x7f7428100480 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.530+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74180044e0 con 0x7f7428100480 2023-07-21T22:22:23.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.530+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418019410 con 0x7f7428100480 2023-07-21T22:22:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.530+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74240095f0 con 0x7f7428100480 2023-07-21T22:22:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.530+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7428072740 con 0x7f7428100480 2023-07-21T22:22:23.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.532+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7428067300 con 0x7f7428100480 2023-07-21T22:22:23.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.536+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f7418003ed0 con 0x7f7428100480 2023-07-21T22:22:23.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.538+0000 7f741e7fc700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74140a14b0 0x7f74140a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:23.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.538+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f74180d0b60 con 0x7f7428100480 2023-07-21T22:22:23.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.539+0000 7f742d59b700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74140a14b0 0x7f74140a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:23.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.539+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f74180fc7f0 con 0x7f7428100480 2023-07-21T22:22:23.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:23.540+0000 7f742d59b700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74140a14b0 0x7f74140a3960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f742400b2e0 tx=0x7f7424005fd0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.050+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7428196f60 con 0x7f7428100480 2023-07-21T22:22:24.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.051+0000 7f741e7fc700 1 -- 172.21.15.66:0/2806274822 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f7418099ac0 con 0x7f7428100480 2023-07-21T22:22:24.096 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:22:24.096 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:19:34.299634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:22:24.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.054+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74140a14b0 msgr2=0x7f74140a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.054+0000 7f742f7ff700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74140a14b0 0x7f74140a3960 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f742400b2e0 tx=0x7f7424005fd0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.054+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 msgr2=0x7f742819bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.054+0000 7f742f7ff700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 0x7f742819bf20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7418010e40 tx=0x7f741800b5a0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 shutdown_connections 2023-07-21T22:22:24.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74140a14b0 0x7f74140a3960 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7428105ae0 0x7f7428195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74280ffad0 0x7f742819b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 --2- 172.21.15.66:0/2806274822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7428100480 0x7f742819bf20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 >> 172.21.15.66:0/2806274822 conn(0x7f74280fb590 msgr2=0x7f74280fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 shutdown_connections 2023-07-21T22:22:24.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.055+0000 7f742f7ff700 1 -- 172.21.15.66:0/2806274822 wait complete. 2023-07-21T22:22:24.101 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 23 2023-07-21T22:22:24.130 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:22:24.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:23 smithi066 ceph-mon[126379]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:24.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:23 smithi149 ceph-mon[129268]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:23 smithi104 ceph-mon[129021]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:24.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 -- 172.21.15.66:0/3731926102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe81015b0 msgr2=0x7febe8109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 --2- 172.21.15.66:0/3731926102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe81015b0 0x7febe8109aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7febdc009990 tx=0x7febdc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 -- 172.21.15.66:0/3731926102 shutdown_connections 2023-07-21T22:22:24.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 --2- 172.21.15.66:0/3731926102 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 0x7febe810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 --2- 172.21.15.66:0/3731926102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe8100ca0 0x7febe8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 --2- 172.21.15.66:0/3731926102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe81015b0 0x7febe8109aa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 -- 172.21.15.66:0/3731926102 >> 172.21.15.66:0/3731926102 conn(0x7febe80fa4b0 msgr2=0x7febe80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 -- 172.21.15.66:0/3731926102 shutdown_connections 2023-07-21T22:22:24.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.435+0000 7febecfff700 1 -- 172.21.15.66:0/3731926102 wait complete. 2023-07-21T22:22:24.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.437+0000 7febecfff700 1 Processor -- start 2023-07-21T22:22:24.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.437+0000 7febecfff700 1 -- start start 2023-07-21T22:22:24.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febecfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe8100ca0 0x7febe80fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febecfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe81015b0 0x7febe80fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febecfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 0x7febe80ff370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febecfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe81a85a0 con 0x7febe810a070 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febecfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe8199d60 con 0x7febe8100ca0 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febecfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe8199f20 con 0x7febe81015b0 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 0x7febe80ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febe659c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe8100ca0 0x7febe80fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 0x7febe80ff370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55440/0 (socket says 172.21.15.66:55440) 2023-07-21T22:22:24.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febe6d9d700 1 -- 172.21.15.66:0/378241674 learned_addr learned my addr 172.21.15.66:0/378241674 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.438+0000 7febe5d9b700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe81015b0 0x7febe80fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febe659c700 1 -- 172.21.15.66:0/378241674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe81015b0 msgr2=0x7febe80fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febe659c700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe81015b0 0x7febe80fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febe659c700 1 -- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 msgr2=0x7febe80ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febe659c700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 0x7febe80ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febe659c700 1 -- 172.21.15.66:0/378241674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc0095f0 con 0x7febe8100ca0 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febe659c700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe8100ca0 0x7febe80fe8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febd800e9f0 tx=0x7febd800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.439+0000 7febd77fe700 1 -- 172.21.15.66:0/378241674 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800c9c0 con 0x7febe8100ca0 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.440+0000 7febd77fe700 1 -- 172.21.15.66:0/378241674 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd8010430 con 0x7febe8100ca0 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.440+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe819a200 con 0x7febe8100ca0 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.440+0000 7febd77fe700 1 -- 172.21.15.66:0/378241674 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8018680 con 0x7febe8100ca0 2023-07-21T22:22:24.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.440+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe819a760 con 0x7febe8100ca0 2023-07-21T22:22:24.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.446+0000 7febd77fe700 1 -- 172.21.15.66:0/378241674 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7febd8010860 con 0x7febe8100ca0 2023-07-21T22:22:24.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.446+0000 7fec00961b80 1 Processor -- start 2023-07-21T22:22:24.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 -- start start 2023-07-21T22:22:24.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e9ac0 0x5573867e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867e4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573866e9cb0 con 0x5573867e4930 2023-07-21T22:22:24.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738673ca40 con 0x5573867e9ac0 2023-07-21T22:22:24.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.447+0000 7fec00961b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557386735ed0 con 0x5573867e91b0 2023-07-21T22:22:24.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49532/0 (socket says 172.21.15.66:49532) 2023-07-21T22:22:24.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 -- 172.21.15.66:0/3701087495 learned_addr learned my addr 172.21.15.66:0/3701087495 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 -- 172.21.15.66:0/3701087495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e9ac0 msgr2=0x5573867e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe659c700 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e9ac0 0x5573867e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e9ac0 0x5573867e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 -- 172.21.15.66:0/3701087495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 msgr2=0x5573867e4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867e4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe6d9d700 1 -- 172.21.15.66:0/3701087495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557386563de0 con 0x5573867e91b0 2023-07-21T22:22:24.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.448+0000 7febe659c700 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e9ac0 0x5573867e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:24.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.449+0000 7febe6d9d700 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e9580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7febe0009f10 tx=0x7febe00091b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fdf899d3a09d2c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.449+0000 7febd4ff9700 1 -- 172.21.15.66:0/3701087495 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0019070 con 0x5573867e91b0 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.449+0000 7febd77fe700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febd00a14e0 0x7febd00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.449+0000 7febd4ff9700 1 -- 172.21.15.66:0/3701087495 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe000f040 con 0x5573867e91b0 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.449+0000 7febd4ff9700 1 -- 172.21.15.66:0/3701087495 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00156e0 con 0x5573867e91b0 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.449+0000 7febd77fe700 1 -- 172.21.15.66:0/378241674 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7febd8014070 con 0x7febe8100ca0 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 -- 172.21.15.66:0/3701087495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 msgr2=0x5573867e9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e9580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7febe0009f10 tx=0x7febe00091b0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 -- 172.21.15.66:0/3701087495 shutdown_connections 2023-07-21T22:22:24.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867e4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e9580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 --2- 172.21.15.66:0/3701087495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e9ac0 0x5573867e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 -- 172.21.15.66:0/3701087495 >> 172.21.15.66:0/3701087495 conn(0x5573867d9ef0 msgr2=0x5573867df830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 -- 172.21.15.66:0/3701087495 shutdown_connections 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.450+0000 7fec00961b80 1 -- 172.21.15.66:0/3701087495 wait complete. 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.451+0000 7fec00961b80 1 -- start start 2023-07-21T22:22:24.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 Processor -- start 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 0x5573867ede80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e9ac0 0x5573867ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573866d5ac0 con 0x5573867e4930 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573866d57c0 con 0x5573867e91b0 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573865b9920 con 0x5573867e9ac0 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.452+0000 7fec00961b80 10 client.0 _ll_register_callbacks cb 0x5573866d3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:24.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 0x5573867ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 0x5573867ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48046/0 (socket says 172.21.15.66:48046) 2023-07-21T22:22:24.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 -- 172.21.15.66:0/4004342888 learned_addr learned my addr 172.21.15.66:0/4004342888 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe6d9d700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e9ac0 0x5573867ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe6d9d700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 -- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e9ac0 msgr2=0x5573867ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e9ac0 0x5573867ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 -- 172.21.15.66:0/4004342888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 msgr2=0x5573867f0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867f0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.453+0000 7febe5d9b700 1 -- 172.21.15.66:0/4004342888 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe0009b70 con 0x5573867e91b0 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.454+0000 7febe659c700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febd00a14e0 0x7febd00a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.454+0000 7febe5d9b700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 0x5573867ede80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7febdc009990 tx=0x7febdc00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=748504a03fd337a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.454+0000 7febcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:24.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.454+0000 7febcdffb700 1 -- 172.21.15.66:0/4004342888 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc01d070 con 0x5573867e91b0 2023-07-21T22:22:24.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.454+0000 7febcdffb700 1 -- 172.21.15.66:0/4004342888 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7febdc004500 con 0x5573867e91b0 2023-07-21T22:22:24.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.454+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55738668f4f0 con 0x5573867e91b0 2023-07-21T22:22:24.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.455+0000 7febe659c700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febd00a14e0 0x7febd00a3990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7febd80ca6a0 tx=0x7febd8010c60 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.455+0000 7febcdffb700 1 -- 172.21.15.66:0/4004342888 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febdc0046c0 con 0x5573867e91b0 2023-07-21T22:22:24.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.455+0000 7fec00961b80 10 client.25757 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:24.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.455+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55738654cc60 con 0x5573867e91b0 2023-07-21T22:22:24.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.455+0000 7febcdffb700 1 -- 172.21.15.66:0/4004342888 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55738668f4f0 con 0x5573867e91b0 2023-07-21T22:22:24.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7febcdffb700 1 -- 172.21.15.66:0/4004342888 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7febdc00f860 con 0x5573867e91b0 2023-07-21T22:22:24.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7fec00961b80 10 client.25757 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7fec00961b80 10 client.25757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:22:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7fec00961b80 20 client.25757 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7fec00961b80 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5573867fe410 0x557386800820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7fec00961b80 4 client.25757 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.456+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55738672eeb0 con 0x5573867fe410 2023-07-21T22:22:24.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.457+0000 7febe6d9d700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5573867fe410 0x557386800820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.457+0000 7febe6d9d700 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5573867fe410 0x557386800820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c910285a63555864 server_cookie=2eee5b8389bb71f3 in_seq=0 out_seq=0 2023-07-21T22:22:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.457+0000 7febcdffb700 10 client.25757 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.458+0000 7febcdffb700 1 -- 172.21.15.66:0/4004342888 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55738672eeb0 con 0x5573867fe410 2023-07-21T22:22:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.458+0000 7febcdffb700 10 client.25757 handle_command_reply: tid=0 2023-07-21T22:22:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.458+0000 7fec00961b80 1 client.25757 shutdown 2023-07-21T22:22:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 msgr2=0x5573867ede80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 0x5573867ede80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7febdc009990 tx=0x7febdc00b610 comp rx=0 tx=0).stop 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 shutdown_connections 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5573867fe410 0x557386800820 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e9ac0 0x5573867ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e4930 0x5573867f0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 --2- 172.21.15.66:0/4004342888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e91b0 0x5573867ede80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 >> 172.21.15.66:0/4004342888 conn(0x5573867d9ef0 msgr2=0x5573867df420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7febcdffb700 0 client.25757 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 shutdown_connections 2023-07-21T22:22:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 1 -- 172.21.15.66:0/4004342888 wait complete. 2023-07-21T22:22:24.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.459+0000 7fec00961b80 20 client.25757 trim_cache size 0 max 16384 2023-07-21T22:22:24.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 Processor -- start 2023-07-21T22:22:24.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 -- start start 2023-07-21T22:22:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573867d99b0 con 0x5573867e9ac0 2023-07-21T22:22:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573867e56d0 con 0x5573867e4930 2023-07-21T22:22:24.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.486+0000 7fec00961b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573867dc010 con 0x5573867e91b0 2023-07-21T22:22:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.487+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.487+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48048/0 (socket says 172.21.15.66:48048) 2023-07-21T22:22:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.487+0000 7febe6d9d700 1 -- 172.21.15.66:0/1669437904 learned_addr learned my addr 172.21.15.66:0/1669437904 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.487+0000 7febe6d9d700 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.487+0000 7febe6d9d700 1 -- 172.21.15.66:0/1669437904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 msgr2=0x5573867e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.487+0000 7febe6d9d700 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febe6d9d700 1 -- 172.21.15.66:0/1669437904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 msgr2=0x5573867e2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febe6d9d700 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867e2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febe6d9d700 1 -- 172.21.15.66:0/1669437904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55738658a250 con 0x5573867e4930 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febe6d9d700 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867f5570 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febe0009010 tx=0x7febe000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df738954b6c94d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febd57fa700 1 -- 172.21.15.66:0/1669437904 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0019040 con 0x5573867e4930 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febd57fa700 1 -- 172.21.15.66:0/1669437904 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe002d030 con 0x5573867e4930 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.488+0000 7febd57fa700 1 -- 172.21.15.66:0/1669437904 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0015480 con 0x5573867e4930 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 -- 172.21.15.66:0/1669437904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 msgr2=0x5573867f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867f5570 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7febe0009010 tx=0x7febe000b040 comp rx=0 tx=0).stop 2023-07-21T22:22:24.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 -- 172.21.15.66:0/1669437904 shutdown_connections 2023-07-21T22:22:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867f5570 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 --2- 172.21.15.66:0/1669437904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.489+0000 7fec00961b80 1 -- 172.21.15.66:0/1669437904 >> 172.21.15.66:0/1669437904 conn(0x5573867d9ef0 msgr2=0x5573867fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.490+0000 7fec00961b80 1 -- 172.21.15.66:0/1669437904 shutdown_connections 2023-07-21T22:22:24.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.490+0000 7fec00961b80 1 -- 172.21.15.66:0/1669437904 wait complete. 2023-07-21T22:22:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.490+0000 7fec00961b80 1 -- start start 2023-07-21T22:22:24.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.491+0000 7fec00961b80 1 Processor -- start 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.491+0000 7fec00961b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7fec00961b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7fec00961b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867ed0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7fec00961b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573867ed7c0 con 0x5573867e9ac0 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7fec00961b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573867ed930 con 0x5573867e4930 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7fec00961b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573867edaf0 con 0x5573867e91b0 2023-07-21T22:22:24.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7fec00961b80 10 client.0 _ll_register_callbacks cb 0x7febdc00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7febe6d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48060/0 (socket says 172.21.15.66:48060) 2023-07-21T22:22:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7febe6d9d700 1 -- 172.21.15.66:0/5242261 learned_addr learned my addr 172.21.15.66:0/5242261 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.492+0000 7febe6d9d700 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867ed0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febe6d9d700 1 -- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 msgr2=0x5573867ecb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:22:24.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febe6d9d700 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867ecb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febe6d9d700 1 -- 172.21.15.66:0/5242261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 msgr2=0x5573867ed0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febe6d9d700 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867ed0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febe6d9d700 1 -- 172.21.15.66:0/5242261 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557386563de0 con 0x5573867e4930 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febe6d9d700 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867ec630 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febe00090a0 tx=0x7febe0014850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=339fe779bfafa01 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.493+0000 7febcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7febcdffb700 1 -- 172.21.15.66:0/5242261 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0019070 con 0x5573867e4930 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febe000f040 con 0x5573867e4930 2023-07-21T22:22:24.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7febcdffb700 1 -- 172.21.15.66:0/5242261 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7febe0026330 con 0x5573867e4930 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7febcdffb700 1 -- 172.21.15.66:0/5242261 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe0027e60 con 0x5573867e4930 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7fec00961b80 10 client.25763 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5573865622b0 con 0x5573867e4930 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.494+0000 7febcdffb700 1 -- 172.21.15.66:0/5242261 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe000f040 con 0x5573867e4930 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7febcdffb700 1 -- 172.21.15.66:0/5242261 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7febe003d070 con 0x5573867e4930 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7fec00961b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7fec00961b80 10 client.25763 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:22:24.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7fec00961b80 20 client.25763 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7fec00961b80 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x557386852df0 0x557386855200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7fec00961b80 4 client.25763 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:22:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5573866f1050 con 0x557386852df0 2023-07-21T22:22:24.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.495+0000 7febe5d9b700 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x557386852df0 0x557386855200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.496+0000 7febe5d9b700 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x557386852df0 0x557386855200 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f23cc33f1d856af server_cookie=c7d38be883c1c82b in_seq=0 out_seq=0 2023-07-21T22:22:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.496+0000 7febcdffb700 10 client.25763 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.496+0000 7febcdffb700 1 -- 172.21.15.66:0/5242261 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5573866f1050 con 0x557386852df0 2023-07-21T22:22:24.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.496+0000 7febcdffb700 10 client.25763 handle_command_reply: tid=0 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.496+0000 7fec00961b80 1 client.25763 shutdown 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 msgr2=0x5573867ec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867ec630 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febe00090a0 tx=0x7febe0014850 comp rx=0 tx=0).stop 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 shutdown_connections 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x557386852df0 0x557386855200 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573867e4930 0x5573867ec630 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5573867e9ac0 0x5573867ed0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 --2- 172.21.15.66:0/5242261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5573867e91b0 0x5573867ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 >> 172.21.15.66:0/5242261 conn(0x5573867d9ef0 msgr2=0x5573867e2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7febcdffb700 0 client.25763 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:24.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 shutdown_connections 2023-07-21T22:22:24.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 1 -- 172.21.15.66:0/5242261 wait complete. 2023-07-21T22:22:24.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.497+0000 7fec00961b80 20 client.25763 trim_cache size 0 max 16384 2023-07-21T22:22:24.547 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:22:24.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febd00a14e0 msgr2=0x7febd00a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febd00a14e0 0x7febd00a3990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7febd80ca6a0 tx=0x7febd8010c60 comp rx=0 tx=0).stop 2023-07-21T22:22:24.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe8100ca0 msgr2=0x7febe80fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe8100ca0 0x7febe80fe8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7febd800e9f0 tx=0x7febd800ed00 comp rx=0 tx=0).stop 2023-07-21T22:22:24.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 shutdown_connections 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febe8100ca0 0x7febe80fe8f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febd00a14e0 0x7febd00a3990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febe810a070 0x7febe80ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 --2- 172.21.15.66:0/378241674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febe81015b0 0x7febe80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.498+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 >> 172.21.15.66:0/378241674 conn(0x7febe80fa4b0 msgr2=0x7febe810c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.499+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 shutdown_connections 2023-07-21T22:22:24.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.499+0000 7febecfff700 1 -- 172.21.15.66:0/378241674 wait complete. 2023-07-21T22:22:24.571 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:22:24.902 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:24 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2806274822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:22:24.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 -- 172.21.15.66:0/3748935690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff098100cc0 msgr2=0x7ff098101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 --2- 172.21.15.66:0/3748935690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff098100cc0 0x7ff098101090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff094009990 tx=0x7ff094009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 -- 172.21.15.66:0/3748935690 shutdown_connections 2023-07-21T22:22:24.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 --2- 172.21.15.66:0/3748935690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff098100cc0 0x7ff098101090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 --2- 172.21.15.66:0/3748935690 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff09810a090 0x7ff09810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 --2- 172.21.15.66:0/3748935690 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0981015d0 0x7ff098109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 -- 172.21.15.66:0/3748935690 >> 172.21.15.66:0/3748935690 conn(0x7ff0980fa4b0 msgr2=0x7ff0980fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 -- 172.21.15.66:0/3748935690 shutdown_connections 2023-07-21T22:22:24.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.880+0000 7ff09fbff700 1 -- 172.21.15.66:0/3748935690 wait complete. 2023-07-21T22:22:24.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.882+0000 7ff09fbff700 1 Processor -- start 2023-07-21T22:22:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.882+0000 7ff09fbff700 1 -- start start 2023-07-21T22:22:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09fbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff098100cc0 0x7ff098197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09fbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0981015d0 0x7ff0981983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09fbff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff09810a090 0x7ff09819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09fbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09819ce90 con 0x7ff0981015d0 2023-07-21T22:22:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09fbff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09819d000 con 0x7ff09810a090 2023-07-21T22:22:24.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09fbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09819d1c0 con 0x7ff098100cc0 2023-07-21T22:22:24.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09d99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff098100cc0 0x7ff098197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09d99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff098100cc0 0x7ff098197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49592/0 (socket says 172.21.15.66:49592) 2023-07-21T22:22:24.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09d99b700 1 -- 172.21.15.66:0/2904331403 learned_addr learned my addr 172.21.15.66:0/2904331403 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0981015d0 0x7ff0981983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.883+0000 7ff09e19c700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff09810a090 0x7ff09819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff09e19c700 1 -- 172.21.15.66:0/2904331403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff098100cc0 msgr2=0x7ff098197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff09e19c700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff098100cc0 0x7ff098197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff09e19c700 1 -- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0981015d0 msgr2=0x7ff0981983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff09e19c700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0981015d0 0x7ff0981983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff09e19c700 1 -- 172.21.15.66:0/2904331403 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940095f0 con 0x7ff09810a090 2023-07-21T22:22:24.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff09e19c700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff09810a090 0x7ff09819c850 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff09000efc0 tx=0x7ff09000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.884+0000 7ff08effd700 1 -- 172.21.15.66:0/2904331403 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090018070 con 0x7ff09810a090 2023-07-21T22:22:24.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.885+0000 7ff08effd700 1 -- 172.21.15.66:0/2904331403 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff09000f040 con 0x7ff09810a090 2023-07-21T22:22:24.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.885+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09819d440 con 0x7ff09810a090 2023-07-21T22:22:24.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.885+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0981a4ce0 con 0x7ff09810a090 2023-07-21T22:22:24.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.885+0000 7ff08effd700 1 -- 172.21.15.66:0/2904331403 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09000cd40 con 0x7ff09810a090 2023-07-21T22:22:24.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.890+0000 7ff08effd700 1 -- 172.21.15.66:0/2904331403 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7ff090003a00 con 0x7ff09810a090 2023-07-21T22:22:24.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.891+0000 7ff0b34c8b80 1 Processor -- start 2023-07-21T22:22:24.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 -- start start 2023-07-21T22:22:24.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337aca1b0 0x564337aca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 0x564337ac5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ac5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643379cacb0 con 0x564337acaac0 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337a1da40 con 0x564337aca1b0 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.892+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337a16ed0 con 0x564337ac5930 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 0x564337ac5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 0x564337ac5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55490/0 (socket says 172.21.15.66:55490) 2023-07-21T22:22:24.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d99b700 1 -- 172.21.15.66:0/1782613989 learned_addr learned my addr 172.21.15.66:0/1782613989 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff08effd700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0840a14e0 0x7ff0840a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff08effd700 1 -- 172.21.15.66:0/2904331403 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff0900ccd60 con 0x7ff09810a090 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d99b700 1 -- 172.21.15.66:0/1782613989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 msgr2=0x564337ac5da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d99b700 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ac5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d99b700 1 -- 172.21.15.66:0/1782613989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337aca1b0 msgr2=0x564337aca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.893+0000 7ff09d19a700 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337aca1b0 0x564337aca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.894+0000 7ff09d99b700 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337aca1b0 0x564337aca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.894+0000 7ff09d99b700 1 -- 172.21.15.66:0/1782613989 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564337844de0 con 0x564337acaac0 2023-07-21T22:22:24.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.894+0000 7ff09d19a700 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337aca1b0 0x564337aca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:24.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.894+0000 7ff09e19c700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0840a14e0 0x7ff0840a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.894+0000 7ff09d99b700 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 0x564337ac5350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff09400b640 tx=0x7ff09400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc758d9541967bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.895+0000 7ff083fff700 1 -- 172.21.15.66:0/1782613989 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09401d030 con 0x564337acaac0 2023-07-21T22:22:24.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.895+0000 7ff083fff700 1 -- 172.21.15.66:0/1782613989 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff09401e040 con 0x564337acaac0 2023-07-21T22:22:24.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.895+0000 7ff083fff700 1 -- 172.21.15.66:0/1782613989 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094004b40 con 0x564337acaac0 2023-07-21T22:22:24.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/1782613989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 msgr2=0x564337ac5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff09e19c700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0840a14e0 0x7ff0840a3990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0900c9fe0 tx=0x7ff090004da0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 0x564337ac5350 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff09400b640 tx=0x7ff09400b670 comp rx=0 tx=0).stop 2023-07-21T22:22:24.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/1782613989 shutdown_connections 2023-07-21T22:22:24.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337aca1b0 0x564337aca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ac5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/1782613989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337acaac0 0x564337ac5350 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/1782613989 >> 172.21.15.66:0/1782613989 conn(0x564337abaef0 msgr2=0x564337ac0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/1782613989 shutdown_connections 2023-07-21T22:22:24.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.896+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/1782613989 wait complete. 2023-07-21T22:22:24.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.897+0000 7ff0b34c8b80 1 -- start start 2023-07-21T22:22:24.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.897+0000 7ff0b34c8b80 1 Processor -- start 2023-07-21T22:22:24.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ad1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337acef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337acf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643379b6ac0 con 0x564337aca1b0 2023-07-21T22:22:24.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643379b67c0 con 0x564337acaac0 2023-07-21T22:22:24.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56433789a920 con 0x564337ac5930 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.898+0000 7ff0b34c8b80 10 client.0 _ll_register_callbacks cb 0x5643379b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ad1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ad1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49608/0 (socket says 172.21.15.66:49608) 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 -- 172.21.15.66:0/2616668384 learned_addr learned my addr 172.21.15.66:0/2616668384 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337acf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d99b700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337acef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 -- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 msgr2=0x564337acf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337acf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 -- 172.21.15.66:0/2616668384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 msgr2=0x564337acef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337acef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d19a700 1 -- 172.21.15.66:0/2616668384 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09810d170 con 0x564337ac5930 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.899+0000 7ff09d99b700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337acef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.900+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ad1c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff088012d10 tx=0x7ff0880096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398289aeec3117e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.900+0000 7ff0817fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:24.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.900+0000 7ff0817fa700 1 -- 172.21.15.66:0/2616668384 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088012110 con 0x564337ac5930 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.900+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643379704f0 con 0x564337ac5930 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.900+0000 7ff0817fa700 1 -- 172.21.15.66:0/2616668384 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff088012270 con 0x564337ac5930 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0817fa700 1 -- 172.21.15.66:0/2616668384 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff08800fc70 con 0x564337ac5930 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0b34c8b80 10 client.25753 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56433782dc60 con 0x564337ac5930 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0817fa700 1 -- 172.21.15.66:0/2616668384 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5643379704f0 con 0x564337ac5930 2023-07-21T22:22:24.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0817fa700 1 -- 172.21.15.66:0/2616668384 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff08801e070 con 0x564337ac5930 2023-07-21T22:22:24.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0b34c8b80 10 client.25753 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:24.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0b34c8b80 10 client.25753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:22:24.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0b34c8b80 20 client.25753 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:24.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.901+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337af8d20 0x564337afb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.902+0000 7ff0b34c8b80 4 client.25753 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:24.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.902+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564337a0feb0 con 0x564337af8d20 2023-07-21T22:22:24.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.903+0000 7ff09d99b700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337af8d20 0x564337afb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.903+0000 7ff09d99b700 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337af8d20 0x564337afb130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1204aab398edfe6 server_cookie=a019c6ee7841fc35 in_seq=0 out_seq=0 2023-07-21T22:22:24.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.903+0000 7ff0817fa700 10 client.25753 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:24.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.904+0000 7ff0817fa700 1 -- 172.21.15.66:0/2616668384 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564337a0feb0 con 0x564337af8d20 2023-07-21T22:22:24.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.904+0000 7ff0817fa700 10 client.25753 handle_command_reply: tid=0 2023-07-21T22:22:24.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.904+0000 7ff0b34c8b80 1 client.25753 shutdown 2023-07-21T22:22:24.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 msgr2=0x564337ad1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ad1c40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff088012d10 tx=0x7ff0880096d0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 shutdown_connections 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337af8d20 0x564337afb130 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337acf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337ad1c40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2616668384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337acef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 >> 172.21.15.66:0/2616668384 conn(0x564337abaef0 msgr2=0x564337ac0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0817fa700 0 client.25753 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 shutdown_connections 2023-07-21T22:22:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.905+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2616668384 wait complete. 2023-07-21T22:22:24.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.906+0000 7ff0b34c8b80 20 client.25753 trim_cache size 0 max 16384 2023-07-21T22:22:24.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.932+0000 7ff0b34c8b80 1 Processor -- start 2023-07-21T22:22:24.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.932+0000 7ff0b34c8b80 1 -- start start 2023-07-21T22:22:24.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 0x564337adee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337aca1b0 0x564337ad17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337ac3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337ad04b0 con 0x564337ac5930 2023-07-21T22:22:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337adf380 con 0x564337acaac0 2023-07-21T22:22:24.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337affa50 con 0x564337aca1b0 2023-07-21T22:22:24.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.933+0000 7ff09d19a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 0x564337adee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 0x564337adee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55508/0 (socket says 172.21.15.66:55508) 2023-07-21T22:22:24.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 -- 172.21.15.66:0/2224357336 learned_addr learned my addr 172.21.15.66:0/2224357336 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337ac3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d99b700 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337aca1b0 0x564337ad17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 -- 172.21.15.66:0/2224357336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337aca1b0 msgr2=0x564337ad17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337aca1b0 0x564337ad17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 -- 172.21.15.66:0/2224357336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 msgr2=0x564337ac3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337ac3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 -- 172.21.15.66:0/2224357336 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56433786b250 con 0x564337ac5930 2023-07-21T22:22:24.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.934+0000 7ff09d19a700 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 0x564337adee40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff088009d80 tx=0x7ff088012580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4270603235b6af33 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.935+0000 7ff08cbff700 1 -- 172.21.15.66:0/2224357336 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088003c70 con 0x564337ac5930 2023-07-21T22:22:24.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.935+0000 7ff08cbff700 1 -- 172.21.15.66:0/2224357336 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0880055a0 con 0x564337ac5930 2023-07-21T22:22:24.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.935+0000 7ff08cbff700 1 -- 172.21.15.66:0/2224357336 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800fed0 con 0x564337ac5930 2023-07-21T22:22:24.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.935+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2224357336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 msgr2=0x564337adee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.935+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 0x564337adee40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff088009d80 tx=0x7ff088012580 comp rx=0 tx=0).stop 2023-07-21T22:22:24.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2224357336 shutdown_connections 2023-07-21T22:22:24.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337ac3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337ac5930 0x564337adee40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/2224357336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337aca1b0 0x564337ad17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2224357336 >> 172.21.15.66:0/2224357336 conn(0x564337abaef0 msgr2=0x564337abef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2224357336 shutdown_connections 2023-07-21T22:22:24.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/2224357336 wait complete. 2023-07-21T22:22:24.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.936+0000 7ff0b34c8b80 1 -- start start 2023-07-21T22:22:24.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 Processor -- start 2023-07-21T22:22:24.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337b40d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337b41260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337b44cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337b45270 con 0x564337aca1b0 2023-07-21T22:22:24.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337b453e0 con 0x564337acaac0 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564337b455a0 con 0x564337ac5930 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.937+0000 7ff0b34c8b80 10 client.0 _ll_register_callbacks cb 0x7ff088012a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337b40d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337b40d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49628/0 (socket says 172.21.15.66:49628) 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 -- 172.21.15.66:0/4223437351 learned_addr learned my addr 172.21.15.66:0/4223437351 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337b44cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d99b700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337b41260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 -- 172.21.15.66:0/4223437351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 msgr2=0x564337b40d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337b40d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.938+0000 7ff09d19a700 1 -- 172.21.15.66:0/4223437351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 msgr2=0x564337b41260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:24.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff09d19a700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337b41260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff09d19a700 1 -- 172.21.15.66:0/4223437351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56433782dc60 con 0x564337acaac0 2023-07-21T22:22:24.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff09d99b700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337b41260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff09d19a700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337b44cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff08800fa30 tx=0x7ff0880096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dcabe371bf2253a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff0817fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff0817fa700 1 -- 172.21.15.66:0/4223437351 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800fc70 con 0x564337acaac0 2023-07-21T22:22:24.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff09401e040 con 0x564337acaac0 2023-07-21T22:22:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.939+0000 7ff0817fa700 1 -- 172.21.15.66:0/4223437351 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff088000e40 con 0x564337acaac0 2023-07-21T22:22:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.940+0000 7ff0817fa700 1 -- 172.21.15.66:0/4223437351 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff08800fdd0 con 0x564337acaac0 2023-07-21T22:22:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.940+0000 7ff0b34c8b80 10 client.25778 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:24.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.940+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5643378432b0 con 0x564337acaac0 2023-07-21T22:22:24.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.940+0000 7ff0817fa700 1 -- 172.21.15.66:0/4223437351 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff09401e040 con 0x564337acaac0 2023-07-21T22:22:24.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0817fa700 1 -- 172.21.15.66:0/4223437351 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff08801e030 con 0x564337acaac0 2023-07-21T22:22:24.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0b34c8b80 10 client.25778 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:24.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0b34c8b80 10 client.25778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:22:24.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0b34c8b80 20 client.25778 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:24.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337b4dac0 0x564337b4fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:24.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0b34c8b80 4 client.25778 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:22:24.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5643379d2050 con 0x564337b4dac0 2023-07-21T22:22:24.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.941+0000 7ff09d99b700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337b4dac0 0x564337b4fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:24.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff09d99b700 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337b4dac0 0x564337b4fed0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0eb9f6216aef12d server_cookie=777b4d61223f057d in_seq=0 out_seq=0 2023-07-21T22:22:24.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0817fa700 10 client.25778 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0817fa700 1 -- 172.21.15.66:0/4223437351 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5643379d2050 con 0x564337b4dac0 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0817fa700 10 client.25778 handle_command_reply: tid=0 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 client.25778 shutdown 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 msgr2=0x564337b44cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337b44cc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff08800fa30 tx=0x7ff0880096d0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 shutdown_connections 2023-07-21T22:22:24.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x564337b4dac0 0x564337b4fed0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564337acaac0 0x564337b44cc0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff08800fa30 tx=0x7ff0880096d0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564337ac5930 0x564337b40d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0817fa700 0 client.25778 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 --2- 172.21.15.66:0/4223437351 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564337aca1b0 0x564337b41260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.942+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 >> 172.21.15.66:0/4223437351 conn(0x564337abaef0 msgr2=0x564337ac3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.943+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 shutdown_connections 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.943+0000 7ff0b34c8b80 1 -- 172.21.15.66:0/4223437351 wait complete. 2023-07-21T22:22:24.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.943+0000 7ff0b34c8b80 20 client.25778 trim_cache size 0 max 16384 2023-07-21T22:22:24.999 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:22:24.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0840a14e0 msgr2=0x7ff0840a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0840a14e0 0x7ff0840a3990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0900c9fe0 tx=0x7ff090004da0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff09810a090 msgr2=0x7ff09819c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff09810a090 0x7ff09819c850 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff09000efc0 tx=0x7ff09000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 shutdown_connections 2023-07-21T22:22:25.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff09810a090 0x7ff09819c850 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0840a14e0 0x7ff0840a3990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff098100cc0 0x7ff098197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 --2- 172.21.15.66:0/2904331403 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0981015d0 0x7ff0981983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 >> 172.21.15.66:0/2904331403 conn(0x7ff0980fa4b0 msgr2=0x7ff09810c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 shutdown_connections 2023-07-21T22:22:25.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:24.944+0000 7ff09fbff700 1 -- 172.21.15.66:0/2904331403 wait complete. 2023-07-21T22:22:25.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:22:25.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 -- 172.21.15.66:0/3584854814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 msgr2=0x7f36b00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 --2- 172.21.15.66:0/3584854814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 0x7f36b00fff40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009990 tx=0x7f36a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 -- 172.21.15.66:0/3584854814 shutdown_connections 2023-07-21T22:22:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 --2- 172.21.15.66:0/3584854814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0105ae0 0x7f36b0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 --2- 172.21.15.66:0/3584854814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b0100480 0x7f36b010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 --2- 172.21.15.66:0/3584854814 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 0x7f36b00fff40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 -- 172.21.15.66:0/3584854814 >> 172.21.15.66:0/3584854814 conn(0x7f36b00fb5f0 msgr2=0x7f36b00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 -- 172.21.15.66:0/3584854814 shutdown_connections 2023-07-21T22:22:25.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.181+0000 7f36b77ff700 1 -- 172.21.15.66:0/3584854814 wait complete. 2023-07-21T22:22:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.182+0000 7f36b77ff700 1 Processor -- start 2023-07-21T22:22:25.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.182+0000 7f36b77ff700 1 -- start start 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b77ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 0x7f36b019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b77ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 0x7f36b019bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b77ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b0105ae0 0x7f36b0195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b77ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01960c0 con 0x7f36b00ffad0 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b77ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0196230 con 0x7f36b0100480 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b77ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b01963f0 con 0x7f36b0105ae0 2023-07-21T22:22:25.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 0x7f36b019bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 0x7f36b019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 0x7f36b019bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48142/0 (socket says 172.21.15.66:48142) 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.183+0000 7f36b4d9a700 1 -- 172.21.15.66:0/903674181 learned_addr learned my addr 172.21.15.66:0/903674181 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b5d9c700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b0105ae0 0x7f36b0195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b4d9a700 1 -- 172.21.15.66:0/903674181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b0105ae0 msgr2=0x7f36b0195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b0105ae0 0x7f36b0195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b4d9a700 1 -- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 msgr2=0x7f36b019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 0x7f36b019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b4d9a700 1 -- 172.21.15.66:0/903674181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a00095f0 con 0x7f36b0100480 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.184+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 0x7f36b019bf20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.185+0000 7f36a67fc700 1 -- 172.21.15.66:0/903674181 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a001d070 con 0x7f36b0100480 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.185+0000 7f36a67fc700 1 -- 172.21.15.66:0/903674181 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a000fb80 con 0x7f36b0100480 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.185+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b0196670 con 0x7f36b0100480 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.185+0000 7f36a67fc700 1 -- 172.21.15.66:0/903674181 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00173f0 con 0x7f36b0100480 2023-07-21T22:22:25.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.185+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b0072920 con 0x7f36b0100480 2023-07-21T22:22:25.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.189+0000 7f36a67fc700 1 -- 172.21.15.66:0/903674181 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f36a000fcf0 con 0x7f36b0100480 2023-07-21T22:22:25.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.190+0000 7f36a67fc700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f369c0a1460 0x7f369c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.190+0000 7f36a67fc700 1 -- 172.21.15.66:0/903674181 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f36a00ce490 con 0x7f36b0100480 2023-07-21T22:22:25.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.191+0000 7f36b559b700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f369c0a1460 0x7f369c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.191+0000 7f36cb0f1b80 1 Processor -- start 2023-07-21T22:22:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36b559b700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f369c0a1460 0x7f369c0a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009be0 tx=0x7f36ac009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:25.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 -- start start 2023-07-21T22:22:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301779c0 0x557130177d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 0x55713016a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713016ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713006fcb0 con 0x5571301779c0 2023-07-21T22:22:25.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571300c2a40 con 0x5571301782d0 2023-07-21T22:22:25.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.192+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571300bbed0 con 0x55713016a930 2023-07-21T22:22:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 0x55713016a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301779c0 0x557130177d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 0x55713016a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48152/0 (socket says 172.21.15.66:48152) 2023-07-21T22:22:25.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b4d9a700 1 -- 172.21.15.66:0/2879465222 learned_addr learned my addr 172.21.15.66:0/2879465222 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b5d9c700 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713016ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b559b700 1 -- 172.21.15.66:0/2879465222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 msgr2=0x55713016ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b559b700 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713016ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b559b700 1 -- 172.21.15.66:0/2879465222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 msgr2=0x55713016a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b559b700 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 0x55713016a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b559b700 1 -- 172.21.15.66:0/2879465222 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712fee9de0 con 0x5571301779c0 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.193+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 0x55713016a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:25.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.194+0000 7f36b559b700 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301779c0 0x557130177d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0117d0 tx=0x7f36ac014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab35577bc9323901 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.194+0000 7f369b7fe700 1 -- 172.21.15.66:0/2879465222 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0119e0 con 0x5571301779c0 2023-07-21T22:22:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.194+0000 7f369b7fe700 1 -- 172.21.15.66:0/2879465222 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ac0044e0 con 0x5571301779c0 2023-07-21T22:22:25.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.194+0000 7f369b7fe700 1 -- 172.21.15.66:0/2879465222 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac003ae0 con 0x5571301779c0 2023-07-21T22:22:25.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:24 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2806274822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:22:25.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2879465222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301779c0 msgr2=0x557130177d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301779c0 0x557130177d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0117d0 tx=0x7f36ac014000 comp rx=0 tx=0).stop 2023-07-21T22:22:25.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2879465222 shutdown_connections 2023-07-21T22:22:25.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713016ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301779c0 0x557130177d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2879465222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301782d0 0x55713016a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2879465222 >> 172.21.15.66:0/2879465222 conn(0x55713015fef0 msgr2=0x557130165830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2879465222 shutdown_connections 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.195+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2879465222 wait complete. 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.196+0000 7f36cb0f1b80 1 -- start start 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.196+0000 7f36cb0f1b80 1 Processor -- start 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713017f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713005bac0 con 0x5571301782d0 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713005b7c0 con 0x5571301779c0 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55712ff3f920 con 0x55713016a930 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36cb0f1b80 10 client.0 _ll_register_callbacks cb 0x557130059760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.197+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48154/0 (socket says 172.21.15.66:48154) 2023-07-21T22:22:25.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b4d9a700 1 -- 172.21.15.66:0/3284502423 learned_addr learned my addr 172.21.15.66:0/3284502423 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b559b700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713017f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b5d9c700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b559b700 1 -- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 msgr2=0x55713017c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b559b700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b559b700 1 -- 172.21.15.66:0/3284502423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 msgr2=0x55713017cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b559b700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b559b700 1 -- 172.21.15.66:0/3284502423 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b0072920 con 0x55713016a930 2023-07-21T22:22:25.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.198+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:25.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f36b559b700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713017f360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00efd0 tx=0x7f36ac005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d47fd972d93e1ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:25.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f3698ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f3698ff9700 1 -- 172.21.15.66:0/3284502423 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ac01e8b0 con 0x55713016a930 2023-07-21T22:22:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571300154f0 con 0x55713016a930 2023-07-21T22:22:25.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f3698ff9700 1 -- 172.21.15.66:0/3284502423 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f36ac01ea10 con 0x55713016a930 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f3698ff9700 1 -- 172.21.15.66:0/3284502423 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36ac01ebd0 con 0x55713016a930 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f36cb0f1b80 10 client.25768 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.199+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f36ac017070 con 0x55713016a930 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f3698ff9700 1 -- 172.21.15.66:0/3284502423 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571300154f0 con 0x55713016a930 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f3698ff9700 1 -- 172.21.15.66:0/3284502423 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f36ac024030 con 0x55713016a930 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f36cb0f1b80 10 client.25768 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:25.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f36cb0f1b80 10 client.25768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f36cb0f1b80 20 client.25768 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55713018ca50 0x55713018ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f36cb0f1b80 4 client.25768 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.200+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571300b4eb0 con 0x55713018ca50 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.201+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55713018ca50 0x55713018ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.201+0000 7f3698ff9700 10 client.25768 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.201+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55713018ca50 0x55713018ee30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b59913cb5363a2ca server_cookie=beb2236f47a31d87 in_seq=0 out_seq=0 2023-07-21T22:22:25.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f3698ff9700 1 -- 172.21.15.66:0/3284502423 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571300b4eb0 con 0x55713018ca50 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f3698ff9700 10 client.25768 handle_command_reply: tid=0 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 client.25768 shutdown 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 msgr2=0x55713017f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713017f360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00efd0 tx=0x7f36ac005700 comp rx=0 tx=0).stop 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 shutdown_connections 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55713018ca50 0x55713018ee30 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f3698ff9700 0 client.25768 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x55713017f360 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/3284502423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.202+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 >> 172.21.15.66:0/3284502423 conn(0x55713015fef0 msgr2=0x557130165420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.203+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 shutdown_connections 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.203+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/3284502423 wait complete. 2023-07-21T22:22:25.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.203+0000 7f36cb0f1b80 20 client.25768 trim_cache size 0 max 16384 2023-07-21T22:22:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.229+0000 7f36cb0f1b80 1 Processor -- start 2023-07-21T22:22:25.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 -- start start 2023-07-21T22:22:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x557130183d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x5571301652d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713015d050 con 0x5571301782d0 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713017dbc0 con 0x5571301779c0 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.230+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713017fb10 con 0x55713016a930 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b559b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x557130183d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48160/0 (socket says 172.21.15.66:48160) 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 -- 172.21.15.66:0/1151995563 learned_addr learned my addr 172.21.15.66:0/1151995563 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:25.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b5d9c700 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x5571301652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 -- 172.21.15.66:0/1151995563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 msgr2=0x557130183d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x557130183d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 -- 172.21.15.66:0/1151995563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 msgr2=0x5571301652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x5571301652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 -- 172.21.15.66:0/1151995563 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55712fed2c60 con 0x5571301779c0 2023-07-21T22:22:25.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.231+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017ef00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f36a00cb650 tx=0x7f36a0005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1662fca923334b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:25.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.232+0000 7f369bfff700 1 -- 172.21.15.66:0/1151995563 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a009f140 con 0x5571301779c0 2023-07-21T22:22:25.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.232+0000 7f369bfff700 1 -- 172.21.15.66:0/1151995563 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36a00a34d0 con 0x5571301779c0 2023-07-21T22:22:25.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.232+0000 7f369bfff700 1 -- 172.21.15.66:0/1151995563 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00a32a0 con 0x5571301779c0 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/1151995563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 msgr2=0x55713017ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017ef00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f36a00cb650 tx=0x7f36a0005ad0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/1151995563 shutdown_connections 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x5571301652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55713016a930 0x557130183d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/1151995563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5571301779c0 0x55713017ef00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/1151995563 >> 172.21.15.66:0/1151995563 conn(0x55713015fef0 msgr2=0x557130163f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/1151995563 shutdown_connections 2023-07-21T22:22:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.233+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/1151995563 wait complete. 2023-07-21T22:22:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.234+0000 7f36cb0f1b80 1 -- start start 2023-07-21T22:22:25.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.234+0000 7f36cb0f1b80 1 Processor -- start 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55713016a930 0x557130183d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 0x55713017ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713017b810 con 0x5571301782d0 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713017b980 con 0x55713016a930 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55713017bb40 con 0x5571301779c0 2023-07-21T22:22:25.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36cb0f1b80 10 client.0 _ll_register_callbacks cb 0x557130059760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:25.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.235+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 0x55713017ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b5d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b4d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 0x55713017ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49686/0 (socket says 172.21.15.66:49686) 2023-07-21T22:22:25.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b5d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55582/0 (socket says 172.21.15.66:55582) 2023-07-21T22:22:25.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b4d9a700 1 -- 172.21.15.66:0/2173865129 learned_addr learned my addr 172.21.15.66:0/2173865129 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:25.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b559b700 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55713016a930 0x557130183d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b4d9a700 1 -- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55713016a930 msgr2=0x557130183d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55713016a930 0x557130183d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b4d9a700 1 -- 172.21.15.66:0/2173865129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 msgr2=0x55713017b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.236+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.237+0000 7f36b4d9a700 1 -- 172.21.15.66:0/2173865129 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b0196670 con 0x5571301779c0 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.237+0000 7f36b4d9a700 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 0x55713017ad90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005910 tx=0x7f36a0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e739697d383e26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.237+0000 7f3698ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.237+0000 7f3698ff9700 1 -- 172.21.15.66:0/2173865129 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a00b92e0 con 0x5571301779c0 2023-07-21T22:22:25.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.237+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36ac0044e0 con 0x5571301779c0 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.237+0000 7f3698ff9700 1 -- 172.21.15.66:0/2173865129 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f36a00b9440 con 0x5571301779c0 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f3698ff9700 1 -- 172.21.15.66:0/2173865129 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36a00cb2b0 con 0x5571301779c0 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f36cb0f1b80 10 client.25774 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55712ff10250 con 0x5571301779c0 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f3698ff9700 1 -- 172.21.15.66:0/2173865129 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36ac0044e0 con 0x5571301779c0 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f3698ff9700 1 -- 172.21.15.66:0/2173865129 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f36a00cbab0 con 0x5571301779c0 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f36cb0f1b80 10 client.25774 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:25.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f36cb0f1b80 10 client.25774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.238+0000 7f36cb0f1b80 20 client.25774 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.239+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5571301e18c0 0x5571301e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.239+0000 7f36cb0f1b80 4 client.25774 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:22:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.239+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557130077050 con 0x5571301e18c0 2023-07-21T22:22:25.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.239+0000 7f36b559b700 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5571301e18c0 0x5571301e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:25.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.239+0000 7f36b559b700 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5571301e18c0 0x5571301e3cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34a696e6f7608d14 server_cookie=4c7965ae3a20c356 in_seq=0 out_seq=0 2023-07-21T22:22:25.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.239+0000 7f3698ff9700 10 client.25774 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:25.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f3698ff9700 1 -- 172.21.15.66:0/2173865129 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557130077050 con 0x5571301e18c0 2023-07-21T22:22:25.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f3698ff9700 10 client.25774 handle_command_reply: tid=0 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 client.25774 shutdown 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 msgr2=0x55713017ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 0x55713017ad90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36a0005910 tx=0x7f36a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 shutdown_connections 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55713016a930 0x557130183d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5571301e18c0 0x5571301e3cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5571301782d0 0x55713017b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 --2- 172.21.15.66:0/2173865129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5571301779c0 0x55713017ad90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f3698ff9700 0 client.25774 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 >> 172.21.15.66:0/2173865129 conn(0x55713015fef0 msgr2=0x5571301652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 shutdown_connections 2023-07-21T22:22:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.240+0000 7f36cb0f1b80 1 -- 172.21.15.66:0/2173865129 wait complete. 2023-07-21T22:22:25.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36cb0f1b80 20 client.25774 trim_cache size 0 max 16384 2023-07-21T22:22:25.290 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:22:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f369c0a1460 msgr2=0x7f369c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f369c0a1460 0x7f369c0a3910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009be0 tx=0x7f36ac009260 comp rx=0 tx=0).stop 2023-07-21T22:22:25.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 msgr2=0x7f36b019bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 0x7f36b019bf20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a00047b0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 shutdown_connections 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f369c0a1460 0x7f369c0a3910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f36b0105ae0 0x7f36b0195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f36b00ffad0 0x7f36b019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 --2- 172.21.15.66:0/903674181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36b0100480 0x7f36b019bf20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.241+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 >> 172.21.15.66:0/903674181 conn(0x7f36b00fb5f0 msgr2=0x7f36b00fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:25.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.242+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 shutdown_connections 2023-07-21T22:22:25.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:25.242+0000 7f36b77ff700 1 -- 172.21.15.66:0/903674181 wait complete. 2023-07-21T22:22:25.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:24 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2806274822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:22:26.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:25 smithi066 ceph-mon[126379]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:26.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:25 smithi149 ceph-mon[129268]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:26.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:22:26.309 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:22:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:25 smithi104 ceph-mon[129021]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:26.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 -- 172.21.15.66:0/2016568650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40731c0 msgr2=0x7f75a410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/2016568650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40731c0 0x7f75a410bdd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75940098f0 tx=0x7f7594009c00 comp rx=0 tx=0).stop 2023-07-21T22:22:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 -- 172.21.15.66:0/2016568650 shutdown_connections 2023-07-21T22:22:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/2016568650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75a40728b0 0x7f75a4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/2016568650 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/2016568650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40731c0 0x7f75a410bdd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 -- 172.21.15.66:0/2016568650 >> 172.21.15.66:0/2016568650 conn(0x7f75a40fb5f0 msgr2=0x7f75a40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:26.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 -- 172.21.15.66:0/2016568650 shutdown_connections 2023-07-21T22:22:26.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.621+0000 7f75ab9ff700 1 -- 172.21.15.66:0/2016568650 wait complete. 2023-07-21T22:22:26.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 Processor -- start 2023-07-21T22:22:26.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 -- start start 2023-07-21T22:22:26.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40728b0 0x7f75a4197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75a40731c0 0x7f75a4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a419cee0 con 0x7f75a410c310 2023-07-21T22:22:26.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a419d050 con 0x7f75a40731c0 2023-07-21T22:22:26.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.623+0000 7f75ab9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a419d210 con 0x7f75a40728b0 2023-07-21T22:22:26.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.624+0000 7f75a9f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.624+0000 7f75a979b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40728b0 0x7f75a4197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.624+0000 7f75a9f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a419c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55606/0 (socket says 172.21.15.66:55606) 2023-07-21T22:22:26.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.624+0000 7f75a9f9c700 1 -- 172.21.15.66:0/1991032384 learned_addr learned my addr 172.21.15.66:0/1991032384 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:26.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.624+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75a40731c0 0x7f75a4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75a9f9c700 1 -- 172.21.15.66:0/1991032384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40728b0 msgr2=0x7f75a4197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75a9f9c700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40728b0 0x7f75a4197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75a9f9c700 1 -- 172.21.15.66:0/1991032384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75a40731c0 msgr2=0x7f75a4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75a9f9c700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75a40731c0 0x7f75a4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75a9f9c700 1 -- 172.21.15.66:0/1991032384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594009520 con 0x7f75a410c310 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75a9f9c700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a419c8a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f759c00ef30 tx=0x7f759c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:26.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f759a7fc700 1 -- 172.21.15.66:0/1991032384 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c018070 con 0x7f75a410c310 2023-07-21T22:22:26.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f759a7fc700 1 -- 172.21.15.66:0/1991032384 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f759c00f040 con 0x7f75a410c310 2023-07-21T22:22:26.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.625+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a419d490 con 0x7f75a410c310 2023-07-21T22:22:26.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.626+0000 7f759a7fc700 1 -- 172.21.15.66:0/1991032384 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00ccf0 con 0x7f75a410c310 2023-07-21T22:22:26.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.626+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a41a4d30 con 0x7f75a410c310 2023-07-21T22:22:26.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.632+0000 7f759a7fc700 1 -- 172.21.15.66:0/1991032384 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f759c003a00 con 0x7f75a410c310 2023-07-21T22:22:26.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.632+0000 7f75bf2e7b80 1 Processor -- start 2023-07-21T22:22:26.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 -- start start 2023-07-21T22:22:26.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a3f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a3fa890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a2f1d30 con 0x55756a3fadd0 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a344ac0 con 0x55756a3f6340 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a33df50 con 0x55756a3f5a30 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75a979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a3fa890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.633+0000 7f75a979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a3fa890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48194/0 (socket says 172.21.15.66:48194) 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 -- 172.21.15.66:0/1035299083 learned_addr learned my addr 172.21.15.66:0/1035299083 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:26.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 -- 172.21.15.66:0/1035299083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 msgr2=0x55756a3f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a3f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 -- 172.21.15.66:0/1035299083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 msgr2=0x55756a3fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 -- 172.21.15.66:0/1035299083 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55756a16be60 con 0x55756a3f6340 2023-07-21T22:22:26.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.634+0000 7f75a979b700 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a3fa890 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009d60 tx=0x7f75a0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c828b792b1cf386 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:26.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.635+0000 7f758f7fe700 1 -- 172.21.15.66:0/1035299083 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0004a30 con 0x55756a3f6340 2023-07-21T22:22:26.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.635+0000 7f758f7fe700 1 -- 172.21.15.66:0/1035299083 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a00054e0 con 0x55756a3f6340 2023-07-21T22:22:26.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.635+0000 7f758f7fe700 1 -- 172.21.15.66:0/1035299083 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a00197f0 con 0x55756a3f6340 2023-07-21T22:22:26.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.635+0000 7f759a7fc700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f75900a13e0 0x7f75900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.635+0000 7f759a7fc700 1 -- 172.21.15.66:0/1991032384 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f759c0cce60 con 0x7f75a410c310 2023-07-21T22:22:26.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/1035299083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 msgr2=0x55756a3fa890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a3fa890 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009d60 tx=0x7f75a0011720 comp rx=0 tx=0).stop 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/1035299083 shutdown_connections 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a3f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/1035299083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a3fa890 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/1035299083 >> 172.21.15.66:0/1035299083 conn(0x55756a3e1f70 msgr2=0x55756a3e7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f75900a13e0 0x7f75900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/1035299083 shutdown_connections 2023-07-21T22:22:26.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.636+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/1035299083 wait complete. 2023-07-21T22:22:26.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.637+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f75900a13e0 0x7f75900a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f75940098c0 tx=0x7f7594019040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:26.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.637+0000 7f75bf2e7b80 1 -- start start 2023-07-21T22:22:26.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.638+0000 7f75bf2e7b80 1 Processor -- start 2023-07-21T22:22:26.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.638+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a401400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.638+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a3fe680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.638+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a2ddb40 con 0x55756a3fadd0 2023-07-21T22:22:26.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a2dd840 con 0x55756a3f5a30 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a1c19a0 con 0x55756a3f6340 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75bf2e7b80 10 client.0 _ll_register_callbacks cb 0x55756a2db7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a401400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a3fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a3fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49732/0 (socket says 172.21.15.66:49732) 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 -- 172.21.15.66:0/908025088 learned_addr learned my addr 172.21.15.66:0/908025088 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:26.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a9f9c700 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 -- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 msgr2=0x55756a401400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a401400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 -- 172.21.15.66:0/908025088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 msgr2=0x55756a3fed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.639+0000 7f75a8f9a700 1 -- 172.21.15.66:0/908025088 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a00099c0 con 0x55756a3f6340 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.640+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a3fe680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7594016000 tx=0x7f7594009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3347f4f0a55f507 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:26.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.640+0000 7f758cff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.640+0000 7f758cff9700 1 -- 172.21.15.66:0/908025088 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759400b560 con 0x55756a3f6340 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.640+0000 7f758cff9700 1 -- 172.21.15.66:0/908025088 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f759400b6c0 con 0x55756a3f6340 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.640+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55756a297570 con 0x55756a3f6340 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.641+0000 7f758cff9700 1 -- 172.21.15.66:0/908025088 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759400b880 con 0x55756a3f6340 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.641+0000 7f75bf2e7b80 10 client.25783 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.641+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55756a154ce0 con 0x55756a3f6340 2023-07-21T22:22:26.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.641+0000 7f758cff9700 1 -- 172.21.15.66:0/908025088 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7594038020 con 0x55756a3f6340 2023-07-21T22:22:26.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f758cff9700 1 -- 172.21.15.66:0/908025088 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7594003680 con 0x55756a3f6340 2023-07-21T22:22:26.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f75bf2e7b80 10 client.25783 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:26.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f75bf2e7b80 10 client.25783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:26.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f75bf2e7b80 20 client.25783 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:26.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a40ecc0 0x55756a4110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f75bf2e7b80 4 client.25783 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:26.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.642+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55756a336f30 con 0x55756a40ecc0 2023-07-21T22:22:26.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.643+0000 7f75a979b700 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a40ecc0 0x55756a4110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.644+0000 7f75a979b700 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a40ecc0 0x55756a4110d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf6b818ef8630596 server_cookie=dd23b48131b4aa4 in_seq=0 out_seq=0 2023-07-21T22:22:26.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.644+0000 7f758cff9700 10 client.25783 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:26.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.644+0000 7f758cff9700 1 -- 172.21.15.66:0/908025088 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55756a336f30 con 0x55756a40ecc0 2023-07-21T22:22:26.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.644+0000 7f758cff9700 10 client.25783 handle_command_reply: tid=0 2023-07-21T22:22:26.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 client.25783 shutdown 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 msgr2=0x55756a3fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a3fe680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7594016000 tx=0x7f7594009f70 comp rx=0 tx=0).stop 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 shutdown_connections 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a401400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a40ecc0 0x55756a4110d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/908025088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a3fe680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f758cff9700 0 client.25783 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:26.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 >> 172.21.15.66:0/908025088 conn(0x55756a3e1f70 msgr2=0x55756a3e7320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:26.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 shutdown_connections 2023-07-21T22:22:26.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.645+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/908025088 wait complete. 2023-07-21T22:22:26.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.646+0000 7f75bf2e7b80 20 client.25783 trim_cache size 0 max 16384 2023-07-21T22:22:26.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 Processor -- start 2023-07-21T22:22:26.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 -- start start 2023-07-21T22:22:26.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a3fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a4006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a3f8970 con 0x55756a3fadd0 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a3df010 con 0x55756a3f5a30 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.681+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a3ffc60 con 0x55756a3f6340 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a3fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a3fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48218/0 (socket says 172.21.15.66:48218) 2023-07-21T22:22:26.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 -- 172.21.15.66:0/3437990095 learned_addr learned my addr 172.21.15.66:0/3437990095 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:26.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a4006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a9f9c700 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 -- 172.21.15.66:0/3437990095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 msgr2=0x55756a4006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a4006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 -- 172.21.15.66:0/3437990095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 msgr2=0x55756a3fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.682+0000 7f75a979b700 1 -- 172.21.15.66:0/3437990095 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55756a1922d0 con 0x55756a3f5a30 2023-07-21T22:22:26.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.683+0000 7f75a979b700 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a3fc1b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f75a001b040 tx=0x7f75a0005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26c26690aee2d45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:26.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.683+0000 7f758ffff700 1 -- 172.21.15.66:0/3437990095 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a0005bc0 con 0x55756a3f5a30 2023-07-21T22:22:26.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.683+0000 7f758ffff700 1 -- 172.21.15.66:0/3437990095 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a002b040 con 0x55756a3f5a30 2023-07-21T22:22:26.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.683+0000 7f758ffff700 1 -- 172.21.15.66:0/3437990095 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a002ba50 con 0x55756a3f5a30 2023-07-21T22:22:26.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/3437990095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 msgr2=0x55756a3fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a3fc1b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f75a001b040 tx=0x7f75a0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/3437990095 shutdown_connections 2023-07-21T22:22:26.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f5a30 0x55756a3fc1b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a3fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/3437990095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f6340 0x55756a4006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/3437990095 >> 172.21.15.66:0/3437990095 conn(0x55756a3e1f70 msgr2=0x55756a3fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:26.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.684+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/3437990095 shutdown_connections 2023-07-21T22:22:26.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.685+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/3437990095 wait complete. 2023-07-21T22:22:26.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.685+0000 7f75bf2e7b80 1 -- start start 2023-07-21T22:22:26.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.685+0000 7f75bf2e7b80 1 Processor -- start 2023-07-21T22:22:26.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a45d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a45d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a405080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a405690 con 0x55756a3fadd0 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a405800 con 0x55756a3f6340 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756a4059c0 con 0x55756a3f5a30 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.686+0000 7f75bf2e7b80 10 client.0 _ll_register_callbacks cb 0x7f759400f890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:26.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a45d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a45d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49746/0 (socket says 172.21.15.66:49746) 2023-07-21T22:22:26.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a8f9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a45d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 -- 172.21.15.66:0/791442280 learned_addr learned my addr 172.21.15.66:0/791442280 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:26.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a9f9c700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a405080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 -- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 msgr2=0x55756a45d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a45d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 -- 172.21.15.66:0/791442280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 msgr2=0x55756a405080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a405080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 -- 172.21.15.66:0/791442280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55756a16be60 con 0x55756a3f5a30 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a45d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f75a979b700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a45d3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009960 tx=0x7f75a0005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=165dc3b97242d8b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:26.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.687+0000 7f758cff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.688+0000 7f758cff9700 1 -- 172.21.15.66:0/791442280 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a002b1b0 con 0x55756a3f5a30 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.688+0000 7f758cff9700 1 -- 172.21.15.66:0/791442280 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f75a002b310 con 0x55756a3f5a30 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.688+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75a00054e0 con 0x55756a3f5a30 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.688+0000 7f758cff9700 1 -- 172.21.15.66:0/791442280 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75a002b4d0 con 0x55756a3f5a30 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 10 client.25789 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55756a16a330 con 0x55756a3f5a30 2023-07-21T22:22:26.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f758cff9700 1 -- 172.21.15.66:0/791442280 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a0043030 con 0x55756a3f5a30 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f758cff9700 1 -- 172.21.15.66:0/791442280 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f75a0031020 con 0x55756a3f5a30 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 10 client.25789 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 10 client.25789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 20 client.25789 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a464da0 0x55756a467180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 4 client.25789 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:22:26.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.689+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55756a2f90d0 con 0x55756a464da0 2023-07-21T22:22:26.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.690+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a464da0 0x55756a467180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:26.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.691+0000 7f75a8f9a700 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a464da0 0x55756a467180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4aa45b1dae6088e server_cookie=e255308f047cf0a2 in_seq=0 out_seq=0 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.691+0000 7f758cff9700 10 client.25789 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f758cff9700 1 -- 172.21.15.66:0/791442280 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55756a2f90d0 con 0x55756a464da0 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f758cff9700 10 client.25789 handle_command_reply: tid=0 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 client.25789 shutdown 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 msgr2=0x55756a45d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a45d3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009960 tx=0x7f75a0005c50 comp rx=0 tx=0).stop 2023-07-21T22:22:26.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 shutdown_connections 2023-07-21T22:22:26.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55756a464da0 0x55756a467180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55756a3fadd0 0x55756a405080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55756a3f5a30 0x55756a45d3b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f758cff9700 0 client.25789 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:26.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 --2- 172.21.15.66:0/791442280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55756a3f6340 0x55756a45d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 >> 172.21.15.66:0/791442280 conn(0x55756a3e1f70 msgr2=0x55756a3e7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:26.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.692+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 shutdown_connections 2023-07-21T22:22:26.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.693+0000 7f75bf2e7b80 1 -- 172.21.15.66:0/791442280 wait complete. 2023-07-21T22:22:26.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.693+0000 7f75bf2e7b80 20 client.25789 trim_cache size 0 max 16384 2023-07-21T22:22:26.744 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:22:26.744 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:22:26.744 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "e450fe8c-5ea6-4810-b346-46c3be32df2a", 2023-07-21T22:22:26.744 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:22:26.745 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:22:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f75900a13e0 msgr2=0x7f75900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f75900a13e0 0x7f75900a3890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f75940098c0 tx=0x7f7594019040 comp rx=0 tx=0).stop 2023-07-21T22:22:26.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 msgr2=0x7f75a419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a419c8a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f759c00ef30 tx=0x7f759c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 shutdown_connections 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f75900a13e0 0x7f75900a3890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f75a410c310 0x7f75a419c8a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f75a40728b0 0x7f75a4197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 --2- 172.21.15.66:0/1991032384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75a40731c0 0x7f75a4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:26.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 >> 172.21.15.66:0/1991032384 conn(0x7f75a40fb5f0 msgr2=0x7f75a410e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 shutdown_connections 2023-07-21T22:22:26.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:26.694+0000 7f75ab9ff700 1 -- 172.21.15.66:0/1991032384 wait complete. 2023-07-21T22:22:26.767 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:22:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 -- 172.21.15.66:0/1208582756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40fe800 msgr2=0x7f19d40febd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 --2- 172.21.15.66:0/1208582756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40fe800 0x7f19d40febd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009990 tx=0x7f19c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 -- 172.21.15.66:0/1208582756 shutdown_connections 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 --2- 172.21.15.66:0/1208582756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40fe800 0x7f19d40febd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 --2- 172.21.15.66:0/1208582756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d410c3b0 0x7f19d410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 --2- 172.21.15.66:0/1208582756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40ff110 0x7f19d410be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.055+0000 7f19d899d700 1 -- 172.21.15.66:0/1208582756 >> 172.21.15.66:0/1208582756 conn(0x7f19d40745d0 msgr2=0x7f19d40749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.056+0000 7f19d899d700 1 -- 172.21.15.66:0/1208582756 shutdown_connections 2023-07-21T22:22:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.056+0000 7f19d899d700 1 -- 172.21.15.66:0/1208582756 wait complete. 2023-07-21T22:22:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.057+0000 7f19d899d700 1 Processor -- start 2023-07-21T22:22:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.057+0000 7f19d899d700 1 -- start start 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d899d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 0x7f19d419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d899d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40ff110 0x7f19d419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d899d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d410c3b0 0x7f19d4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d899d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4198080 con 0x7f19d40fe800 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d899d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d41981f0 con 0x7f19d40ff110 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d899d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d41983b0 con 0x7f19d410c3b0 2023-07-21T22:22:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 0x7f19d419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 0x7f19d419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55678/0 (socket says 172.21.15.66:55678) 2023-07-21T22:22:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d37fe700 1 -- 172.21.15.66:0/2128448158 learned_addr learned my addr 172.21.15.66:0/2128448158 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40ff110 0x7f19d419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d410c3b0 0x7f19d4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d37fe700 1 -- 172.21.15.66:0/2128448158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d410c3b0 msgr2=0x7f19d4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.058+0000 7f19d37fe700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d410c3b0 0x7f19d4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d37fe700 1 -- 172.21.15.66:0/2128448158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40ff110 msgr2=0x7f19d419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d37fe700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40ff110 0x7f19d419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d37fe700 1 -- 172.21.15.66:0/2128448158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40095f0 con 0x7f19d40fe800 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d37fe700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 0x7f19d419d9a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f19c4005910 tx=0x7f19c4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d0ff9700 1 -- 172.21.15.66:0/2128448158 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c401d070 con 0x7f19d40fe800 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d4198630 con 0x7f19d40fe800 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d0ff9700 1 -- 172.21.15.66:0/2128448158 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c4022470 con 0x7f19d40fe800 2023-07-21T22:22:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.059+0000 7f19d0ff9700 1 -- 172.21.15.66:0/2128448158 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4017650 con 0x7f19d40fe800 2023-07-21T22:22:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.060+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d41a4990 con 0x7f19d40fe800 2023-07-21T22:22:27.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.066+0000 7f19d0ff9700 1 -- 172.21.15.66:0/2128448158 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f19c40225e0 con 0x7f19d40fe800 2023-07-21T22:22:27.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.066+0000 7f19ed4e1b80 1 Processor -- start 2023-07-21T22:22:27.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.066+0000 7f19ed4e1b80 1 -- start start 2023-07-21T22:22:27.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f68700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f6b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80e63cb0 con 0x564f80f68c40 2023-07-21T22:22:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80eb6a40 con 0x564f80f5d490 2023-07-21T22:22:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80eafed0 con 0x564f80f5dda0 2023-07-21T22:22:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f5d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48248/0 (socket says 172.21.15.66:48248) 2023-07-21T22:22:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19d3fff700 1 -- 172.21.15.66:0/1355419809 learned_addr learned my addr 172.21.15.66:0/1355419809 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.067+0000 7f19d37fe700 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f68700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f6b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d3fff700 1 -- 172.21.15.66:0/1355419809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 msgr2=0x564f80f68700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d3fff700 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f68700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d3fff700 1 -- 172.21.15.66:0/1355419809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 msgr2=0x564f80f6b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d3fff700 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f6b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d3fff700 1 -- 172.21.15.66:0/1355419809 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f80cddde0 con 0x564f80f5d490 2023-07-21T22:22:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f6b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:22:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.068+0000 7f19d3fff700 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f5d860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0115b0 tx=0x7f19cc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5680114a8e407 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.069+0000 7f19d0ff9700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19bc0a1580 0x7f19bc0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.069+0000 7f19b9ffb700 1 -- 172.21.15.66:0/1355419809 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00e040 con 0x564f80f5d490 2023-07-21T22:22:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.069+0000 7f19b9ffb700 1 -- 172.21.15.66:0/1355419809 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19cc00f040 con 0x564f80f5d490 2023-07-21T22:22:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.069+0000 7f19b9ffb700 1 -- 172.21.15.66:0/1355419809 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0046f0 con 0x564f80f5d490 2023-07-21T22:22:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.069+0000 7f19d0ff9700 1 -- 172.21.15.66:0/2128448158 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f19c40ce500 con 0x7f19d40fe800 2023-07-21T22:22:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.069+0000 7f19d37fe700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19bc0a1580 0x7f19bc0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/1355419809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 msgr2=0x564f80f5d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f5d860 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0115b0 tx=0x7f19cc009920 comp rx=0 tx=0).stop 2023-07-21T22:22:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19d37fe700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19bc0a1580 0x7f19bc0a3a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19c40cb780 tx=0x7f19c40d1ed0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/1355419809 shutdown_connections 2023-07-21T22:22:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f5d860 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f6b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/1355419809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f68700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/1355419809 >> 172.21.15.66:0/1355419809 conn(0x564f80f53ef0 msgr2=0x564f80f59860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/1355419809 shutdown_connections 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.070+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/1355419809 wait complete. 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.071+0000 7f19ed4e1b80 1 -- start start 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.071+0000 7f19ed4e1b80 1 Processor -- start 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 0x564f80f685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5dda0 0x564f80f658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f68c40 0x564f80f65ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80e4fac0 con 0x564f80f5dda0 2023-07-21T22:22:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80e4f7c0 con 0x564f80f68c40 2023-07-21T22:22:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80d33920 con 0x564f80f5d490 2023-07-21T22:22:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.072+0000 7f19ed4e1b80 10 client.0 _ll_register_callbacks cb 0x564f80e4d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 0x564f80f685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 0x564f80f685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49790/0 (socket says 172.21.15.66:49790) 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 -- 172.21.15.66:0/2528268692 learned_addr learned my addr 172.21.15.66:0/2528268692 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f68c40 0x564f80f65ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5dda0 0x564f80f658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 -- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f68c40 msgr2=0x564f80f65ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f68c40 0x564f80f65ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 -- 172.21.15.66:0/2528268692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5dda0 msgr2=0x564f80f658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5dda0 0x564f80f658c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 -- 172.21.15.66:0/2528268692 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc011290 con 0x564f80f5d490 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5dda0 0x564f80f658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 0x564f80f685d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19cc004310 tx=0x7f19cc009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cf8f28ba415daf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.073+0000 7f19b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.074+0000 7f19b2ffd700 1 -- 172.21.15.66:0/2528268692 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc026070 con 0x564f80f5d490 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.074+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f80e094f0 con 0x564f80f5d490 2023-07-21T22:22:27.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.074+0000 7f19b2ffd700 1 -- 172.21.15.66:0/2528268692 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f19cc003ec0 con 0x564f80f5d490 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.074+0000 7f19b2ffd700 1 -- 172.21.15.66:0/2528268692 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19cc004080 con 0x564f80f5d490 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.074+0000 7f19ed4e1b80 10 client.25801 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.074+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564f80cc6c60 con 0x564f80f5d490 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19b2ffd700 1 -- 172.21.15.66:0/2528268692 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564f80e094f0 con 0x564f80f5d490 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19b2ffd700 1 -- 172.21.15.66:0/2528268692 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19cc015550 con 0x564f80f5d490 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19ed4e1b80 10 client.25801 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:27.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19ed4e1b80 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19ed4e1b80 20 client.25801 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80f89510 0x564f80f8b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19ed4e1b80 4 client.25801 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.075+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f80ea8eb0 con 0x564f80f89510 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.076+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80f89510 0x564f80f8b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.076+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80f89510 0x564f80f8b920 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1aeda845685af4 server_cookie=ac061fb95c03d5c4 in_seq=0 out_seq=0 2023-07-21T22:22:27.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.076+0000 7f19b2ffd700 10 client.25801 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19b2ffd700 1 -- 172.21.15.66:0/2528268692 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f80ea8eb0 con 0x564f80f89510 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19b2ffd700 10 client.25801 handle_command_reply: tid=0 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 client.25801 shutdown 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 msgr2=0x564f80f685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 0x564f80f685d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19cc004310 tx=0x7f19cc009650 comp rx=0 tx=0).stop 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 shutdown_connections 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f68c40 0x564f80f65ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80f89510 0x564f80f8b920 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5d490 0x564f80f685d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19b2ffd700 0 client.25801 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2528268692 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5dda0 0x564f80f658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.077+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 >> 172.21.15.66:0/2528268692 conn(0x564f80f53ef0 msgr2=0x564f80f593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.078+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 shutdown_connections 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.078+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2528268692 wait complete. 2023-07-21T22:22:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.078+0000 7f19ed4e1b80 20 client.25801 trim_cache size 0 max 16384 2023-07-21T22:22:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 Processor -- start 2023-07-21T22:22:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 -- start start 2023-07-21T22:22:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f6f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f5fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f5ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80f53b30 con 0x564f80f68c40 2023-07-21T22:22:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80f668a0 con 0x564f80f5d490 2023-07-21T22:22:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.096+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80f8bf80 con 0x564f80f5dda0 2023-07-21T22:22:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f5fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f5fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49802/0 (socket says 172.21.15.66:49802) 2023-07-21T22:22:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 -- 172.21.15.66:0/2368813154 learned_addr learned my addr 172.21.15.66:0/2368813154 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f6f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d3fff700 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f5ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 -- 172.21.15.66:0/2368813154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 msgr2=0x564f80f6f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f6f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 -- 172.21.15.66:0/2368813154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 msgr2=0x564f80f5ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f5ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 -- 172.21.15.66:0/2368813154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f80d04250 con 0x564f80f5dda0 2023-07-21T22:22:27.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.097+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f5fa50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x564f80f66e00 tx=0x7f19c801cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404c8f914399fb77 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:27.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.098+0000 7f19ba7fc700 1 -- 172.21.15.66:0/2368813154 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8017040 con 0x564f80f5dda0 2023-07-21T22:22:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.098+0000 7f19ba7fc700 1 -- 172.21.15.66:0/2368813154 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19c8020070 con 0x564f80f5dda0 2023-07-21T22:22:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.098+0000 7f19ba7fc700 1 -- 172.21.15.66:0/2368813154 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8005280 con 0x564f80f5dda0 2023-07-21T22:22:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2368813154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 msgr2=0x564f80f5fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f5fa50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x564f80f66e00 tx=0x7f19c801cf00 comp rx=0 tx=0).stop 2023-07-21T22:22:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2368813154 shutdown_connections 2023-07-21T22:22:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5d490 0x564f80f6f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f68c40 0x564f80f5ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/2368813154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f5dda0 0x564f80f5fa50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2368813154 >> 172.21.15.66:0/2368813154 conn(0x564f80f53ef0 msgr2=0x564f80f5f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:27.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2368813154 shutdown_connections 2023-07-21T22:22:27.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/2368813154 wait complete. 2023-07-21T22:22:27.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.099+0000 7f19ed4e1b80 1 -- start start 2023-07-21T22:22:27.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 Processor -- start 2023-07-21T22:22:27.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5d490 0x564f80f6db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5dda0 0x564f80f6e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f68c40 0x564f80f6e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80fe5bb0 con 0x564f80f5d490 2023-07-21T22:22:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80fe5cf0 con 0x564f80f5dda0 2023-07-21T22:22:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f80fe5e30 con 0x564f80f68c40 2023-07-21T22:22:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.100+0000 7f19ed4e1b80 10 client.0 _ll_register_callbacks cb 0x7f19c40d1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:27.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5d490 0x564f80f6db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5d490 0x564f80f6db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55720/0 (socket says 172.21.15.66:55720) 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d3fff700 1 -- 172.21.15.66:0/867337853 learned_addr learned my addr 172.21.15.66:0/867337853 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5dda0 0x564f80f6e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d3fff700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f68c40 0x564f80f6e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d2ffd700 1 -- 172.21.15.66:0/867337853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f68c40 msgr2=0x564f80f6e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f68c40 0x564f80f6e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d2ffd700 1 -- 172.21.15.66:0/867337853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5d490 msgr2=0x564f80f6db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5d490 0x564f80f6db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.101+0000 7f19d2ffd700 1 -- 172.21.15.66:0/867337853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c801c950 con 0x564f80f5dda0 2023-07-21T22:22:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.102+0000 7f19d2ffd700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5dda0 0x564f80f6e060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x564f80f5d460 tx=0x7f19c8009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d9989bcbfddf80f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.102+0000 7f19b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.102+0000 7f19b2ffd700 1 -- 172.21.15.66:0/867337853 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8017040 con 0x564f80f5dda0 2023-07-21T22:22:27.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.102+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19cc00f040 con 0x564f80f5dda0 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.102+0000 7f19b2ffd700 1 -- 172.21.15.66:0/867337853 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f19c8005030 con 0x564f80f5dda0 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.102+0000 7f19b2ffd700 1 -- 172.21.15.66:0/867337853 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19c80051f0 con 0x564f80f5dda0 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 10 client.25829 fetch_fsmap learned FSMap version 23 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564f80cdc2b0 con 0x564f80f5dda0 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19b2ffd700 1 -- 172.21.15.66:0/867337853 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19cc00f040 con 0x564f80f5dda0 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19b2ffd700 1 -- 172.21.15.66:0/867337853 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19c8003680 con 0x564f80f5dda0 2023-07-21T22:22:27.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 10 client.25829 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 10 client.25829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 20 client.25829 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80fe7480 0x564f80fe9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 4 client.25829 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f80e6b050 con 0x564f80fe7480 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.103+0000 7f19d3fff700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80fe7480 0x564f80fe9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:27.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.104+0000 7f19b2ffd700 10 client.25829 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.104+0000 7f19d3fff700 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80fe7480 0x564f80fe9890 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbff3e241497c425 server_cookie=30f62a7c9ee6b302 in_seq=0 out_seq=0 2023-07-21T22:22:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.104+0000 7f19b2ffd700 1 -- 172.21.15.66:0/867337853 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f80e6b050 con 0x564f80fe7480 2023-07-21T22:22:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.104+0000 7f19b2ffd700 10 client.25829 handle_command_reply: tid=0 2023-07-21T22:22:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.104+0000 7f19ed4e1b80 1 client.25829 shutdown 2023-07-21T22:22:27.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5dda0 msgr2=0x564f80f6e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5dda0 0x564f80f6e060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x564f80f5d460 tx=0x7f19c8009ea0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 shutdown_connections 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564f80fe7480 0x564f80fe9890 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564f80f68c40 0x564f80f6e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564f80f5d490 0x564f80f6db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 --2- 172.21.15.66:0/867337853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564f80f5dda0 0x564f80f6e060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 >> 172.21.15.66:0/867337853 conn(0x564f80f53ef0 msgr2=0x564f80f5fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19b2ffd700 0 client.25829 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 shutdown_connections 2023-07-21T22:22:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 1 -- 172.21.15.66:0/867337853 wait complete. 2023-07-21T22:22:27.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.105+0000 7f19ed4e1b80 20 client.25829 trim_cache size 0 max 16384 2023-07-21T22:22:27.161 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:22:27.162 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:22:27.162 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:22:27.162 INFO:teuthology.orchestra.run.smithi066.stdout: "e450fe8c-5ea6-4810-b346-46c3be32df2a": { 2023-07-21T22:22:27.162 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:22:27.162 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "e450fe8c-5ea6-4810-b346-46c3be32df2a", 2023-07-21T22:22:27.162 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:22:27.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:22:27.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:22:27.163 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:22:27.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19bc0a1580 msgr2=0x7f19bc0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19bc0a1580 0x7f19bc0a3a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f19c40cb780 tx=0x7f19c40d1ed0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 msgr2=0x7f19d419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 0x7f19d419d9a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f19c4005910 tx=0x7f19c4004710 comp rx=0 tx=0).stop 2023-07-21T22:22:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 shutdown_connections 2023-07-21T22:22:27.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19bc0a1580 0x7f19bc0a3a30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19d410c3b0 0x7f19d4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19d40fe800 0x7f19d419d9a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 --2- 172.21.15.66:0/2128448158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19d40ff110 0x7f19d419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 >> 172.21.15.66:0/2128448158 conn(0x7f19d40745d0 msgr2=0x7f19d40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.106+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 shutdown_connections 2023-07-21T22:22:27.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:27.107+0000 7f19d899d700 1 -- 172.21.15.66:0/2128448158 wait complete. 2023-07-21T22:22:27.168 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:26 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:22:27.168 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:26 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:22:27.168 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:26 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:22:27.168 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:26 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:22:27.170 INFO:tasks.cephfs.filesystem:scrub status for tag:e450fe8c-5ea6-4810-b346-46c3be32df2a - {'path': '/', 'tag': 'e450fe8c-5ea6-4810-b346-46c3be32df2a', 'options': 'recursive,force'} 2023-07-21T22:22:27.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:26 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:22:27.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:26 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:22:27.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:26 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:22:27.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:26 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:22:27.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:26 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:22:27.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:26 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:22:27.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:26 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:22:27.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:26 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:22:28.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:27 smithi066 ceph-mon[126379]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:22:28.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:27 smithi149 ceph-mon[129268]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:22:28.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:27 smithi104 ceph-mon[129021]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T22:22:30.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:29 smithi066 ceph-mon[126379]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:30.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:29 smithi149 ceph-mon[129268]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:29 smithi104 ceph-mon[129021]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-07-21T22:22:31.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:30 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:22:31.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:30 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:22:31.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:30 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:22:31.440 INFO:tasks.workunit.client.0.smithi066.stderr:Updating files: 62% (7262/11710) Updating files: 63% (7378/11710) Updating files: 64% (7495/11710) Updating files: 65% (7612/11710) Updating files: 66% (7729/11710) Updating files: 67% (7846/11710) Updating files: 68% (7963/11710) Updating files: 69% (8080/11710) Updating files: 70% (8197/11710) Updating files: 71% (8315/11710) Updating files: 72% (8432/11710) Updating files: 73% (8549/11710) Updating files: 74% (8666/11710) Updating files: 75% (8783/11710) Updating files: 76% (8900/11710) Updating files: 77% (9017/11710) Updating files: 78% (9134/11710) Updating files: 79% (9251/11710) Updating files: 80% (9368/11710) Updating files: 81% (9486/11710) Updating files: 82% (9603/11710) Updating files: 83% (9720/11710) Updating files: 84% (9837/11710) Updating files: 85% (9954/11710) Updating files: 86% (10071/11710) Updating files: 87% (10188/11710) Updating files: 88% (10305/11710) Updating files: 89% (10422/11710) Updating files: 90% (10539/11710) Updating files: 91% (10657/11710) Updating files: 92% (10774/11710) Updating files: 93% (10891/11710) Updating files: 94% (11008/11710) Updating files: 95% (11125/11710) Updating files: 96% (11242/11710) Updating files: 97% (11359/11710) Updating files: 97% (11469/11710) Updating files: 98% (11476/11710) Updating files: 99% (11593/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-07-21T22:22:32.180 INFO:tasks.workunit.client.0.smithi066.stderr:Note: switching to 'bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea'. 2023-07-21T22:22:32.180 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.180 INFO:tasks.workunit.client.0.smithi066.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-07-21T22:22:32.180 INFO:tasks.workunit.client.0.smithi066.stderr:changes and commit them, and you can discard any commits you make in this 2023-07-21T22:22:32.180 INFO:tasks.workunit.client.0.smithi066.stderr:state without impacting any branches by switching back to a branch. 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr:If you want to create a new branch to retain commits you create, you may 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr:do so (now or later) by using -c with the switch command. Example: 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr: git switch -c 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.181 INFO:tasks.workunit.client.0.smithi066.stderr:Or undo this operation with: 2023-07-21T22:22:32.182 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.182 INFO:tasks.workunit.client.0.smithi066.stderr: git switch - 2023-07-21T22:22:32.182 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.182 INFO:tasks.workunit.client.0.smithi066.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-07-21T22:22:32.182 INFO:tasks.workunit.client.0.smithi066.stderr: 2023-07-21T22:22:32.182 INFO:tasks.workunit.client.0.smithi066.stderr:HEAD is now at bffe049d41f temp: trigger builds on shaman 2023-07-21T22:22:32.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:31 smithi066 ceph-mon[126379]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:32.197 DEBUG:teuthology.orchestra.run.smithi066:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-07-21T22:22:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:31 smithi149 ceph-mon[129268]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:32.252 INFO:tasks.workunit.client.0.smithi066.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-07-21T22:22:32.258 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T22:22:32.258 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-07-21T22:22:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:31 smithi104 ceph-mon[129021]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2023-07-21T22:22:32.990 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-07-21T22:22:33.086 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-07-21T22:22:33.152 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-07-21T22:22:33.156 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T22:22:33.157 INFO:tasks.workunit.client.0.smithi066.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-07-21T22:22:33.221 INFO:tasks.workunit.client.0.smithi066.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-07-21T22:22:33.228 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:22:33.229 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-07-21T22:22:33.273 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2023-07-21T22:22:33.275 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2023-07-21T22:22:33.275 DEBUG:teuthology.orchestra.run.smithi066:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2023-07-21T22:22:33.373 INFO:tasks.workunit.client.0.smithi066.stderr:+ dbench 1 2023-07-21T22:22:33.404 INFO:tasks.workunit.client.0.smithi066.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-21T22:22:33.404 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-07-21T22:22:33.404 INFO:tasks.workunit.client.0.smithi066.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-21T22:22:33.405 INFO:tasks.workunit.client.0.smithi066.stdout:0 of 1 processes prepared for launch 0 sec 2023-07-21T22:22:33.505 INFO:tasks.workunit.client.0.smithi066.stdout:1 of 1 processes prepared for launch 0 sec 2023-07-21T22:22:33.505 INFO:tasks.workunit.client.0.smithi066.stdout:releasing clients 2023-07-21T22:22:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:33 smithi066 ceph-mon[126379]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:34.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:33 smithi149 ceph-mon[129268]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:34.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:33 smithi104 ceph-mon[129021]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:34.505 INFO:tasks.workunit.client.0.smithi066.stdout: 1 2768 31.97 MB/sec warmup 1 sec latency 6.248 ms 2023-07-21T22:22:35.505 INFO:tasks.workunit.client.0.smithi066.stdout: 1 5759 22.94 MB/sec warmup 2 sec latency 5.795 ms 2023-07-21T22:22:36.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:35 smithi066 ceph-mon[126379]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:35 smithi149 ceph-mon[129268]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:35 smithi104 ceph-mon[129021]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-07-21T22:22:36.505 INFO:tasks.workunit.client.0.smithi066.stdout: 1 8139 20.37 MB/sec warmup 3 sec latency 12.385 ms 2023-07-21T22:22:37.505 INFO:tasks.workunit.client.0.smithi066.stdout: 1 11042 19.58 MB/sec warmup 4 sec latency 18.595 ms 2023-07-21T22:22:38.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:37 smithi066 ceph-mon[126379]: pgmap v37: 129 pgs: 129 active+clean; 3.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 170 KiB/s wr, 28 op/s 2023-07-21T22:22:38.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:38.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:38.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:37 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:38.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:37 smithi149 ceph-mon[129268]: pgmap v37: 129 pgs: 129 active+clean; 3.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 170 KiB/s wr, 28 op/s 2023-07-21T22:22:38.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:38.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:38.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:37 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:37 smithi104 ceph-mon[129021]: pgmap v37: 129 pgs: 129 active+clean; 3.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 170 KiB/s wr, 28 op/s 2023-07-21T22:22:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:22:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:37 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:38.505 INFO:tasks.workunit.client.0.smithi066.stdout: 1 13819 18.30 MB/sec warmup 5 sec latency 11.724 ms 2023-07-21T22:22:39.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:38 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:38 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:39.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:38 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:38 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:38 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:39.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:38 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:38 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:22:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:38 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:22:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:38 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:22:39.505 INFO:tasks.workunit.client.0.smithi066.stdout: 1 16066 17.14 MB/sec warmup 6 sec latency 14.521 ms 2023-07-21T22:22:40.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:39 smithi066 ceph-mon[126379]: pgmap v38: 129 pgs: 129 active+clean; 3.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 170 KiB/s wr, 26 op/s 2023-07-21T22:22:40.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:39 smithi149 ceph-mon[129268]: pgmap v38: 129 pgs: 129 active+clean; 3.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 170 KiB/s wr, 26 op/s 2023-07-21T22:22:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:39 smithi104 ceph-mon[129021]: pgmap v38: 129 pgs: 129 active+clean; 3.0 MiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 170 KiB/s wr, 26 op/s 2023-07-21T22:22:40.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 18566 16.73 MB/sec warmup 7 sec latency 8.313 ms 2023-07-21T22:22:41.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20711 16.14 MB/sec warmup 8 sec latency 297.575 ms 2023-07-21T22:22:42.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:41 smithi066 ceph-mon[126379]: pgmap v39: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 803 KiB/s wr, 106 op/s 2023-07-21T22:22:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:41 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:22:42.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:41 smithi149 ceph-mon[129268]: pgmap v39: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 803 KiB/s wr, 106 op/s 2023-07-21T22:22:42.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:41 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:22:42.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:41 smithi104 ceph-mon[129021]: pgmap v39: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 803 KiB/s wr, 106 op/s 2023-07-21T22:22:42.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:41 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:22:42.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20711 14.35 MB/sec warmup 9 sec latency 1297.647 ms 2023-07-21T22:22:43.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20711 12.91 MB/sec warmup 10 sec latency 2297.774 ms 2023-07-21T22:22:44.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:43 smithi066 ceph-mon[126379]: pgmap v40: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 803 KiB/s wr, 103 op/s 2023-07-21T22:22:44.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:43 smithi149 ceph-mon[129268]: pgmap v40: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 803 KiB/s wr, 103 op/s 2023-07-21T22:22:44.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:43 smithi104 ceph-mon[129021]: pgmap v40: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 803 KiB/s wr, 103 op/s 2023-07-21T22:22:44.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20711 11.74 MB/sec warmup 11 sec latency 3297.939 ms 2023-07-21T22:22:45.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20755 10.76 MB/sec warmup 12 sec latency 4024.562 ms 2023-07-21T22:22:46.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:45 smithi149 ceph-mon[129268]: pgmap v41: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 803 KiB/s wr, 104 op/s 2023-07-21T22:22:46.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:45 smithi104 ceph-mon[129021]: pgmap v41: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 803 KiB/s wr, 104 op/s 2023-07-21T22:22:46.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:45 smithi066 ceph-mon[126379]: pgmap v41: 129 pgs: 129 active+clean; 9.5 MiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 803 KiB/s wr, 104 op/s 2023-07-21T22:22:46.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20755 9.93 MB/sec warmup 13 sec latency 1260.166 ms 2023-07-21T22:22:47.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:46 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:22:47.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:46 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:22:47.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:46 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:22:47.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20755 9.23 MB/sec warmup 14 sec latency 2260.273 ms 2023-07-21T22:22:48.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:47 smithi149 ceph-mon[129268]: pgmap v42: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 984 KiB/s wr, 131 op/s 2023-07-21T22:22:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:47 smithi104 ceph-mon[129021]: pgmap v42: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 984 KiB/s wr, 131 op/s 2023-07-21T22:22:48.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:47 smithi066 ceph-mon[126379]: pgmap v42: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 984 KiB/s wr, 131 op/s 2023-07-21T22:22:48.506 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20755 8.61 MB/sec warmup 15 sec latency 3260.414 ms 2023-07-21T22:22:49.507 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20755 8.07 MB/sec warmup 16 sec latency 4260.552 ms 2023-07-21T22:22:50.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:49 smithi149 ceph-mon[129268]: pgmap v43: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 815 KiB/s wr, 108 op/s 2023-07-21T22:22:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:49 smithi104 ceph-mon[129021]: pgmap v43: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 815 KiB/s wr, 108 op/s 2023-07-21T22:22:50.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:49 smithi066 ceph-mon[126379]: pgmap v43: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 815 KiB/s wr, 108 op/s 2023-07-21T22:22:50.507 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20956 7.63 MB/sec warmup 17 sec latency 4926.818 ms 2023-07-21T22:22:51.507 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20956 7.21 MB/sec warmup 18 sec latency 1248.739 ms 2023-07-21T22:22:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:51 smithi149 ceph-mon[129268]: pgmap v44: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 888 KiB/s wr, 115 op/s 2023-07-21T22:22:52.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:51 smithi104 ceph-mon[129021]: pgmap v44: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 888 KiB/s wr, 115 op/s 2023-07-21T22:22:52.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:51 smithi066 ceph-mon[126379]: pgmap v44: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 888 KiB/s wr, 115 op/s 2023-07-21T22:22:52.507 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20956 6.83 MB/sec warmup 19 sec latency 2248.832 ms 2023-07-21T22:22:53.507 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20956 6.49 MB/sec warmup 20 sec latency 3248.950 ms 2023-07-21T22:22:54.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:53 smithi149 ceph-mon[129268]: pgmap v45: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 255 KiB/s wr, 34 op/s 2023-07-21T22:22:54.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:53 smithi104 ceph-mon[129021]: pgmap v45: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 255 KiB/s wr, 34 op/s 2023-07-21T22:22:54.452 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:53 smithi066 ceph-mon[126379]: pgmap v45: 129 pgs: 129 active+clean; 11 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 255 KiB/s wr, 34 op/s 2023-07-21T22:22:54.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20956 6.18 MB/sec warmup 21 sec latency 4249.736 ms 2023-07-21T22:22:55.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20989 5.91 MB/sec warmup 22 sec latency 4946.626 ms 2023-07-21T22:22:56.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:55 smithi149 ceph-mon[129268]: pgmap v46: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.6 MiB/s wr, 60 op/s 2023-07-21T22:22:56.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:55 smithi104 ceph-mon[129021]: pgmap v46: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.6 MiB/s wr, 60 op/s 2023-07-21T22:22:56.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:55 smithi066 ceph-mon[126379]: pgmap v46: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.6 MiB/s wr, 60 op/s 2023-07-21T22:22:56.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20989 5.65 MB/sec warmup 23 sec latency 1281.584 ms 2023-07-21T22:22:57.171 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:22:57.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1126905789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 msgr2=0x7fb29c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1126905789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c0fff40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009990 tx=0x7fb28c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1126905789 shutdown_connections 2023-07-21T22:22:57.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1126905789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb29c100480 0x7fb29c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1126905789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb29c105ae0 0x7fb29c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1126905789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c0fff40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1126905789 >> 172.21.15.66:0/1126905789 conn(0x7fb29c0fb5f0 msgr2=0x7fb29c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1126905789 shutdown_connections 2023-07-21T22:22:57.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.350+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1126905789 wait complete. 2023-07-21T22:22:57.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.351+0000 7fb2a3fff700 1 Processor -- start 2023-07-21T22:22:57.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.351+0000 7fb2a3fff700 1 -- start start 2023-07-21T22:22:57.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb29c100480 0x7fb29c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb29c105ae0 0x7fb29c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a3fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c19cdf0 con 0x7fb29c0ffad0 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a3fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c19cf60 con 0x7fb29c105ae0 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a3fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c19d120 con 0x7fb29c100480 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41832/0 (socket says 172.21.15.66:41832) 2023-07-21T22:22:57.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1103757570 learned_addr learned my addr 172.21.15.66:0/1103757570 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a159a700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb29c100480 0x7fb29c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a259c700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb29c105ae0 0x7fb29c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1103757570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb29c100480 msgr2=0x7fb29c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb29c100480 0x7fb29c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1103757570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb29c105ae0 msgr2=0x7fb29c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb29c105ae0 0x7fb29c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.352+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1103757570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c0095f0 con 0x7fb29c0ffad0 2023-07-21T22:22:57.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.353+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c197e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e9a0 tx=0x7fb29800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.353+0000 7fb292ffd700 1 -- 172.21.15.66:0/1103757570 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298009790 con 0x7fb29c0ffad0 2023-07-21T22:22:57.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.353+0000 7fb292ffd700 1 -- 172.21.15.66:0/1103757570 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29800cbd0 con 0x7fb29c0ffad0 2023-07-21T22:22:57.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.353+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c19d400 con 0x7fb29c0ffad0 2023-07-21T22:22:57.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.353+0000 7fb292ffd700 1 -- 172.21.15.66:0/1103757570 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298018680 con 0x7fb29c0ffad0 2023-07-21T22:22:57.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.353+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c1a4c40 con 0x7fb29c0ffad0 2023-07-21T22:22:57.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.358+0000 7fb292ffd700 1 -- 172.21.15.66:0/1103757570 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fb29800cd40 con 0x7fb29c0ffad0 2023-07-21T22:22:57.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 Processor -- start 2023-07-21T22:22:57.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 -- start start 2023-07-21T22:22:57.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59e1b0 0x55b25c59e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59eac0 0x55b25c599350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c599da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c49ecb0 con 0x55b25c59eac0 2023-07-21T22:22:57.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c4f1a40 con 0x55b25c599930 2023-07-21T22:22:57.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.359+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c4eaed0 con 0x55b25c59e1b0 2023-07-21T22:22:57.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59eac0 0x55b25c599350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c599da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c599da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42444/0 (socket says 172.21.15.66:42444) 2023-07-21T22:22:57.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59eac0 0x55b25c599350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41842/0 (socket says 172.21.15.66:41842) 2023-07-21T22:22:57.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 -- 172.21.15.66:0/3589253638 learned_addr learned my addr 172.21.15.66:0/3589253638 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a259c700 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59e1b0 0x55b25c59e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb292ffd700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb2880a13e0 0x7fb2880a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb292ffd700 1 -- 172.21.15.66:0/1103757570 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb298014070 con 0x7fb29c0ffad0 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 -- 172.21.15.66:0/3589253638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59e1b0 msgr2=0x55b25c59e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59e1b0 0x55b25c59e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 -- 172.21.15.66:0/3589253638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59eac0 msgr2=0x55b25c599350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59eac0 0x55b25c599350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a259c700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb2880a13e0 0x7fb2880a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.360+0000 7fb2a159a700 1 -- 172.21.15.66:0/3589253638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b25c318de0 con 0x55b25c599930 2023-07-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.361+0000 7fb2a159a700 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c599da0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb28c015000 tx=0x7fb28c005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f834b2bd36b67403 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.361+0000 7fb2a259c700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb2880a13e0 0x7fb2880a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2940078a0 tx=0x7fb294004b40 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.361+0000 7fb287fff700 1 -- 172.21.15.66:0/3589253638 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c01d070 con 0x55b25c599930 2023-07-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.361+0000 7fb287fff700 1 -- 172.21.15.66:0/3589253638 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c01e040 con 0x55b25c599930 2023-07-21T22:22:57.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.361+0000 7fb287fff700 1 -- 172.21.15.66:0/3589253638 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c004880 con 0x55b25c599930 2023-07-21T22:22:57.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3589253638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 msgr2=0x55b25c599da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c599da0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb28c015000 tx=0x7fb28c005ea0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3589253638 shutdown_connections 2023-07-21T22:22:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c599da0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59e1b0 0x55b25c59e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3589253638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59eac0 0x55b25c599350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3589253638 >> 172.21.15.66:0/3589253638 conn(0x55b25c58eef0 msgr2=0x55b25c594830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3589253638 shutdown_connections 2023-07-21T22:22:57.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3589253638 wait complete. 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.362+0000 7fb2b7912b80 1 -- start start 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 Processor -- start 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c48aac0 con 0x55b25c59e1b0 2023-07-21T22:22:57.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c48a7c0 con 0x55b25c59eac0 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c36e920 con 0x55b25c599930 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2b7912b80 10 client.0 _ll_register_callbacks cb 0x55b25c488760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41858/0 (socket says 172.21.15.66:41858) 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.363+0000 7fb2a159a700 1 -- 172.21.15.66:0/101046040 learned_addr learned my addr 172.21.15.66:0/101046040 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:57.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a259c700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 msgr2=0x55b25c5a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/101046040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 msgr2=0x55b25c5a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/101046040 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c10d210 con 0x55b25c599930 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a159a700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:57.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a259c700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:57.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2980ca790 tx=0x7fb2980d0f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4c87c2e198c98b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2857fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:57.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2857fa700 1 -- 172.21.15.66:0/101046040 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29806fea0 con 0x55b25c599930 2023-07-21T22:22:57.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b25c4444f0 con 0x55b25c599930 2023-07-21T22:22:57.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2857fa700 1 -- 172.21.15.66:0/101046040 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb29809a040 con 0x55b25c599930 2023-07-21T22:22:57.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.364+0000 7fb2857fa700 1 -- 172.21.15.66:0/101046040 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2980999a0 con 0x55b25c599930 2023-07-21T22:22:57.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 10 client.25819 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:57.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b25c301c60 con 0x55b25c599930 2023-07-21T22:22:57.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2857fa700 1 -- 172.21.15.66:0/101046040 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b25c4444f0 con 0x55b25c599930 2023-07-21T22:22:57.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2857fa700 1 -- 172.21.15.66:0/101046040 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb2980a2090 con 0x55b25c599930 2023-07-21T22:22:57.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:57.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:57.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 20 client.25819 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:57.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c5b33f0 0x55b25c5b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 4 client.25819 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:57.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b25c4e3eb0 con 0x55b25c5b33f0 2023-07-21T22:22:57.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.365+0000 7fb2a159a700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c5b33f0 0x55b25c5b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.366+0000 7fb2857fa700 10 client.25819 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:57.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.366+0000 7fb2a159a700 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c5b33f0 0x55b25c5b5800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3180c23d329e12b5 server_cookie=39ebb84cdd736bf8 in_seq=0 out_seq=0 2023-07-21T22:22:57.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2857fa700 1 -- 172.21.15.66:0/101046040 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b25c4e3eb0 con 0x55b25c5b33f0 2023-07-21T22:22:57.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2857fa700 10 client.25819 handle_command_reply: tid=0 2023-07-21T22:22:57.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 client.25819 shutdown 2023-07-21T22:22:57.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 msgr2=0x55b25c5a5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5c30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2980ca790 tx=0x7fb2980d0f30 comp rx=0 tx=0).stop 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 shutdown_connections 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c5b33f0 0x55b25c5b5800 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5c30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2857fa700 0 client.25819 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/101046040 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 >> 172.21.15.66:0/101046040 conn(0x55b25c58eef0 msgr2=0x55b25c594420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 shutdown_connections 2023-07-21T22:22:57.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 1 -- 172.21.15.66:0/101046040 wait complete. 2023-07-21T22:22:57.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.367+0000 7fb2b7912b80 20 client.25819 trim_cache size 0 max 16384 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 Processor -- start 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 -- start start 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c59a220 con 0x55b25c59e1b0 2023-07-21T22:22:57.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c591010 con 0x55b25c59eac0 2023-07-21T22:22:57.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c59a4d0 con 0x55b25c599930 2023-07-21T22:22:57.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.385+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41868/0 (socket says 172.21.15.66:41868) 2023-07-21T22:22:57.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 -- 172.21.15.66:0/3839406621 learned_addr learned my addr 172.21.15.66:0/3839406621 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:57.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a259c700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 -- 172.21.15.66:0/3839406621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 msgr2=0x55b25c5a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 -- 172.21.15.66:0/3839406621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 msgr2=0x55b25c5a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 -- 172.21.15.66:0/3839406621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b25c33f250 con 0x55b25c59e1b0 2023-07-21T22:22:57.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a259c700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb2a159a700 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a26e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00bb30 tx=0x7fb28c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24b5d8c1aab7923 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.386+0000 7fb290ff9700 1 -- 172.21.15.66:0/3839406621 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c01d040 con 0x55b25c59e1b0 2023-07-21T22:22:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb290ff9700 1 -- 172.21.15.66:0/3839406621 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c00bbe0 con 0x55b25c59e1b0 2023-07-21T22:22:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb290ff9700 1 -- 172.21.15.66:0/3839406621 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c033be0 con 0x55b25c59e1b0 2023-07-21T22:22:57.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3839406621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 msgr2=0x55b25c5a26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a26e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00bb30 tx=0x7fb28c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3839406621 shutdown_connections 2023-07-21T22:22:57.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c59eac0 0x55b25c5a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c599930 0x55b25c5a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/3839406621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a26e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3839406621 >> 172.21.15.66:0/3839406621 conn(0x55b25c58eef0 msgr2=0x55b25c5979c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3839406621 shutdown_connections 2023-07-21T22:22:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.387+0000 7fb2b7912b80 1 -- 172.21.15.66:0/3839406621 wait complete. 2023-07-21T22:22:57.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.388+0000 7fb2b7912b80 1 -- start start 2023-07-21T22:22:57.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.388+0000 7fb2b7912b80 1 Processor -- start 2023-07-21T22:22:57.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c5a86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59eac0 0x55b25c6002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c6007e0 con 0x55b25c59e1b0 2023-07-21T22:22:57.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c600950 con 0x55b25c599930 2023-07-21T22:22:57.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b25c600b10 con 0x55b25c59eac0 2023-07-21T22:22:57.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2b7912b80 10 client.0 _ll_register_callbacks cb 0x55b25c488760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:57.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c5a86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c5a86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42462/0 (socket says 172.21.15.66:42462) 2023-07-21T22:22:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1288372208 learned_addr learned my addr 172.21.15.66:0/1288372208 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:57.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a259c700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59eac0 0x55b25c6002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1288372208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59eac0 msgr2=0x55b25c6002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59eac0 0x55b25c6002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1288372208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 msgr2=0x55b25c5a8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:57.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a1d9b700 1 -- 172.21.15.66:0/1288372208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b25c301c60 con 0x55b25c599930 2023-07-21T22:22:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.389+0000 7fb2a159a700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2a1d9b700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c5a86f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb298099fd0 tx=0x7fb298099740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc671a8d786787f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2857fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2857fa700 1 -- 172.21.15.66:0/1288372208 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980cd370 con 0x55b25c599930 2023-07-21T22:22:57.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb28c01e040 con 0x55b25c599930 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2857fa700 1 -- 172.21.15.66:0/1288372208 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb29809a040 con 0x55b25c599930 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2857fa700 1 -- 172.21.15.66:0/1288372208 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb29809e070 con 0x55b25c599930 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2b7912b80 10 client.25847 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b25c318de0 con 0x55b25c599930 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2857fa700 1 -- 172.21.15.66:0/1288372208 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c01e040 con 0x55b25c599930 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.390+0000 7fb2857fa700 1 -- 172.21.15.66:0/1288372208 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb2980b2a20 con 0x55b25c599930 2023-07-21T22:22:57.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2b7912b80 10 client.25847 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2b7912b80 10 client.25847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2b7912b80 20 client.25847 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c608050 0x55b25c60a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2b7912b80 4 client.25847 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b25c4a6050 con 0x55b25c608050 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2a159a700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c608050 0x55b25c60a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2857fa700 10 client.25847 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:57.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.391+0000 7fb2a159a700 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c608050 0x55b25c60a460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4a47b1662c40f server_cookie=e429898a35671c10 in_seq=0 out_seq=0 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2857fa700 1 -- 172.21.15.66:0/1288372208 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b25c4a6050 con 0x55b25c608050 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2857fa700 10 client.25847 handle_command_reply: tid=0 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 client.25847 shutdown 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 msgr2=0x55b25c5a86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c5a86f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb298099fd0 tx=0x7fb298099740 comp rx=0 tx=0).stop 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 shutdown_connections 2023-07-21T22:22:57.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25c599930 0x55b25c5a86f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b25c608050 0x55b25c60a460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b25c59eac0 0x55b25c6002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2857fa700 0 client.25847 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 --2- 172.21.15.66:0/1288372208 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b25c59e1b0 0x55b25c5a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 >> 172.21.15.66:0/1288372208 conn(0x55b25c58eef0 msgr2=0x55b25c5a26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 shutdown_connections 2023-07-21T22:22:57.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 1 -- 172.21.15.66:0/1288372208 wait complete. 2023-07-21T22:22:57.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.392+0000 7fb2b7912b80 20 client.25847 trim_cache size 0 max 16384 2023-07-21T22:22:57.452 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:22:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:22:57.452 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:22:57.453 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:22:57.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb2880a13e0 msgr2=0x7fb2880a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb2880a13e0 0x7fb2880a3890 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb2940078a0 tx=0x7fb294004b40 comp rx=0 tx=0).stop 2023-07-21T22:22:57.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 msgr2=0x7fb29c197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c197e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb29800e9a0 tx=0x7fb29800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 shutdown_connections 2023-07-21T22:22:57.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb29c105ae0 0x7fb29c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb2880a13e0 0x7fb2880a3890 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb29c0ffad0 0x7fb29c197e10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 --2- 172.21.15.66:0/1103757570 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb29c100480 0x7fb29c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 >> 172.21.15.66:0/1103757570 conn(0x7fb29c0fb5f0 msgr2=0x7fb29c10ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 shutdown_connections 2023-07-21T22:22:57.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.393+0000 7fb2a3fff700 1 -- 172.21.15.66:0/1103757570 wait complete. 2023-07-21T22:22:57.458 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:22:57.459 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:22:57.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20989 5.41 MB/sec warmup 24 sec latency 2281.650 ms 2023-07-21T22:22:57.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 -- 172.21.15.66:0/959997163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 msgr2=0x7f71bc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 --2- 172.21.15.66:0/959997163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc072c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009990 tx=0x7f71ac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 -- 172.21.15.66:0/959997163 shutdown_connections 2023-07-21T22:22:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 --2- 172.21.15.66:0/959997163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71bc10c310 0x7f71bc10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 --2- 172.21.15.66:0/959997163 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc072c80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 --2- 172.21.15.66:0/959997163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71bc0731c0 0x7f71bc10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.623+0000 7f71c1fff700 1 -- 172.21.15.66:0/959997163 >> 172.21.15.66:0/959997163 conn(0x7f71bc0fb590 msgr2=0x7f71bc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.624+0000 7f71c1fff700 1 -- 172.21.15.66:0/959997163 shutdown_connections 2023-07-21T22:22:57.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.624+0000 7f71c1fff700 1 -- 172.21.15.66:0/959997163 wait complete. 2023-07-21T22:22:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.624+0000 7f71c1fff700 1 Processor -- start 2023-07-21T22:22:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 -- start start 2023-07-21T22:22:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71bc0731c0 0x7f71bc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71bc10c310 0x7f71bc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc19ce20 con 0x7f71bc0728b0 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc19cf90 con 0x7f71bc0731c0 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71c1fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc19d150 con 0x7f71bc10c310 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41900/0 (socket says 172.21.15.66:41900) 2023-07-21T22:22:57.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71bb7fe700 1 -- 172.21.15.66:0/308641051 learned_addr learned my addr 172.21.15.66:0/308641051 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.625+0000 7f71bbfff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71bc10c310 0x7f71bc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71baffd700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71bc0731c0 0x7f71bc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71bb7fe700 1 -- 172.21.15.66:0/308641051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71bc10c310 msgr2=0x7f71bc19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71bb7fe700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71bc10c310 0x7f71bc19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71bb7fe700 1 -- 172.21.15.66:0/308641051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71bc0731c0 msgr2=0x7f71bc198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71bb7fe700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71bc0731c0 0x7f71bc198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71bb7fe700 1 -- 172.21.15.66:0/308641051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0095f0 con 0x7f71bc0728b0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71bb7fe700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc197e40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005910 tx=0x7f71ac004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01d070 con 0x7f71bc0728b0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc19d3d0 con 0x7f71bc0728b0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71ac022470 con 0x7f71bc0728b0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.626+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac017650 con 0x7f71bc0728b0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.627+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc1a4c70 con 0x7f71bc0728b0 2023-07-21T22:22:57.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.627+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc106c30 con 0x7f71bc0728b0 2023-07-21T22:22:57.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.629+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f71ac0225e0 con 0x7f71bc0728b0 2023-07-21T22:22:57.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.631+0000 7f71b8ff9700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71a40a1430 0x7f71a40a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:57.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.631+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f71ac0ce500 con 0x7f71bc0728b0 2023-07-21T22:22:57.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.631+0000 7f71baffd700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71a40a1430 0x7f71a40a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:57.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.632+0000 7f71baffd700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71a40a1430 0x7f71a40a38e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71bc199450 tx=0x7f71b0009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:57.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.635+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f71ac05c770 con 0x7f71bc0728b0 2023-07-21T22:22:57.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.891+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71bc199230 con 0x7f71bc0728b0 2023-07-21T22:22:57.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.892+0000 7f71b8ff9700 1 -- 172.21.15.66:0/308641051 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f71ac097400 con 0x7f71bc0728b0 2023-07-21T22:22:57.936 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:22:57.937 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:22:45.900882+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:22:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71a40a1430 msgr2=0x7f71a40a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71a40a1430 0x7f71a40a38e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f71bc199450 tx=0x7f71b0009260 comp rx=0 tx=0).stop 2023-07-21T22:22:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 msgr2=0x7f71bc197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc197e40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005910 tx=0x7f71ac004710 comp rx=0 tx=0).stop 2023-07-21T22:22:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 shutdown_connections 2023-07-21T22:22:57.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71a40a1430 0x7f71a40a38e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71bc10c310 0x7f71bc19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71bc0728b0 0x7f71bc197e40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 --2- 172.21.15.66:0/308641051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71bc0731c0 0x7f71bc198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 >> 172.21.15.66:0/308641051 conn(0x7f71bc0fb590 msgr2=0x7f71bc103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 shutdown_connections 2023-07-21T22:22:57.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:57.894+0000 7f71c1fff700 1 -- 172.21.15.66:0/308641051 wait complete. 2023-07-21T22:22:57.941 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 25 2023-07-21T22:22:57.957 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:22:58.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 -- 172.21.15.66:0/269822110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5700ffae0 msgr2=0x7fe5700fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 --2- 172.21.15.66:0/269822110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5700ffae0 0x7fe5700fff50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098f0 tx=0x7fe55c009c00 comp rx=0 tx=0).stop 2023-07-21T22:22:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 -- 172.21.15.66:0/269822110 shutdown_connections 2023-07-21T22:22:58.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 --2- 172.21.15.66:0/269822110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe570105af0 0x7fe570105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 --2- 172.21.15.66:0/269822110 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe570100490 0x7fe57010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 --2- 172.21.15.66:0/269822110 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe5700ffae0 0x7fe5700fff50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 -- 172.21.15.66:0/269822110 >> 172.21.15.66:0/269822110 conn(0x7fe5700fb5e0 msgr2=0x7fe5700fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 -- 172.21.15.66:0/269822110 shutdown_connections 2023-07-21T22:22:58.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.124+0000 7fe576dff700 1 -- 172.21.15.66:0/269822110 wait complete. 2023-07-21T22:22:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.126+0000 7fe576dff700 1 Processor -- start 2023-07-21T22:22:58.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.126+0000 7fe576dff700 1 -- start start 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe576dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 0x7fe570197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe576dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe570100490 0x7fe570198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe576dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 0x7fe57019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe576dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57019cec0 con 0x7fe570105af0 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe576dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57019d030 con 0x7fe5700ffae0 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe576dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57019d1f0 con 0x7fe570100490 2023-07-21T22:22:58.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 0x7fe57019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe574b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 0x7fe570197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe574b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 0x7fe570197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42500/0 (socket says 172.21.15.66:42500) 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe574b9b700 1 -- 172.21.15.66:0/3974311607 learned_addr learned my addr 172.21.15.66:0/3974311607 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe56ffff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe570100490 0x7fe570198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 0x7fe57019c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41914/0 (socket says 172.21.15.66:41914) 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 -- 172.21.15.66:0/3974311607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe570100490 msgr2=0x7fe570198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe570100490 0x7fe570198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 -- 172.21.15.66:0/3974311607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 msgr2=0x7fe570197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 0x7fe570197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.127+0000 7fe57539c700 1 -- 172.21.15.66:0/3974311607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009520 con 0x7fe570105af0 2023-07-21T22:22:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe574b9b700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 0x7fe570197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe57539c700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 0x7fe57019c880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe56800efc0 tx=0x7fe56800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe56dffb700 1 -- 172.21.15.66:0/3974311607 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568018070 con 0x7fe570105af0 2023-07-21T22:22:58.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe56dffb700 1 -- 172.21.15.66:0/3974311607 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe56800f040 con 0x7fe570105af0 2023-07-21T22:22:58.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57019d470 con 0x7fe570105af0 2023-07-21T22:22:58.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe56dffb700 1 -- 172.21.15.66:0/3974311607 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56800cd40 con 0x7fe570105af0 2023-07-21T22:22:58.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.128+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5701a4d10 con 0x7fe570105af0 2023-07-21T22:22:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.132+0000 7fe56dffb700 1 -- 172.21.15.66:0/3974311607 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fe568003a00 con 0x7fe570105af0 2023-07-21T22:22:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe58a6c1b80 1 Processor -- start 2023-07-21T22:22:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe58a6c1b80 1 -- start start 2023-07-21T22:22:58.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe56dffb700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe5600a1460 0x7fe5600a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe56dffb700 1 -- 172.21.15.66:0/3974311607 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe5680cce50 con 0x7fe570105af0 2023-07-21T22:22:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.133+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8dcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c7e1cb0 con 0x55957c8e1ac0 2023-07-21T22:22:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c834a40 con 0x55957c8dc930 2023-07-21T22:22:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c82ded0 con 0x55957c8e11b0 2023-07-21T22:22:58.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe56ffff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe5600a1460 0x7fe5600a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43494/0 (socket says 172.21.15.66:43494) 2023-07-21T22:22:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 -- 172.21.15.66:0/3342708986 learned_addr learned my addr 172.21.15.66:0/3342708986 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe57539c700 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8dcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 -- 172.21.15.66:0/3342708986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 msgr2=0x55957c8dcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8dcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 -- 172.21.15.66:0/3342708986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 msgr2=0x55957c8dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8dc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.134+0000 7fe574b9b700 1 -- 172.21.15.66:0/3342708986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957c65bde0 con 0x55957c8e11b0 2023-07-21T22:22:58.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.135+0000 7fe574b9b700 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e1580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5640115f0 tx=0x7fe564011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f75bd79f7fe6c278 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.135+0000 7fe56ffff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe5600a1460 0x7fe5600a3910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005e90 tx=0x7fe55c010c20 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.135+0000 7fe55affd700 1 -- 172.21.15.66:0/3342708986 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564019630 con 0x55957c8e11b0 2023-07-21T22:22:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.135+0000 7fe55affd700 1 -- 172.21.15.66:0/3342708986 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe564019c50 con 0x55957c8e11b0 2023-07-21T22:22:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.135+0000 7fe55affd700 1 -- 172.21.15.66:0/3342708986 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56400eb60 con 0x55957c8e11b0 2023-07-21T22:22:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3342708986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 msgr2=0x55957c8e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e1580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5640115f0 tx=0x7fe564011900 comp rx=0 tx=0).stop 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3342708986 shutdown_connections 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8dcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e1580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3342708986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3342708986 >> 172.21.15.66:0/3342708986 conn(0x55957c8d1ef0 msgr2=0x55957c8d7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3342708986 shutdown_connections 2023-07-21T22:22:58.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3342708986 wait complete. 2023-07-21T22:22:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.136+0000 7fe58a6c1b80 1 -- start start 2023-07-21T22:22:58.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.137+0000 7fe58a6c1b80 1 Processor -- start 2023-07-21T22:22:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.137+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8e8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c7cdac0 con 0x55957c8e1ac0 2023-07-21T22:22:58.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c7cd7c0 con 0x55957c8dc930 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c6b1920 con 0x55957c8e11b0 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe58a6c1b80 10 client.0 _ll_register_callbacks cb 0x55957c7cb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe57539c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43498/0 (socket says 172.21.15.66:43498) 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 -- 172.21.15.66:0/1749575534 learned_addr learned my addr 172.21.15.66:0/1749575534 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe574b9b700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8e8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 -- 172.21.15.66:0/1749575534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 msgr2=0x55957c8e8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8e8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 -- 172.21.15.66:0/1749575534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 msgr2=0x55957c8e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe56ffff700 1 -- 172.21.15.66:0/1749575534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe564009dd0 con 0x55957c8e11b0 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.138+0000 7fe57539c700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe56ffff700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e5e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c010 tx=0x7fe55c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a76a025e23dd8216 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe54ffff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:22:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe54ffff700 1 -- 172.21.15.66:0/1749575534 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00bc10 con 0x55957c8e11b0 2023-07-21T22:22:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55957c7874f0 con 0x55957c8e11b0 2023-07-21T22:22:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe54ffff700 1 -- 172.21.15.66:0/1749575534 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe55c0050c0 con 0x55957c8e11b0 2023-07-21T22:22:58.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe54ffff700 1 -- 172.21.15.66:0/1749575534 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe55c005280 con 0x55957c8e11b0 2023-07-21T22:22:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe58a6c1b80 10 client.25843 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55957c644c60 con 0x55957c8e11b0 2023-07-21T22:22:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.139+0000 7fe54ffff700 1 -- 172.21.15.66:0/1749575534 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55957c7874f0 con 0x55957c8e11b0 2023-07-21T22:22:58.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe54ffff700 1 -- 172.21.15.66:0/1749575534 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe55c028470 con 0x55957c8e11b0 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe58a6c1b80 10 client.25843 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe58a6c1b80 10 client.25843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe58a6c1b80 20 client.25843 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c8f6440 0x55957c8f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe58a6c1b80 4 client.25843 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55957c826eb0 con 0x55957c8f6440 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.140+0000 7fe574b9b700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c8f6440 0x55957c8f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.141+0000 7fe54ffff700 10 client.25843 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.141+0000 7fe574b9b700 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c8f6440 0x55957c8f8850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=856a5128f845c53f server_cookie=7b5f54e57201c1a2 in_seq=0 out_seq=0 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.141+0000 7fe54ffff700 1 -- 172.21.15.66:0/1749575534 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55957c826eb0 con 0x55957c8f6440 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.141+0000 7fe54ffff700 10 client.25843 handle_command_reply: tid=0 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 client.25843 shutdown 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 msgr2=0x55957c8e5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e5e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c010 tx=0x7fe55c005dc0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 shutdown_connections 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c8f6440 0x55957c8f8850 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c8e8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e1ac0 0x55957c8e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe54ffff700 0 client.25843 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/1749575534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e11b0 0x55957c8e5e70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 >> 172.21.15.66:0/1749575534 conn(0x55957c8d1ef0 msgr2=0x55957c8d73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 shutdown_connections 2023-07-21T22:22:58.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/1749575534 wait complete. 2023-07-21T22:22:58.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.142+0000 7fe58a6c1b80 20 client.25843 trim_cache size 0 max 16384 2023-07-21T22:22:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 Processor -- start 2023-07-21T22:22:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 -- start start 2023-07-21T22:22:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8dc930 0x55957c8ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c8f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8e1ac0 0x55957c8dab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c8e3ea0 con 0x55957c8e11b0 2023-07-21T22:22:58.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c8dd6d0 con 0x55957c8e1ac0 2023-07-21T22:22:58.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.156+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c8d4010 con 0x55957c8dc930 2023-07-21T22:22:58.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c8f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c8f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41942/0 (socket says 172.21.15.66:41942) 2023-07-21T22:22:58.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 -- 172.21.15.66:0/3256796584 learned_addr learned my addr 172.21.15.66:0/3256796584 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe574b9b700 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8dc930 0x55957c8ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe57539c700 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8e1ac0 0x55957c8dab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 -- 172.21.15.66:0/3256796584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8dc930 msgr2=0x55957c8ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8dc930 0x55957c8ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 -- 172.21.15.66:0/3256796584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8e1ac0 msgr2=0x55957c8dab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8e1ac0 0x55957c8dab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 -- 172.21.15.66:0/3256796584 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55957c682250 con 0x55957c8e11b0 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe56ffff700 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c8f1d20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009fd0 tx=0x7fe55c00b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcd782e6de756c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe55b7fe700 1 -- 172.21.15.66:0/3256796584 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00f530 con 0x55957c8e11b0 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe55b7fe700 1 -- 172.21.15.66:0/3256796584 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe55c028af0 con 0x55957c8e11b0 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.157+0000 7fe55b7fe700 1 -- 172.21.15.66:0/3256796584 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0038e0 con 0x55957c8e11b0 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3256796584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 msgr2=0x55957c8f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c8f1d20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009fd0 tx=0x7fe55c00b2e0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3256796584 shutdown_connections 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8e1ac0 0x55957c8dab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8dc930 0x55957c8ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/3256796584 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c8f1d20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3256796584 >> 172.21.15.66:0/3256796584 conn(0x55957c8d1ef0 msgr2=0x55957c8f2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3256796584 shutdown_connections 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.158+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/3256796584 wait complete. 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 -- start start 2023-07-21T22:22:58.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 Processor -- start 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c93e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c93e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e1ac0 0x55957c949310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c93f0d0 con 0x55957c8e11b0 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c949850 con 0x55957c8dc930 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55957c9499c0 con 0x55957c8e1ac0 2023-07-21T22:22:58.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe58a6c1b80 10 client.0 _ll_register_callbacks cb 0x55957c7cb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.159+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c93e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c93e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41946/0 (socket says 172.21.15.66:41946) 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe57539c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e1ac0 0x55957c949310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 -- 172.21.15.66:0/331962982 learned_addr learned my addr 172.21.15.66:0/331962982 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe574b9b700 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c93e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 -- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e1ac0 msgr2=0x55957c949310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e1ac0 0x55957c949310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 -- 172.21.15.66:0/331962982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 msgr2=0x55957c93e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c93e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 -- 172.21.15.66:0/331962982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55957c644c60 con 0x55957c8e11b0 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe56ffff700 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c93e9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005e00 tx=0x7fe55c0286a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712ee95d38fbc839 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe54ffff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe54ffff700 1 -- 172.21.15.66:0/331962982 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c005b00 con 0x55957c8e11b0 2023-07-21T22:22:58.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe564019c50 con 0x55957c8e11b0 2023-07-21T22:22:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.160+0000 7fe54ffff700 1 -- 172.21.15.66:0/331962982 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe55c0052f0 con 0x55957c8e11b0 2023-07-21T22:22:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe54ffff700 1 -- 172.21.15.66:0/331962982 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe55c0054b0 con 0x55957c8e11b0 2023-07-21T22:22:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 10 client.15822 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:58.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55957c65a2b0 con 0x55957c8e11b0 2023-07-21T22:22:58.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe54ffff700 1 -- 172.21.15.66:0/331962982 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe564019c50 con 0x55957c8e11b0 2023-07-21T22:22:58.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe54ffff700 1 -- 172.21.15.66:0/331962982 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe55c038070 con 0x55957c8e11b0 2023-07-21T22:22:58.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:58.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 10 client.15822 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 20 client.15822 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c94ac60 0x55957c94d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 4 client.15822 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.161+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55957c7e9050 con 0x55957c94ac60 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.162+0000 7fe574b9b700 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c94ac60 0x55957c94d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.162+0000 7fe54ffff700 10 client.15822 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:58.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.162+0000 7fe574b9b700 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c94ac60 0x55957c94d040 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a98d1c8c38689fa server_cookie=2feb6139cccb7d38 in_seq=0 out_seq=0 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.162+0000 7fe54ffff700 1 -- 172.21.15.66:0/331962982 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55957c7e9050 con 0x55957c94ac60 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.162+0000 7fe54ffff700 10 client.15822 handle_command_reply: tid=0 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.162+0000 7fe58a6c1b80 1 client.15822 shutdown 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 msgr2=0x55957c93e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c93e9c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005e00 tx=0x7fe55c0286a0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 shutdown_connections 2023-07-21T22:22:58.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55957c94ac60 0x55957c94d040 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55957c8dc930 0x55957c93e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55957c8e1ac0 0x55957c949310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe54ffff700 0 client.15822 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 --2- 172.21.15.66:0/331962982 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55957c8e11b0 0x55957c93e9c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 >> 172.21.15.66:0/331962982 conn(0x55957c8d1ef0 msgr2=0x55957c8dab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 shutdown_connections 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 1 -- 172.21.15.66:0/331962982 wait complete. 2023-07-21T22:22:58.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe58a6c1b80 20 client.15822 trim_cache size 0 max 16384 2023-07-21T22:22:58.218 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:22:58.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe5600a1460 msgr2=0x7fe5600a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe576dff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe5600a1460 0x7fe5600a3910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005e90 tx=0x7fe55c010c20 comp rx=0 tx=0).stop 2023-07-21T22:22:58.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 msgr2=0x7fe57019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.163+0000 7fe576dff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 0x7fe57019c880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe56800efc0 tx=0x7fe56800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 shutdown_connections 2023-07-21T22:22:58.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5700ffae0 0x7fe570197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe5600a1460 0x7fe5600a3910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe570105af0 0x7fe57019c880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 --2- 172.21.15.66:0/3974311607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe570100490 0x7fe570198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 >> 172.21.15.66:0/3974311607 conn(0x7fe5700fb5e0 msgr2=0x7fe57010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 shutdown_connections 2023-07-21T22:22:58.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.164+0000 7fe576dff700 1 -- 172.21.15.66:0/3974311607 wait complete. 2023-07-21T22:22:58.224 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:57 smithi066 ceph-mon[126379]: pgmap v47: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.6 MiB/s wr, 60 op/s 2023-07-21T22:22:58.224 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:57 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/308641051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:22:58.226 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:22:58.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:57 smithi149 ceph-mon[129268]: pgmap v47: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.6 MiB/s wr, 60 op/s 2023-07-21T22:22:58.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:57 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/308641051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:22:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:57 smithi104 ceph-mon[129021]: pgmap v47: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 2.6 MiB/s wr, 60 op/s 2023-07-21T22:22:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:57 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/308641051' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:22:58.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 -- 172.21.15.66:0/1582054028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf34105a00 msgr2=0x7fdf34105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1582054028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf34105a00 0x7fdf34105dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098f0 tx=0x7fdf28009c00 comp rx=0 tx=0).stop 2023-07-21T22:22:58.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 -- 172.21.15.66:0/1582054028 shutdown_connections 2023-07-21T22:22:58.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1582054028 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf341003a0 0x7fdf3410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1582054028 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf34105a00 0x7fdf34105dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1582054028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf340ff9f0 0x7fdf340ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 -- 172.21.15.66:0/1582054028 >> 172.21.15.66:0/1582054028 conn(0x7fdf340fb510 msgr2=0x7fdf340fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 -- 172.21.15.66:0/1582054028 shutdown_connections 2023-07-21T22:22:58.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.392+0000 7fdf3999d700 1 -- 172.21.15.66:0/1582054028 wait complete. 2023-07-21T22:22:58.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.393+0000 7fdf3999d700 1 Processor -- start 2023-07-21T22:22:58.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.393+0000 7fdf3999d700 1 -- start start 2023-07-21T22:22:58.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.393+0000 7fdf3999d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf340ff9f0 0x7fdf3419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.393+0000 7fdf3999d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf341003a0 0x7fdf3419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3999d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 0x7fdf34197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3999d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf341980c0 con 0x7fdf34105a00 2023-07-21T22:22:58.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3999d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf34198230 con 0x7fdf340ff9f0 2023-07-21T22:22:58.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3999d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf341983f0 con 0x7fdf341003a0 2023-07-21T22:22:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 0x7fdf34197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf341003a0 0x7fdf3419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 0x7fdf34197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41962/0 (socket says 172.21.15.66:41962) 2023-07-21T22:22:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 -- 172.21.15.66:0/1797228 learned_addr learned my addr 172.21.15.66:0/1797228 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf341003a0 0x7fdf3419df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43538/0 (socket says 172.21.15.66:43538) 2023-07-21T22:22:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3899b700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf340ff9f0 0x7fdf3419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 -- 172.21.15.66:0/1797228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf341003a0 msgr2=0x7fdf3419df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf341003a0 0x7fdf3419df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 -- 172.21.15.66:0/1797228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf340ff9f0 msgr2=0x7fdf3419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf340ff9f0 0x7fdf3419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.394+0000 7fdf3919c700 1 -- 172.21.15.66:0/1797228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009520 con 0x7fdf34105a00 2023-07-21T22:22:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.395+0000 7fdf3919c700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 0x7fdf34197a50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c110 tx=0x7fdf2c00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.395+0000 7fdf31ffb700 1 -- 172.21.15.66:0/1797228 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c017070 con 0x7fdf34105a00 2023-07-21T22:22:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.395+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf34198670 con 0x7fdf34105a00 2023-07-21T22:22:58.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.395+0000 7fdf31ffb700 1 -- 172.21.15.66:0/1797228 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf2c00e050 con 0x7fdf34105a00 2023-07-21T22:22:58.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.395+0000 7fdf31ffb700 1 -- 172.21.15.66:0/1797228 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0078f0 con 0x7fdf34105a00 2023-07-21T22:22:58.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.395+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf341a49d0 con 0x7fdf34105a00 2023-07-21T22:22:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.398+0000 7fdf31ffb700 1 -- 172.21.15.66:0/1797228 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fdf2c007a50 con 0x7fdf34105a00 2023-07-21T22:22:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.399+0000 7fdf4e580b80 1 Processor -- start 2023-07-21T22:22:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 -- start start 2023-07-21T22:22:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716086f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x55716087d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557160775cb0 con 0x55716086f490 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571607c8a40 con 0x55716087ac40 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571607c1ed0 con 0x55716086fda0 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf31ffb700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdf240a1480 0x7fdf240a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf31ffb700 1 -- 172.21.15.66:0/1797228 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdf2c0ccd10 con 0x7fdf34105a00 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716086f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf3899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716086f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41964/0 (socket says 172.21.15.66:41964) 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf33fff700 1 -- 172.21.15.66:0/3561041393 learned_addr learned my addr 172.21.15.66:0/3561041393 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.400+0000 7fdf3919c700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdf240a1480 0x7fdf240a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf33fff700 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x55716087d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3899b700 1 -- 172.21.15.66:0/3561041393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 msgr2=0x55716087d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3899b700 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x55716087d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3899b700 1 -- 172.21.15.66:0/3561041393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 msgr2=0x55716086f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3899b700 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716086f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3899b700 1 -- 172.21.15.66:0/3561041393 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571605efde0 con 0x55716086fda0 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3899b700 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087a700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009230 tx=0x7fdf28005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e71b58f9d03e05 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf3919c700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdf240a1480 0x7fdf240a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0c9f90 tx=0x7fdf2c006240 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf1effd700 1 -- 172.21.15.66:0/3561041393 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800f720 con 0x55716086fda0 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf1effd700 1 -- 172.21.15.66:0/3561041393 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf2801d040 con 0x55716086fda0 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.401+0000 7fdf1effd700 1 -- 172.21.15.66:0/3561041393 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28005400 con 0x55716086fda0 2023-07-21T22:22:58.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 -- 172.21.15.66:0/3561041393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 msgr2=0x55716087a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087a700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009230 tx=0x7fdf28005dc0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 -- 172.21.15.66:0/3561041393 shutdown_connections 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x55716087d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716086f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/3561041393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087a700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 -- 172.21.15.66:0/3561041393 >> 172.21.15.66:0/3561041393 conn(0x557160865ef0 msgr2=0x55716086b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 -- 172.21.15.66:0/3561041393 shutdown_connections 2023-07-21T22:22:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.402+0000 7fdf4e580b80 1 -- 172.21.15.66:0/3561041393 wait complete. 2023-07-21T22:22:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.403+0000 7fdf4e580b80 1 -- start start 2023-07-21T22:22:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.403+0000 7fdf4e580b80 1 Processor -- start 2023-07-21T22:22:58.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x55716087a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086fda0 0x5571608778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x557160877fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557160761ac0 con 0x55716086fda0 2023-07-21T22:22:58.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571607617c0 con 0x55716087ac40 2023-07-21T22:22:58.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557160645920 con 0x55716086f490 2023-07-21T22:22:58.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf4e580b80 10 client.0 _ll_register_callbacks cb 0x55716075f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:58.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf3899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086fda0 0x5571608778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x55716087a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x55716087a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43552/0 (socket says 172.21.15.66:43552) 2023-07-21T22:22:58.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf33fff700 1 -- 172.21.15.66:0/1999995379 learned_addr learned my addr 172.21.15.66:0/1999995379 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf33fff700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x557160877fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf3899b700 1 -- 172.21.15.66:0/1999995379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 msgr2=0x55716087a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf3899b700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x55716087a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf3899b700 1 -- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 msgr2=0x557160877fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf3899b700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x557160877fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf3899b700 1 -- 172.21.15.66:0/1999995379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3410d130 con 0x55716086fda0 2023-07-21T22:22:58.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.404+0000 7fdf33fff700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x55716087a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf3899b700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086fda0 0x5571608778c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005e90 tx=0x7fdf2800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1426acd59dd1cfb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf13fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf13fff700 1 -- 172.21.15.66:0/1999995379 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800bb60 con 0x55716086fda0 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55716071b4f0 con 0x55716086fda0 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf13fff700 1 -- 172.21.15.66:0/1999995379 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdf28005480 con 0x55716086fda0 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf13fff700 1 -- 172.21.15.66:0/1999995379 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf28005920 con 0x55716086fda0 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf4e580b80 10 client.15834 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.405+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5571605d8c60 con 0x55716086fda0 2023-07-21T22:22:58.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf13fff700 1 -- 172.21.15.66:0/1999995379 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf28033080 con 0x55716086fda0 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf13fff700 1 -- 172.21.15.66:0/1999995379 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf28015020 con 0x55716086fda0 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf4e580b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf4e580b80 10 client.15834 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf4e580b80 20 client.15834 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55716088a130 0x55716088c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf4e580b80 4 client.15834 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:58.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571607baeb0 con 0x55716088a130 2023-07-21T22:22:58.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.406+0000 7fdf33fff700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55716088a130 0x55716088c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.407+0000 7fdf33fff700 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55716088a130 0x55716088c510 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fe50cb14db5d85c server_cookie=2d79f8518f542e01 in_seq=0 out_seq=0 2023-07-21T22:22:58.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.407+0000 7fdf13fff700 10 client.15834 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:58.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf13fff700 1 -- 172.21.15.66:0/1999995379 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571607baeb0 con 0x55716088a130 2023-07-21T22:22:58.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf13fff700 10 client.15834 handle_command_reply: tid=0 2023-07-21T22:22:58.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 client.15834 shutdown 2023-07-21T22:22:58.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086fda0 msgr2=0x5571608778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086fda0 0x5571608778c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005e90 tx=0x7fdf2800b8c0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 shutdown_connections 2023-07-21T22:22:58.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55716088a130 0x55716088c510 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x557160877fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x55716087a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf13fff700 0 client.15834 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:58.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/1999995379 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086fda0 0x5571608778c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 >> 172.21.15.66:0/1999995379 conn(0x557160865ef0 msgr2=0x55716086b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 shutdown_connections 2023-07-21T22:22:58.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.408+0000 7fdf4e580b80 1 -- 172.21.15.66:0/1999995379 wait complete. 2023-07-21T22:22:58.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.409+0000 7fdf4e580b80 20 client.15834 trim_cache size 0 max 16384 2023-07-21T22:22:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 Processor -- start 2023-07-21T22:22:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 -- start start 2023-07-21T22:22:58.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x557160872350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716086fda0 0x55716087a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 0x557160872ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571608705f0 con 0x55716087ac40 2023-07-21T22:22:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557160877f70 con 0x55716086fda0 2023-07-21T22:22:58.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.423+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557160876e30 con 0x55716086f490 2023-07-21T22:22:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 0x557160872ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 0x557160872ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41982/0 (socket says 172.21.15.66:41982) 2023-07-21T22:22:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf33fff700 1 -- 172.21.15.66:0/486752838 learned_addr learned my addr 172.21.15.66:0/486752838 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf33fff700 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x557160872350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716086fda0 0x55716087a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 -- 172.21.15.66:0/486752838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 msgr2=0x557160872350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x557160872350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 -- 172.21.15.66:0/486752838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 msgr2=0x557160872ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 0x557160872ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 -- 172.21.15.66:0/486752838 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557160616250 con 0x55716086fda0 2023-07-21T22:22:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf33fff700 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 0x557160872ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.424+0000 7fdf3899b700 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716086fda0 0x55716087a1e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf28021500 tx=0x7fdf28005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b3ce042a69643e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf1f7fe700 1 -- 172.21.15.66:0/486752838 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28005920 con 0x55716086fda0 2023-07-21T22:22:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf1f7fe700 1 -- 172.21.15.66:0/486752838 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55716071b4f0 con 0x55716086fda0 2023-07-21T22:22:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf1f7fe700 1 -- 172.21.15.66:0/486752838 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28005280 con 0x55716086fda0 2023-07-21T22:22:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 -- 172.21.15.66:0/486752838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716086fda0 msgr2=0x55716087a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716086fda0 0x55716087a1e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdf28021500 tx=0x7fdf28005dc0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 -- 172.21.15.66:0/486752838 shutdown_connections 2023-07-21T22:22:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716087ac40 0x557160872ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086f490 0x557160872350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/486752838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716086fda0 0x55716087a1e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 -- 172.21.15.66:0/486752838 >> 172.21.15.66:0/486752838 conn(0x557160865ef0 msgr2=0x557160869f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 -- 172.21.15.66:0/486752838 shutdown_connections 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.425+0000 7fdf4e580b80 1 -- 172.21.15.66:0/486752838 wait complete. 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.426+0000 7fdf4e580b80 1 -- start start 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.426+0000 7fdf4e580b80 1 Processor -- start 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716087f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x5571608d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571608d7460 con 0x55716086f490 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571608d75a0 con 0x55716087ac40 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571608d7760 con 0x55716086fda0 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf4e580b80 10 client.0 _ll_register_callbacks cb 0x7fdf200098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716087f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf3899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf33fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716087f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41988/0 (socket says 172.21.15.66:41988) 2023-07-21T22:22:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.427+0000 7fdf33fff700 1 -- 172.21.15.66:0/875877630 learned_addr learned my addr 172.21.15.66:0/875877630 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x5571608d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 -- 172.21.15.66:0/875877630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 msgr2=0x55716087fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 -- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 msgr2=0x5571608d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x5571608d6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 -- 172.21.15.66:0/875877630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2002b040 con 0x55716086f490 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf3899b700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf33fff700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716087f570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55716088cd80 tx=0x7fdf20009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed0923ec32408aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf13fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf13fff700 1 -- 172.21.15.66:0/875877630 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf200120d0 con 0x55716086f490 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf2801d040 con 0x55716086f490 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf13fff700 1 -- 172.21.15.66:0/875877630 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdf20017040 con 0x55716086f490 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf13fff700 1 -- 172.21.15.66:0/875877630 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf20012230 con 0x55716086f490 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.428+0000 7fdf4e580b80 10 client.15840 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5571605d8c60 con 0x55716086f490 2023-07-21T22:22:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf13fff700 1 -- 172.21.15.66:0/875877630 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf2801d040 con 0x55716086f490 2023-07-21T22:22:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf13fff700 1 -- 172.21.15.66:0/875877630 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdf2002f070 con 0x55716086f490 2023-07-21T22:22:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 10 client.15840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:22:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 20 client.15840 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5571608def50 0x5571608e1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 4 client.15840 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:22:58.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55716077d050 con 0x5571608def50 2023-07-21T22:22:58.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.429+0000 7fdf3899b700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5571608def50 0x5571608e1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf3899b700 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5571608def50 0x5571608e1360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44e759fb4decc0d4 server_cookie=12780933c36899a1 in_seq=0 out_seq=0 2023-07-21T22:22:58.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf13fff700 10 client.15840 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:58.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf13fff700 1 -- 172.21.15.66:0/875877630 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55716077d050 con 0x5571608def50 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf13fff700 10 client.15840 handle_command_reply: tid=0 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 client.15840 shutdown 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 msgr2=0x55716087f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716087f570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55716088cd80 tx=0x7fdf20009260 comp rx=0 tx=0).stop 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 shutdown_connections 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5571608def50 0x5571608e1360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55716087ac40 0x5571608d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55716086f490 0x55716087f570 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 --2- 172.21.15.66:0/875877630 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55716086fda0 0x55716087fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 >> 172.21.15.66:0/875877630 conn(0x557160865ef0 msgr2=0x557160872ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.430+0000 7fdf13fff700 0 client.15840 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:22:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 shutdown_connections 2023-07-21T22:22:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf4e580b80 1 -- 172.21.15.66:0/875877630 wait complete. 2023-07-21T22:22:58.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf4e580b80 20 client.15840 trim_cache size 0 max 16384 2023-07-21T22:22:58.491 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:22:58.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdf240a1480 msgr2=0x7fdf240a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdf240a1480 0x7fdf240a3930 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0c9f90 tx=0x7fdf2c006240 comp rx=0 tx=0).stop 2023-07-21T22:22:58.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 msgr2=0x7fdf34197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 0x7fdf34197a50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c110 tx=0x7fdf2c00c420 comp rx=0 tx=0).stop 2023-07-21T22:22:58.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 shutdown_connections 2023-07-21T22:22:58.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf340ff9f0 0x7fdf3419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdf240a1480 0x7fdf240a3930 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf34105a00 0x7fdf34197a50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 --2- 172.21.15.66:0/1797228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf341003a0 0x7fdf3419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.431+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 >> 172.21.15.66:0/1797228 conn(0x7fdf340fb510 msgr2=0x7fdf34072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.432+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 shutdown_connections 2023-07-21T22:22:58.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.432+0000 7fdf3999d700 1 -- 172.21.15.66:0/1797228 wait complete. 2023-07-21T22:22:58.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:22:58.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 20989 5.20 MB/sec warmup 25 sec latency 3281.696 ms 2023-07-21T22:22:58.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 -- 172.21.15.66:0/4005615113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93200ffad0 msgr2=0x7f93200fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 --2- 172.21.15.66:0/4005615113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93200ffad0 0x7f93200fff40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f930c009990 tx=0x7f930c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 -- 172.21.15.66:0/4005615113 shutdown_connections 2023-07-21T22:22:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 --2- 172.21.15.66:0/4005615113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9320100480 0x7f932010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 --2- 172.21.15.66:0/4005615113 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9320105ae0 0x7f9320105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 --2- 172.21.15.66:0/4005615113 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f93200ffad0 0x7f93200fff40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 -- 172.21.15.66:0/4005615113 >> 172.21.15.66:0/4005615113 conn(0x7f93200fb5f0 msgr2=0x7f93200fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 -- 172.21.15.66:0/4005615113 shutdown_connections 2023-07-21T22:22:58.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.659+0000 7f9326bff700 1 -- 172.21.15.66:0/4005615113 wait complete. 2023-07-21T22:22:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.660+0000 7f9326bff700 1 Processor -- start 2023-07-21T22:22:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.660+0000 7f9326bff700 1 -- start start 2023-07-21T22:22:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.660+0000 7f9326bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 0x7f9320072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.660+0000 7f9326bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 0x7f9320072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f9326bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9320105ae0 0x7f93201a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f9326bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320073580 con 0x7f9320100480 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f9326bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93201a42e0 con 0x7f9320105ae0 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f9326bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93201a4420 con 0x7f93200ffad0 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f931ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 0x7f9320072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 0x7f9320072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 0x7f9320072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43590/0 (socket says 172.21.15.66:43590) 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f931ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 0x7f9320072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42002/0 (socket says 172.21.15.66:42002) 2023-07-21T22:22:58.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 -- 172.21.15.66:0/1628699221 learned_addr learned my addr 172.21.15.66:0/1628699221 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932519c700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9320105ae0 0x7f93201a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 -- 172.21.15.66:0/1628699221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9320105ae0 msgr2=0x7f93201a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9320105ae0 0x7f93201a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 -- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 msgr2=0x7f9320072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 0x7f9320072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 -- 172.21.15.66:0/1628699221 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930c0095f0 con 0x7f93200ffad0 2023-07-21T22:22:58.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f931ffff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 0x7f9320072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.661+0000 7f932499b700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 0x7f9320072810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f931400e9a0 tx=0x7f931400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.662+0000 7f931dffb700 1 -- 172.21.15.66:0/1628699221 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314009790 con 0x7f93200ffad0 2023-07-21T22:22:58.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.662+0000 7f931dffb700 1 -- 172.21.15.66:0/1628699221 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931400cbd0 con 0x7f93200ffad0 2023-07-21T22:22:58.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.662+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93201a4680 con 0x7f93200ffad0 2023-07-21T22:22:58.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.662+0000 7f931dffb700 1 -- 172.21.15.66:0/1628699221 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314018680 con 0x7f93200ffad0 2023-07-21T22:22:58.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.662+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93201a4b60 con 0x7f93200ffad0 2023-07-21T22:22:58.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.665+0000 7f931dffb700 1 -- 172.21.15.66:0/1628699221 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f931400cd40 con 0x7f93200ffad0 2023-07-21T22:22:58.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.666+0000 7f933a4d7b80 1 Processor -- start 2023-07-21T22:22:58.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.666+0000 7f933a4d7b80 1 -- start start 2023-07-21T22:22:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517cc1b0 0x55b7517cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7516cccb0 con 0x55b7517c7930 2023-07-21T22:22:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b75171fa40 con 0x55b7517ccac0 2023-07-21T22:22:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b751718ed0 con 0x55b7517cc1b0 2023-07-21T22:22:58.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f932519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517cc1b0 0x55b7517cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f932519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517cc1b0 0x55b7517cc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43604/0 (socket says 172.21.15.66:43604) 2023-07-21T22:22:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f932519c700 1 -- 172.21.15.66:0/952645181 learned_addr learned my addr 172.21.15.66:0/952645181 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f931ffff700 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f931dffb700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f93100a13e0 0x7f93100a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.667+0000 7f932499b700 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932519c700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f93100a13e0 0x7f93100a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f931dffb700 1 -- 172.21.15.66:0/1628699221 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9314014070 con 0x7f93200ffad0 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932499b700 1 -- 172.21.15.66:0/952645181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517cc1b0 msgr2=0x55b7517cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932499b700 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517cc1b0 0x55b7517cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932499b700 1 -- 172.21.15.66:0/952645181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 msgr2=0x55b7517c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932499b700 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932499b700 1 -- 172.21.15.66:0/952645181 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b751546de0 con 0x55b7517ccac0 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f932499b700 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c7350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93140ca790 tx=0x7f931401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e9d2cb3874fcb8f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f930affd700 1 -- 172.21.15.66:0/952645181 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931409a040 con 0x55b7517ccac0 2023-07-21T22:22:58.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f930affd700 1 -- 172.21.15.66:0/952645181 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931409e070 con 0x55b7517ccac0 2023-07-21T22:22:58.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.668+0000 7f930affd700 1 -- 172.21.15.66:0/952645181 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140caa70 con 0x55b7517ccac0 2023-07-21T22:22:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 -- 172.21.15.66:0/952645181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 msgr2=0x55b7517c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f932519c700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f93100a13e0 0x7f93100a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93180078a0 tx=0x7f9318009040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c7350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f93140ca790 tx=0x7f931401f000 comp rx=0 tx=0).stop 2023-07-21T22:22:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 -- 172.21.15.66:0/952645181 shutdown_connections 2023-07-21T22:22:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517cc1b0 0x55b7517cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/952645181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c7350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 -- 172.21.15.66:0/952645181 >> 172.21.15.66:0/952645181 conn(0x55b7517bcef0 msgr2=0x55b7517c2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 -- 172.21.15.66:0/952645181 shutdown_connections 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.669+0000 7f933a4d7b80 1 -- 172.21.15.66:0/952645181 wait complete. 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.670+0000 7f933a4d7b80 1 -- start start 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.670+0000 7f933a4d7b80 1 Processor -- start 2023-07-21T22:22:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517cc1b0 0x55b7517d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7516b8ac0 con 0x55b7517c7930 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7516b87c0 con 0x55b7517cc1b0 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b75159c920 con 0x55b7517ccac0 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f933a4d7b80 10 client.0 _ll_register_callbacks cb 0x55b7516b6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f932499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f932519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f932519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43606/0 (socket says 172.21.15.66:43606) 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f931ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517cc1b0 0x55b7517d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f932519c700 1 -- 172.21.15.66:0/2121098695 learned_addr learned my addr 172.21.15.66:0/2121098695 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f931ffff700 1 -- 172.21.15.66:0/2121098695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 msgr2=0x55b7517d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f931ffff700 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f931ffff700 1 -- 172.21.15.66:0/2121098695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 msgr2=0x55b7517d3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f931ffff700 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517d3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.671+0000 7f931ffff700 1 -- 172.21.15.66:0/2121098695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93201a4b60 con 0x55b7517cc1b0 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f931ffff700 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517cc1b0 0x55b7517d0e80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f930c00b2e0 tx=0x7f930c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caeab87c44442879 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f92fffff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f92fffff700 1 -- 172.21.15.66:0/2121098695 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c01d070 con 0x55b7517cc1b0 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7516724f0 con 0x55b7517cc1b0 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f92fffff700 1 -- 172.21.15.66:0/2121098695 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f930c004910 con 0x55b7517cc1b0 2023-07-21T22:22:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f92fffff700 1 -- 172.21.15.66:0/2121098695 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f930c004ad0 con 0x55b7517cc1b0 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f933a4d7b80 10 client.25898 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.672+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b75152fc60 con 0x55b7517cc1b0 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f92fffff700 1 -- 172.21.15.66:0/2121098695 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7516724f0 con 0x55b7517cc1b0 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f92fffff700 1 -- 172.21.15.66:0/2121098695 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f930c024450 con 0x55b7517cc1b0 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f933a4d7b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f933a4d7b80 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f933a4d7b80 20 client.25898 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b7517e1490 0x55b7517e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f933a4d7b80 4 client.25898 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b751711eb0 con 0x55b7517e1490 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.673+0000 7f932499b700 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b7517e1490 0x55b7517e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f932499b700 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b7517e1490 0x55b7517e38a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d0cb8c24873b6a server_cookie=219d134c98485f64 in_seq=0 out_seq=0 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f92fffff700 10 client.25898 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f92fffff700 1 -- 172.21.15.66:0/2121098695 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b751711eb0 con 0x55b7517e1490 2023-07-21T22:22:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f92fffff700 10 client.25898 handle_command_reply: tid=0 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f933a4d7b80 1 client.25898 shutdown 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517cc1b0 msgr2=0x55b7517d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.674+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517cc1b0 0x55b7517d0e80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f930c00b2e0 tx=0x7f930c005270 comp rx=0 tx=0).stop 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 shutdown_connections 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b7517e1490 0x55b7517e38a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f92fffff700 0 client.25898 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517c7930 0x55b7517d3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2121098695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517cc1b0 0x55b7517d0e80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 >> 172.21.15.66:0/2121098695 conn(0x55b7517bcef0 msgr2=0x55b7517c2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 shutdown_connections 2023-07-21T22:22:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2121098695 wait complete. 2023-07-21T22:22:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.675+0000 7f933a4d7b80 20 client.25898 trim_cache size 0 max 16384 2023-07-21T22:22:58.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.688+0000 7f933a4d7b80 1 Processor -- start 2023-07-21T22:22:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.688+0000 7f933a4d7b80 1 -- start start 2023-07-21T22:22:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517c7930 0x55b7517d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7517bc9b0 con 0x55b7517cc1b0 2023-07-21T22:22:58.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7517c86d0 con 0x55b7517ccac0 2023-07-21T22:22:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7517bf010 con 0x55b7517c7930 2023-07-21T22:22:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517d37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42030/0 (socket says 172.21.15.66:42030) 2023-07-21T22:22:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 -- 172.21.15.66:0/616083625 learned_addr learned my addr 172.21.15.66:0/616083625 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f932519c700 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f932499b700 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517c7930 0x55b7517d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 -- 172.21.15.66:0/616083625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517c7930 msgr2=0x55b7517d84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517c7930 0x55b7517d84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 -- 172.21.15.66:0/616083625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 msgr2=0x55b7517c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.689+0000 7f931ffff700 1 -- 172.21.15.66:0/616083625 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7517c1230 con 0x55b7517cc1b0 2023-07-21T22:22:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.690+0000 7f931ffff700 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517d37a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f234f7951f4256 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.690+0000 7f930b7fe700 1 -- 172.21.15.66:0/616083625 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c01d070 con 0x55b7517cc1b0 2023-07-21T22:22:58.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.690+0000 7f930b7fe700 1 -- 172.21.15.66:0/616083625 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f930c01e040 con 0x55b7517cc1b0 2023-07-21T22:22:58.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.690+0000 7f930b7fe700 1 -- 172.21.15.66:0/616083625 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c024c90 con 0x55b7517cc1b0 2023-07-21T22:22:58.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 -- 172.21.15.66:0/616083625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 msgr2=0x55b7517d37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517d37a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c005f00 comp rx=0 tx=0).stop 2023-07-21T22:22:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 -- 172.21.15.66:0/616083625 shutdown_connections 2023-07-21T22:22:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517ccac0 0x55b7517c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517c7930 0x55b7517d84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/616083625 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517d37a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 -- 172.21.15.66:0/616083625 >> 172.21.15.66:0/616083625 conn(0x55b7517bcef0 msgr2=0x55b7517c5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 -- 172.21.15.66:0/616083625 shutdown_connections 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 -- 172.21.15.66:0/616083625 wait complete. 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 -- start start 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.691+0000 7f933a4d7b80 1 Processor -- start 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 0x55b7517cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7517d05f0 con 0x55b7517cc1b0 2023-07-21T22:22:58.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7517d0760 con 0x55b7517c7930 2023-07-21T22:22:58.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7517d0920 con 0x55b7517ccac0 2023-07-21T22:22:58.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f933a4d7b80 10 client.0 _ll_register_callbacks cb 0x7f930c00bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:22:58.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f931ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 0x55b7517cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 0x55b7517cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42614/0 (socket says 172.21.15.66:42614) 2023-07-21T22:22:58.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 -- 172.21.15.66:0/2553749804 learned_addr learned my addr 172.21.15.66:0/2553749804 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 -- 172.21.15.66:0/2553749804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 msgr2=0x55b7517d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 -- 172.21.15.66:0/2553749804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 msgr2=0x55b7517cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:58.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.692+0000 7f932499b700 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f932499b700 1 -- 172.21.15.66:0/2553749804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b75152fc60 con 0x55b7517c7930 2023-07-21T22:22:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f931ffff700 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f932499b700 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 0x55b7517cf630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93140ca8a0 tx=0x7f9314010aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4818ace57b7ff9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:58.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f92fffff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f92fffff700 1 -- 172.21.15.66:0/2553749804 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140cd610 con 0x55b7517c7930 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f92fffff700 1 -- 172.21.15.66:0/2553749804 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93140cd340 con 0x55b7517c7930 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f931409e070 con 0x55b7517c7930 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f92fffff700 1 -- 172.21.15.66:0/2553749804 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9314099530 con 0x55b7517c7930 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f933a4d7b80 10 client.25904 fetch_fsmap learned FSMap version 25 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55b75156d250 con 0x55b7517c7930 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.693+0000 7f92fffff700 1 -- 172.21.15.66:0/2553749804 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f931409e070 con 0x55b7517c7930 2023-07-21T22:22:58.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f92fffff700 1 -- 172.21.15.66:0/2553749804 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f931409a2a0 con 0x55b7517c7930 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f933a4d7b80 10 client.25904 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f933a4d7b80 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f933a4d7b80 20 client.25904 populate_metadata read hostname 'smithi066' 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b751835bc0 0x55b751837fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f933a4d7b80 4 client.25904 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7516d4050 con 0x55b751835bc0 2023-07-21T22:22:58.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f931ffff700 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b751835bc0 0x55b751837fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f92fffff700 10 client.25904 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.694+0000 7f931ffff700 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b751835bc0 0x55b751837fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0f78f87163e6a2 server_cookie=b44e3469236fa77d in_seq=0 out_seq=0 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f92fffff700 1 -- 172.21.15.66:0/2553749804 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7516d4050 con 0x55b751835bc0 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f92fffff700 10 client.25904 handle_command_reply: tid=0 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 client.25904 shutdown 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 msgr2=0x55b7517cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 0x55b7517cf630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93140ca8a0 tx=0x7f9314010aa0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 shutdown_connections 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b7517c7930 0x55b7517cf630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55b751835bc0 0x55b751837fd0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b7517ccac0 0x55b7517d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 --2- 172.21.15.66:0/2553749804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55b7517cc1b0 0x55b7517cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f92fffff700 0 client.25904 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 >> 172.21.15.66:0/2553749804 conn(0x55b7517bcef0 msgr2=0x55b7517c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 shutdown_connections 2023-07-21T22:22:58.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 1 -- 172.21.15.66:0/2553749804 wait complete. 2023-07-21T22:22:58.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.695+0000 7f933a4d7b80 20 client.25904 trim_cache size 0 max 16384 2023-07-21T22:22:58.754 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:22:58.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f93100a13e0 msgr2=0x7f93100a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f93100a13e0 0x7f93100a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93180078a0 tx=0x7f9318009040 comp rx=0 tx=0).stop 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 msgr2=0x7f9320072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 0x7f9320072810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f931400e9a0 tx=0x7f931400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 shutdown_connections 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9320105ae0 0x7f93201a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f93100a13e0 0x7f93100a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93200ffad0 0x7f9320072810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 --2- 172.21.15.66:0/1628699221 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9320100480 0x7f9320072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 >> 172.21.15.66:0/1628699221 conn(0x7f93200fb5f0 msgr2=0x7f93200fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 shutdown_connections 2023-07-21T22:22:58.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:58.696+0000 7f9326bff700 1 -- 172.21.15.66:0/1628699221 wait complete. 2023-07-21T22:22:59.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 21729 5.22 MB/sec warmup 26 sec latency 3739.740 ms 2023-07-21T22:22:59.759 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:22:59.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:22:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 -- 172.21.15.66:0/630127464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 msgr2=0x7fe7d00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/630127464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d00fff40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009990 tx=0x7fe7c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 -- 172.21.15.66:0/630127464 shutdown_connections 2023-07-21T22:22:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/630127464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0100480 0x7fe7d010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/630127464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7d0105ae0 0x7fe7d0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/630127464 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d00fff40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 -- 172.21.15.66:0/630127464 >> 172.21.15.66:0/630127464 conn(0x7fe7d00fb5f0 msgr2=0x7fe7d00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:59.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 -- 172.21.15.66:0/630127464 shutdown_connections 2023-07-21T22:22:59.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.934+0000 7fe7d5bff700 1 -- 172.21.15.66:0/630127464 wait complete. 2023-07-21T22:22:59.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 Processor -- start 2023-07-21T22:22:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 -- start start 2023-07-21T22:22:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7d0100480 0x7fe7d0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0105ae0 0x7fe7d01a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d0073580 con 0x7fe7d00ffad0 2023-07-21T22:22:59.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d01a42c0 con 0x7fe7d0105ae0 2023-07-21T22:22:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.935+0000 7fe7d5bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d01a4400 con 0x7fe7d0100480 2023-07-21T22:22:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42050/0 (socket says 172.21.15.66:42050) 2023-07-21T22:22:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/4087962726 learned_addr learned my addr 172.21.15.66:0/4087962726 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0105ae0 0x7fe7d01a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7ceffd700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7d0100480 0x7fe7d0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/4087962726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7d0100480 msgr2=0x7fe7d0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7d0100480 0x7fe7d0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/4087962726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0105ae0 msgr2=0x7fe7d01a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0105ae0 0x7fe7d01a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/4087962726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40095f0 con 0x7fe7d00ffad0 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0105ae0 0x7fe7d01a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:22:59.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.936+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d0072810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000e9c0 tx=0x7fe7c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.937+0000 7fe7ccff9700 1 -- 172.21.15.66:0/4087962726 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00097b0 con 0x7fe7d00ffad0 2023-07-21T22:22:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.937+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d01a4660 con 0x7fe7d00ffad0 2023-07-21T22:22:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.937+0000 7fe7ccff9700 1 -- 172.21.15.66:0/4087962726 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7c000ee60 con 0x7fe7d00ffad0 2023-07-21T22:22:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.937+0000 7fe7ccff9700 1 -- 172.21.15.66:0/4087962726 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00054f0 con 0x7fe7d00ffad0 2023-07-21T22:22:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.937+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d01a4af0 con 0x7fe7d00ffad0 2023-07-21T22:22:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.941+0000 7fe7ccff9700 1 -- 172.21.15.66:0/4087962726 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fe7c0004750 con 0x7fe7d00ffad0 2023-07-21T22:22:59.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.941+0000 7fe7e95d7b80 1 Processor -- start 2023-07-21T22:22:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 -- start start 2023-07-21T22:22:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 0x55f7d6599600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599b40 0x55f7d65943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6594e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d6499d30 con 0x55f7d6599b40 2023-07-21T22:22:59.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d64ecac0 con 0x55f7d6599230 2023-07-21T22:22:59.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.942+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d64e5f50 con 0x55f7d65949b0 2023-07-21T22:22:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 0x55f7d6599600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 0x55f7d6599600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42652/0 (socket says 172.21.15.66:42652) 2023-07-21T22:22:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 -- 172.21.15.66:0/4278511756 learned_addr learned my addr 172.21.15.66:0/4278511756 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:22:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599b40 0x55f7d65943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7ceffd700 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6594e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7ccff9700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe7b80a13e0 0x7fe7b80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 -- 172.21.15.66:0/4278511756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 msgr2=0x55f7d6594e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6594e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 -- 172.21.15.66:0/4278511756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599b40 msgr2=0x55f7d65943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599b40 0x55f7d65943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7cffff700 1 -- 172.21.15.66:0/4278511756 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7d6313e60 con 0x55f7d6599230 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7ceffd700 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6594e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:22:59.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.943+0000 7fe7ccff9700 1 -- 172.21.15.66:0/4087962726 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe7c0014070 con 0x7fe7d00ffad0 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.944+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe7b80a13e0 0x7fe7b80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.944+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 0x55f7d6599600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8003e40 tx=0x7fe7c8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e8e420780ea43d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.944+0000 7fe7b5ffb700 1 -- 172.21.15.66:0/4278511756 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8007d20 con 0x55f7d6599230 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.944+0000 7fe7b5ffb700 1 -- 172.21.15.66:0/4278511756 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7c800f040 con 0x55f7d6599230 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.944+0000 7fe7b5ffb700 1 -- 172.21.15.66:0/4278511756 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c80156c0 con 0x55f7d6599230 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/4278511756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 msgr2=0x55f7d6599600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:22:59.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7cffff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe7b80a13e0 0x7fe7b80a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8020000 tx=0x7fe7c8023000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 0x55f7d6599600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8003e40 tx=0x7fe7c8009920 comp rx=0 tx=0).stop 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/4278511756 shutdown_connections 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6599230 0x55f7d6599600 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6594e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/4278511756 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599b40 0x55f7d65943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/4278511756 >> 172.21.15.66:0/4278511756 conn(0x55f7d6589f70 msgr2=0x55f7d658f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/4278511756 shutdown_connections 2023-07-21T22:22:59.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.945+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/4278511756 wait complete. 2023-07-21T22:22:59.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 -- start start 2023-07-21T22:22:59.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 Processor -- start 2023-07-21T22:23:00.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d65949b0 0x55f7d65a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d659ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 0x55f7d659e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d6485b40 con 0x55f7d6599230 2023-07-21T22:23:00.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d6485840 con 0x55f7d65949b0 2023-07-21T22:23:00.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.946+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d63699a0 con 0x55f7d6599b40 2023-07-21T22:23:00.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7e95d7b80 10 client.0 _ll_register_callbacks cb 0x55f7d64837e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:00.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d659ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 0x55f7d659e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d65949b0 0x55f7d65a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 0x55f7d659e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43674/0 (socket says 172.21.15.66:43674) 2023-07-21T22:23:00.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 -- 172.21.15.66:0/3195671425 learned_addr learned my addr 172.21.15.66:0/3195671425 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 -- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d65949b0 msgr2=0x55f7d65a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d65949b0 0x55f7d65a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 -- 172.21.15.66:0/3195671425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 msgr2=0x55f7d659ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d659ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cffff700 1 -- 172.21.15.66:0/3195671425 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c8003b20 con 0x55f7d6599b40 2023-07-21T22:23:00.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.947+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d65949b0 0x55f7d65a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:00.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7cffff700 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 0x55f7d659e610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8003fd0 tx=0x7fe7c8009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eacc01c743e51b80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:00.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7aeffd700 1 -- 172.21.15.66:0/3195671425 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8015650 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7d643f570 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7aeffd700 1 -- 172.21.15.66:0/3195671425 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe7c80159f0 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7aeffd700 1 -- 172.21.15.66:0/3195671425 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c8015bb0 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7e95d7b80 10 client.25897 fetch_fsmap learned FSMap version 25 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.948+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f7d62fcce0 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7aeffd700 1 -- 172.21.15.66:0/3195671425 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7d643f570 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7aeffd700 1 -- 172.21.15.66:0/3195671425 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe7c800e890 con 0x55f7d6599b40 2023-07-21T22:23:00.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7e95d7b80 10 client.25897 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7e95d7b80 10 client.25897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7e95d7b80 20 client.25897 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d65ae350 0x55f7d65b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7e95d7b80 4 client.25897 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7d64def30 con 0x55f7d65ae350 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.949+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d65ae350 0x55f7d65b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.950+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d65ae350 0x55f7d65b0760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d47b927ea7f988 server_cookie=c3ac0efb1b6a1ec1 in_seq=0 out_seq=0 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.950+0000 7fe7aeffd700 10 client.25897 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.950+0000 7fe7aeffd700 1 -- 172.21.15.66:0/3195671425 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7d64def30 con 0x55f7d65ae350 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.950+0000 7fe7aeffd700 10 client.25897 handle_command_reply: tid=0 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.950+0000 7fe7e95d7b80 1 client.25897 shutdown 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 msgr2=0x55f7d659e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 0x55f7d659e610 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8003fd0 tx=0x7fe7c8009370 comp rx=0 tx=0).stop 2023-07-21T22:23:00.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 shutdown_connections 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d65949b0 0x55f7d65a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d65ae350 0x55f7d65b0760 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599b40 0x55f7d659e610 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3195671425 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d659ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 >> 172.21.15.66:0/3195671425 conn(0x55f7d6589f70 msgr2=0x55f7d658f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7aeffd700 0 client.25897 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 shutdown_connections 2023-07-21T22:23:00.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3195671425 wait complete. 2023-07-21T22:23:00.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.951+0000 7fe7e95d7b80 20 client.25897 trim_cache size 0 max 16384 2023-07-21T22:23:00.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.969+0000 7fe7e95d7b80 1 Processor -- start 2023-07-21T22:23:00.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.969+0000 7fe7e95d7b80 1 -- start start 2023-07-21T22:23:00.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6593a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d6593f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d65a13e0 con 0x55f7d6599230 2023-07-21T22:23:00.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d64def30 con 0x55f7d6589f70 2023-07-21T22:23:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d643f570 con 0x55f7d65949b0 2023-07-21T22:23:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6593a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6593a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43688/0 (socket says 172.21.15.66:43688) 2023-07-21T22:23:00.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7ceffd700 1 -- 172.21.15.66:0/3590041585 learned_addr learned my addr 172.21.15.66:0/3590041585 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cffff700 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d6593f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/3590041585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 msgr2=0x55f7d6593a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6593a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/3590041585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 msgr2=0x55f7d6593f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d6593f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.970+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/3590041585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7d633a2d0 con 0x55f7d6589f70 2023-07-21T22:23:00.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65a55f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00ca710 tx=0x7fe7c009d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a649f357e8fb8df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7b67fc700 1 -- 172.21.15.66:0/3590041585 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0099fc0 con 0x55f7d6589f70 2023-07-21T22:23:00.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7b67fc700 1 -- 172.21.15.66:0/3590041585 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7c009dff0 con 0x55f7d6589f70 2023-07-21T22:23:00.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7b67fc700 1 -- 172.21.15.66:0/3590041585 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00994b0 con 0x55f7d6589f70 2023-07-21T22:23:00.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3590041585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 msgr2=0x55f7d65a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65a55f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00ca710 tx=0x7fe7c009d040 comp rx=0 tx=0).stop 2023-07-21T22:23:00.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3590041585 shutdown_connections 2023-07-21T22:23:00.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65a55f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d6599230 0x55f7d6593f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/3590041585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d65949b0 0x55f7d6593a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3590041585 >> 172.21.15.66:0/3590041585 conn(0x55f7d65950e0 msgr2=0x55f7d65aa8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3590041585 shutdown_connections 2023-07-21T22:23:00.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.971+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/3590041585 wait complete. 2023-07-21T22:23:00.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.972+0000 7fe7e95d7b80 1 -- start start 2023-07-21T22:23:00.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.972+0000 7fe7e95d7b80 1 Processor -- start 2023-07-21T22:23:00.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d65949b0 0x55f7d65fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599230 0x55f7d65a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d65a0620 con 0x55f7d65949b0 2023-07-21T22:23:00.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d65a0790 con 0x55f7d6589f70 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7d65a0950 con 0x55f7d6599230 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7e95d7b80 10 client.0 _ll_register_callbacks cb 0x7fe7c0096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d65949b0 0x55f7d65fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7cffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599230 0x55f7d65a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7cf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7ceffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d65949b0 0x55f7d65fca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42088/0 (socket says 172.21.15.66:42088) 2023-07-21T22:23:00.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.973+0000 7fe7ceffd700 1 -- 172.21.15.66:0/511649818 learned_addr learned my addr 172.21.15.66:0/511649818 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/511649818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599230 msgr2=0x55f7d65a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599230 0x55f7d65a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/511649818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d65949b0 msgr2=0x55f7d65fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d65949b0 0x55f7d65fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cf7fe700 1 -- 172.21.15.66:0/511649818 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d01a4af0 con 0x55f7d6589f70 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cffff700 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599230 0x55f7d65a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:00.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7cf7fe700 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65fc4d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000cdd0 tx=0x7fe7c00cd660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc8a50bd4769046 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7aeffd700 1 -- 172.21.15.66:0/511649818 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00a2410 con 0x55f7d6589f70 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7aeffd700 1 -- 172.21.15.66:0/511649818 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe7c0098360 con 0x55f7d6589f70 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7c800f040 con 0x55f7d6589f70 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7aeffd700 1 -- 172.21.15.66:0/511649818 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c00cd120 con 0x55f7d6589f70 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7e95d7b80 10 client.25922 fetch_fsmap learned FSMap version 25 2023-07-21T22:23:00.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f7d6312330 con 0x55f7d6589f70 2023-07-21T22:23:00.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.974+0000 7fe7aeffd700 1 -- 172.21.15.66:0/511649818 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7c800f040 con 0x55f7d6589f70 2023-07-21T22:23:00.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7aeffd700 1 -- 172.21.15.66:0/511649818 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe7c00a2060 con 0x55f7d6589f70 2023-07-21T22:23:00.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7e95d7b80 10 client.25922 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:23:00.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7e95d7b80 10 client.25922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:00.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7e95d7b80 20 client.25922 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:00.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d6599b40 0x55f7d6605fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7e95d7b80 4 client.25922 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:23:00.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f7d64a10d0 con 0x55f7d6599b40 2023-07-21T22:23:00.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7ceffd700 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d6599b40 0x55f7d6605fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7aeffd700 10 client.25922 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.975+0000 7fe7ceffd700 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d6599b40 0x55f7d6605fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2924c789b6bdb7bf server_cookie=43c68a7730403784 in_seq=0 out_seq=0 2023-07-21T22:23:00.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.976+0000 7fe7aeffd700 1 -- 172.21.15.66:0/511649818 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f7d64a10d0 con 0x55f7d6599b40 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.976+0000 7fe7aeffd700 10 client.25922 handle_command_reply: tid=0 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.976+0000 7fe7e95d7b80 1 client.25922 shutdown 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.976+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 msgr2=0x55f7d65fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.976+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65fc4d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000cdd0 tx=0x7fe7c00cd660 comp rx=0 tx=0).stop 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 shutdown_connections 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f7d6589f70 0x55f7d65fc4d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f7d6599b40 0x55f7d6605fe0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f7d6599230 0x55f7d65a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 --2- 172.21.15.66:0/511649818 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f7d65949b0 0x55f7d65fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 >> 172.21.15.66:0/511649818 conn(0x55f7d65950e0 msgr2=0x55f7d6593a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7aeffd700 0 client.25922 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 shutdown_connections 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 1 -- 172.21.15.66:0/511649818 wait complete. 2023-07-21T22:23:00.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7e95d7b80 20 client.25922 trim_cache size 0 max 16384 2023-07-21T22:23:00.035 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:23:00.035 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:23:00.035 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "79c224da-9d89-4fd6-a6f0-7a04431b25b6", 2023-07-21T22:23:00.035 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:23:00.035 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:23:00.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe7b80a13e0 msgr2=0x7fe7b80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe7b80a13e0 0x7fe7b80a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8020000 tx=0x7fe7c8023000 comp rx=0 tx=0).stop 2023-07-21T22:23:00.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 msgr2=0x7fe7d0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.977+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d0072810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe7c000e9c0 tx=0x7fe7c000ecd0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 shutdown_connections 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0105ae0 0x7fe7d01a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe7b80a13e0 0x7fe7b80a3890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe7d00ffad0 0x7fe7d0072810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 --2- 172.21.15.66:0/4087962726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe7d0100480 0x7fe7d0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 >> 172.21.15.66:0/4087962726 conn(0x7fe7d00fb5f0 msgr2=0x7fe7d01035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 shutdown_connections 2023-07-21T22:23:00.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:22:59.978+0000 7fe7d5bff700 1 -- 172.21.15.66:0/4087962726 wait complete. 2023-07-21T22:23:00.041 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:23:00.106 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:22:59 smithi066 ceph-mon[126379]: pgmap v48: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 32 op/s 2023-07-21T22:23:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:22:59 smithi149 ceph-mon[129268]: pgmap v48: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 32 op/s 2023-07-21T22:23:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 -- 172.21.15.66:0/1213694044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 msgr2=0x7f077810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 --2- 172.21.15.66:0/1213694044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f077810c910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f077000b0a0 tx=0x7f077000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 -- 172.21.15.66:0/1213694044 shutdown_connections 2023-07-21T22:23:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 --2- 172.21.15.66:0/1213694044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0778105ae0 0x7f0778105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 --2- 172.21.15.66:0/1213694044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f077810c910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 --2- 172.21.15.66:0/1213694044 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f07780ffad0 0x7f07780fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 -- 172.21.15.66:0/1213694044 >> 172.21.15.66:0/1213694044 conn(0x7f07780fb5f0 msgr2=0x7f07780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 -- 172.21.15.66:0/1213694044 shutdown_connections 2023-07-21T22:23:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.211+0000 7f077e5ff700 1 -- 172.21.15.66:0/1213694044 wait complete. 2023-07-21T22:23:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.212+0000 7f077e5ff700 1 Processor -- start 2023-07-21T22:23:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.212+0000 7f077e5ff700 1 -- start start 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.212+0000 7f077e5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07780ffad0 0x7f0778195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077e5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f07781962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077e5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0778105ae0 0x7f077819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077e5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819ad80 con 0x7f0778105ae0 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077e5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819aef0 con 0x7f07780ffad0 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077e5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077819b0b0 con 0x7f0778100480 2023-07-21T22:23:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f07777fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f07781962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077cb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0778105ae0 0x7f077819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f07777fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f07781962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43706/0 (socket says 172.21.15.66:43706) 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f0777fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07780ffad0 0x7f0778195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f07777fe700 1 -- 172.21.15.66:0/2925656433 learned_addr learned my addr 172.21.15.66:0/2925656433 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f0777fff700 1 -- 172.21.15.66:0/2925656433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 msgr2=0x7f07781962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f0777fff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f07781962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f0777fff700 1 -- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0778105ae0 msgr2=0x7f077819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f0777fff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0778105ae0 0x7f077819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f0777fff700 1 -- 172.21.15.66:0/2925656433 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770009d00 con 0x7f07780ffad0 2023-07-21T22:23:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.213+0000 7f077cb9c700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0778105ae0 0x7f077819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.214+0000 7f0777fff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07780ffad0 0x7f0778195da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f076800e9f0 tx=0x7f076800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.214+0000 7f07757fa700 1 -- 172.21.15.66:0/2925656433 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076800c9c0 con 0x7f07780ffad0 2023-07-21T22:23:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.214+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077819b390 con 0x7f07780ffad0 2023-07-21T22:23:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.214+0000 7f07757fa700 1 -- 172.21.15.66:0/2925656433 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0768010430 con 0x7f07780ffad0 2023-07-21T22:23:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.214+0000 7f07757fa700 1 -- 172.21.15.66:0/2925656433 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768018680 con 0x7f07780ffad0 2023-07-21T22:23:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.214+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07781a2bd0 con 0x7f07780ffad0 2023-07-21T22:23:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.217+0000 7f07757fa700 1 -- 172.21.15.66:0/2925656433 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f0768010860 con 0x7f07780ffad0 2023-07-21T22:23:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.218+0000 7f0791e42b80 1 Processor -- start 2023-07-21T22:23:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 -- start start 2023-07-21T22:23:00.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f07757fa700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07600a13e0 0x7f07600a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f07757fa700 1 -- 172.21.15.66:0/2925656433 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0768014070 con 0x7f07780ffad0 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab824870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab829450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab829de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab724cb0 con 0x55c1ab829990 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab777a40 con 0x55c1ab824db0 2023-07-21T22:23:00.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0791e42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab770ed0 con 0x55c1ab8244a0 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f077cb9c700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07600a13e0 0x7f07600a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f07777fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab829450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f0777fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab824870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f07777fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab829450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42724/0 (socket says 172.21.15.66:42724) 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.219+0000 7f07777fe700 1 -- 172.21.15.66:0/1120776252 learned_addr learned my addr 172.21.15.66:0/1120776252 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f077cb9c700 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab829de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f0777fff700 1 -- 172.21.15.66:0/1120776252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 msgr2=0x55c1ab829450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f0777fff700 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab829450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f0777fff700 1 -- 172.21.15.66:0/1120776252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 msgr2=0x55c1ab829de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f0777fff700 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab829de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f0777fff700 1 -- 172.21.15.66:0/1120776252 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1ab59ede0 con 0x55c1ab8244a0 2023-07-21T22:23:00.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f077cb9c700 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab829de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:23:00.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f0777fff700 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab824870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07680ca630 tx=0x7f0768096350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45b2c884a2885be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f077cb9c700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07600a13e0 0x7f07600a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f077000b780 tx=0x7f077000baf0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f075e7fc700 1 -- 172.21.15.66:0/1120776252 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768099ee0 con 0x55c1ab8244a0 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f075e7fc700 1 -- 172.21.15.66:0/1120776252 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f076809df10 con 0x55c1ab8244a0 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.220+0000 7f075e7fc700 1 -- 172.21.15.66:0/1120776252 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680995d0 con 0x55c1ab8244a0 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 -- 172.21.15.66:0/1120776252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 msgr2=0x55c1ab824870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab824870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07680ca630 tx=0x7f0768096350 comp rx=0 tx=0).stop 2023-07-21T22:23:00.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 -- 172.21.15.66:0/1120776252 shutdown_connections 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab829de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab824870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1120776252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab829450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 -- 172.21.15.66:0/1120776252 >> 172.21.15.66:0/1120776252 conn(0x55c1ab814ef0 msgr2=0x55c1ab81a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 -- 172.21.15.66:0/1120776252 shutdown_connections 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.221+0000 7f0791e42b80 1 -- 172.21.15.66:0/1120776252 wait complete. 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.222+0000 7f0791e42b80 1 -- start start 2023-07-21T22:23:00.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.222+0000 7f0791e42b80 1 Processor -- start 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab82ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab82d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab82d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab710ac0 con 0x55c1ab829990 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab7107c0 con 0x55c1ab824db0 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab5f4920 con 0x55c1ab8244a0 2023-07-21T22:23:00.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0791e42b80 10 client.0 _ll_register_callbacks cb 0x55c1ab70e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:00.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f077cb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab82d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab82ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab82ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43724/0 (socket says 172.21.15.66:43724) 2023-07-21T22:23:00.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f07777fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab82d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 -- 172.21.15.66:0/3151152704 learned_addr learned my addr 172.21.15.66:0/3151152704 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f077cb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab82d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42114/0 (socket says 172.21.15.66:42114) 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 -- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 msgr2=0x55c1ab82d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab82d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 -- 172.21.15.66:0/3151152704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 msgr2=0x55c1ab82d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab82d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.223+0000 7f0777fff700 1 -- 172.21.15.66:0/3151152704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07781a2bd0 con 0x55c1ab8244a0 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f0777fff700 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab82ff40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f07680963e0 tx=0x7f0768010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1400c29e2bcf71d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f07577fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:00.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f07577fe700 1 -- 172.21.15.66:0/3151152704 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768099f60 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f07577fe700 1 -- 172.21.15.66:0/3151152704 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07680b81b0 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1ab6ca4f0 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f07577fe700 1 -- 172.21.15.66:0/3151152704 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07680993d0 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f0791e42b80 10 client.25915 fetch_fsmap learned FSMap version 25 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c1ab587c60 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f07577fe700 1 -- 172.21.15.66:0/3151152704 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07680ca030 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f07577fe700 1 -- 172.21.15.66:0/3151152704 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f07680a1ea0 con 0x55c1ab8244a0 2023-07-21T22:23:00.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f0791e42b80 10 client.25915 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.224+0000 7f0791e42b80 10 client.25915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f0791e42b80 20 client.25915 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f0791e42b80 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab83d3d0 0x55c1ab83f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f0791e42b80 4 client.25915 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1ab769eb0 con 0x55c1ab83d3d0 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f07777fe700 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab83d3d0 0x55c1ab83f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f07577fe700 10 client.25915 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.225+0000 7f07777fe700 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab83d3d0 0x55c1ab83f7e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a31f008805f36d server_cookie=a809d5c69b3f186f in_seq=0 out_seq=0 2023-07-21T22:23:00.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f07577fe700 1 -- 172.21.15.66:0/3151152704 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c1ab769eb0 con 0x55c1ab83d3d0 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f07577fe700 10 client.25915 handle_command_reply: tid=0 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 client.25915 shutdown 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 msgr2=0x55c1ab82ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab82ff40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f07680963e0 tx=0x7f0768010b60 comp rx=0 tx=0).stop 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 shutdown_connections 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab83d3d0 0x55c1ab83f7e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab829990 0x55c1ab82d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab8244a0 0x55c1ab82ff40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 --2- 172.21.15.66:0/3151152704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab82d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 >> 172.21.15.66:0/3151152704 conn(0x55c1ab814ef0 msgr2=0x55c1ab81a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f07577fe700 0 client.25915 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 shutdown_connections 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.226+0000 7f0791e42b80 1 -- 172.21.15.66:0/3151152704 wait complete. 2023-07-21T22:23:00.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.227+0000 7f0791e42b80 20 client.25915 trim_cache size 0 max 16384 2023-07-21T22:23:00.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.244+0000 7f0791e42b80 1 Processor -- start 2023-07-21T22:23:00.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.244+0000 7f0791e42b80 1 -- start start 2023-07-21T22:23:00.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab824db0 0x55c1ab82ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab829990 0x55c1ab827a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0791e42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab839680 con 0x55c1ab8244a0 2023-07-21T22:23:00.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0791e42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab830530 con 0x55c1ab829990 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0791e42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab81ba60 con 0x55c1ab824db0 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42126/0 (socket says 172.21.15.66:42126) 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 -- 172.21.15.66:0/4037294225 learned_addr learned my addr 172.21.15.66:0/4037294225 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f07777fe700 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab824db0 0x55c1ab82ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f077cb9c700 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab829990 0x55c1ab827a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 -- 172.21.15.66:0/4037294225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab824db0 msgr2=0x55c1ab82ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab824db0 0x55c1ab82ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 -- 172.21.15.66:0/4037294225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab829990 msgr2=0x55c1ab827a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab829990 0x55c1ab827a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.245+0000 7f0777fff700 1 -- 172.21.15.66:0/4037294225 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1ab5c5250 con 0x55c1ab8244a0 2023-07-21T22:23:00.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.246+0000 7f0777fff700 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8347d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f076800c5b0 tx=0x7f0768098490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706b419ce19609f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.246+0000 7f075effd700 1 -- 172.21.15.66:0/4037294225 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680ca910 con 0x55c1ab8244a0 2023-07-21T22:23:00.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.246+0000 7f075effd700 1 -- 172.21.15.66:0/4037294225 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07680ca040 con 0x55c1ab8244a0 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.246+0000 7f075effd700 1 -- 172.21.15.66:0/4037294225 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680cac20 con 0x55c1ab8244a0 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.246+0000 7f0791e42b80 1 -- 172.21.15.66:0/4037294225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 msgr2=0x55c1ab8347d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.246+0000 7f0791e42b80 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8347d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f076800c5b0 tx=0x7f0768098490 comp rx=0 tx=0).stop 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 -- 172.21.15.66:0/4037294225 shutdown_connections 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab829990 0x55c1ab827a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8347d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 --2- 172.21.15.66:0/4037294225 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab824db0 0x55c1ab82ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 -- 172.21.15.66:0/4037294225 >> 172.21.15.66:0/4037294225 conn(0x55c1ab814ef0 msgr2=0x55c1ab838f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 -- 172.21.15.66:0/4037294225 shutdown_connections 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 -- 172.21.15.66:0/4037294225 wait complete. 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 -- start start 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.247+0000 7f0791e42b80 1 Processor -- start 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab832e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 0x55c1ab8333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab833b80 con 0x55c1ab8244a0 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab891260 con 0x55c1ab824db0 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1ab8913a0 con 0x55c1ab829990 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0791e42b80 10 client.0 _ll_register_callbacks cb 0x7f076c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f0777fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 0x55c1ab8333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 0x55c1ab8333f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43748/0 (socket says 172.21.15.66:43748) 2023-07-21T22:23:00.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 -- 172.21.15.66:0/1406281144 learned_addr learned my addr 172.21.15.66:0/1406281144 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f07777fe700 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab832e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 -- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 msgr2=0x55c1ab832e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab832e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 -- 172.21.15.66:0/1406281144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 msgr2=0x55c1ab8348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.248+0000 7f077cb9c700 1 -- 172.21.15.66:0/1406281144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1ab587c60 con 0x55c1ab829990 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f077cb9c700 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 0x55c1ab8333f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f077000b0a0 tx=0x7f07700099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f20388dfc413fb22 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:00.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f07577fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f07577fe700 1 -- 172.21.15.66:0/1406281144 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077000e040 con 0x55c1ab829990 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f076809df10 con 0x55c1ab829990 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f07577fe700 1 -- 172.21.15.66:0/1406281144 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f077001fcb0 con 0x55c1ab829990 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f07577fe700 1 -- 172.21.15.66:0/1406281144 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077001fe70 con 0x55c1ab829990 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f0791e42b80 10 client.25921 fetch_fsmap learned FSMap version 25 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55c1ab59d2b0 con 0x55c1ab829990 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.249+0000 7f07577fe700 1 -- 172.21.15.66:0/1406281144 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f076809df10 con 0x55c1ab829990 2023-07-21T22:23:00.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f07577fe700 1 -- 172.21.15.66:0/1406281144 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 25) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0770005140 con 0x55c1ab829990 2023-07-21T22:23:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0791e42b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T22:23:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0791e42b80 10 client.25921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0791e42b80 20 client.25921 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab892a30 0x55c1ab894e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0791e42b80 4 client.25921 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:23:00.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c1ab72c050 con 0x55c1ab892a30 2023-07-21T22:23:00.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0777fff700 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab892a30 0x55c1ab894e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f0777fff700 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab892a30 0x55c1ab894e40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe07931cce7c3417 server_cookie=8eb447f480f14819 in_seq=0 out_seq=0 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.250+0000 7f07577fe700 10 client.25921 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f07577fe700 1 -- 172.21.15.66:0/1406281144 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c1ab72c050 con 0x55c1ab892a30 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f07577fe700 10 client.25921 handle_command_reply: tid=0 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 client.25921 shutdown 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 msgr2=0x55c1ab8333f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 0x55c1ab8333f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f077000b0a0 tx=0x7f07700099e0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 shutdown_connections 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c1ab892a30 0x55c1ab894e40 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c1ab829990 0x55c1ab8333f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c1ab8244a0 0x55c1ab8348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f07577fe700 0 client.25921 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 --2- 172.21.15.66:0/1406281144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1ab824db0 0x55c1ab832e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 >> 172.21.15.66:0/1406281144 conn(0x55c1ab814ef0 msgr2=0x55c1ab827a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 shutdown_connections 2023-07-21T22:23:00.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 1 -- 172.21.15.66:0/1406281144 wait complete. 2023-07-21T22:23:00.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f0791e42b80 20 client.25921 trim_cache size 0 max 16384 2023-07-21T22:23:00.304 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:23:00.304 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:23:00.304 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:23:00.304 INFO:teuthology.orchestra.run.smithi066.stdout: "79c224da-9d89-4fd6-a6f0-7a04431b25b6": { 2023-07-21T22:23:00.305 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:23:00.305 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "79c224da-9d89-4fd6-a6f0-7a04431b25b6", 2023-07-21T22:23:00.305 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:23:00.305 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:23:00.305 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:23:00.305 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:23:00.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.251+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07600a13e0 msgr2=0x7f07600a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07600a13e0 0x7f07600a3890 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f077000b780 tx=0x7f077000baf0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07780ffad0 msgr2=0x7f0778195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:00.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07780ffad0 0x7f0778195da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f076800e9f0 tx=0x7f076800ed00 comp rx=0 tx=0).stop 2023-07-21T22:23:00.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 shutdown_connections 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07780ffad0 0x7f0778195da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07600a13e0 0x7f07600a3890 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0778105ae0 0x7f077819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 --2- 172.21.15.66:0/2925656433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0778100480 0x7f07781962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 >> 172.21.15.66:0/2925656433 conn(0x7f07780fb5f0 msgr2=0x7f0778106930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 shutdown_connections 2023-07-21T22:23:00.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:00.252+0000 7f077e5ff700 1 -- 172.21.15.66:0/2925656433 wait complete. 2023-07-21T22:23:00.314 INFO:tasks.cephfs.filesystem:scrub status for tag:79c224da-9d89-4fd6-a6f0-7a04431b25b6 - {'path': '/', 'tag': '79c224da-9d89-4fd6-a6f0-7a04431b25b6', 'options': 'recursive,force'} 2023-07-21T22:23:00.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:22:59 smithi104 ceph-mon[129021]: pgmap v48: 129 pgs: 129 active+clean; 39 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 2.4 MiB/s wr, 32 op/s 2023-07-21T22:23:00.508 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22115 5.04 MB/sec warmup 27 sec latency 309.525 ms 2023-07-21T22:23:01.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:23:01.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:01.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:01.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:01.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:00 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:23:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:01.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:00 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:23:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:00 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:01.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22238 4.87 MB/sec warmup 28 sec latency 853.110 ms 2023-07-21T22:23:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:01 smithi104 ceph-mon[129021]: pgmap v49: 129 pgs: 129 active+clean; 38 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.2 MiB/s wr, 73 op/s 2023-07-21T22:23:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:01 smithi104 ceph-mon[129021]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-07-21T22:23:02.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:01 smithi066 ceph-mon[126379]: pgmap v49: 129 pgs: 129 active+clean; 38 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.2 MiB/s wr, 73 op/s 2023-07-21T22:23:02.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:01 smithi066 ceph-mon[126379]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-07-21T22:23:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:01 smithi149 ceph-mon[129268]: pgmap v49: 129 pgs: 129 active+clean; 38 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 3.2 MiB/s wr, 73 op/s 2023-07-21T22:23:02.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:01 smithi149 ceph-mon[129268]: mgrmap e26: x(active, since 92s), standbys: z, y 2023-07-21T22:23:02.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22238 4.70 MB/sec warmup 29 sec latency 1853.219 ms 2023-07-21T22:23:03.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22303 4.55 MB/sec warmup 30 sec latency 2368.311 ms 2023-07-21T22:23:04.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:03 smithi104 ceph-mon[129021]: pgmap v50: 129 pgs: 129 active+clean; 38 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.1 MiB/s wr, 67 op/s 2023-07-21T22:23:04.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:03 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:04.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:03 smithi066 ceph-mon[126379]: pgmap v50: 129 pgs: 129 active+clean; 38 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.1 MiB/s wr, 67 op/s 2023-07-21T22:23:04.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:03 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:04.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:03 smithi149 ceph-mon[129268]: pgmap v50: 129 pgs: 129 active+clean; 38 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 3.1 MiB/s wr, 67 op/s 2023-07-21T22:23:04.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:03 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:04.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22303 4.41 MB/sec warmup 31 sec latency 1441.734 ms 2023-07-21T22:23:05.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22304 4.27 MB/sec warmup 32 sec latency 2113.290 ms 2023-07-21T22:23:06.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:05 smithi104 ceph-mon[129021]: pgmap v51: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.2 MiB/s wr, 103 op/s 2023-07-21T22:23:06.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:05 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:23:06.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:05 smithi066 ceph-mon[126379]: pgmap v51: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.2 MiB/s wr, 103 op/s 2023-07-21T22:23:06.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:05 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:23:06.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:05 smithi149 ceph-mon[129268]: pgmap v51: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 3.2 MiB/s wr, 103 op/s 2023-07-21T22:23:06.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:05 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:23:06.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22304 4.14 MB/sec warmup 33 sec latency 1328.719 ms 2023-07-21T22:23:07.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:06 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:23:07.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:06 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:23:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:06 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:23:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:06 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:23:07.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:06 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:23:07.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:06 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:23:07.509 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22304 4.02 MB/sec warmup 34 sec latency 2328.857 ms 2023-07-21T22:23:08.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:07 smithi104 ceph-mon[129021]: pgmap v52: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 856 KiB/s wr, 77 op/s 2023-07-21T22:23:08.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:07 smithi066 ceph-mon[126379]: pgmap v52: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 856 KiB/s wr, 77 op/s 2023-07-21T22:23:08.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:07 smithi149 ceph-mon[129268]: pgmap v52: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 856 KiB/s wr, 77 op/s 2023-07-21T22:23:08.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22304 3.90 MB/sec warmup 35 sec latency 3329.021 ms 2023-07-21T22:23:09.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:08 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:09.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:08 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:09.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:08 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:09.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22304 3.79 MB/sec warmup 36 sec latency 4329.186 ms 2023-07-21T22:23:10.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:09 smithi104 ceph-mon[129021]: pgmap v53: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 856 KiB/s wr, 77 op/s 2023-07-21T22:23:10.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:09 smithi066 ceph-mon[126379]: pgmap v53: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 856 KiB/s wr, 77 op/s 2023-07-21T22:23:10.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:09 smithi149 ceph-mon[129268]: pgmap v53: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 856 KiB/s wr, 77 op/s 2023-07-21T22:23:10.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 22342 3.70 MB/sec warmup 37 sec latency 4991.931 ms 2023-07-21T22:23:11.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:10 smithi104 ceph-mon[129021]: pgmap v54: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 867 KiB/s wr, 81 op/s 2023-07-21T22:23:11.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:10 smithi066 ceph-mon[126379]: pgmap v54: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 867 KiB/s wr, 81 op/s 2023-07-21T22:23:11.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:10 smithi149 ceph-mon[129268]: pgmap v54: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 867 KiB/s wr, 81 op/s 2023-07-21T22:23:11.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 23933 3.78 MB/sec warmup 38 sec latency 56.147 ms 2023-07-21T22:23:12.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25476 3.97 MB/sec warmup 39 sec latency 10.764 ms 2023-07-21T22:23:13.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25541 3.87 MB/sec warmup 40 sec latency 970.986 ms 2023-07-21T22:23:14.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:13 smithi104 ceph-mon[129021]: pgmap v55: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 108 KiB/s wr, 40 op/s 2023-07-21T22:23:14.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:13 smithi066 ceph-mon[126379]: pgmap v55: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 108 KiB/s wr, 40 op/s 2023-07-21T22:23:14.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:13 smithi149 ceph-mon[129268]: pgmap v55: 129 pgs: 129 active+clean; 30 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 108 KiB/s wr, 40 op/s 2023-07-21T22:23:14.510 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25541 3.78 MB/sec warmup 41 sec latency 1971.038 ms 2023-07-21T22:23:15.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25597 3.69 MB/sec warmup 42 sec latency 2642.668 ms 2023-07-21T22:23:16.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:15 smithi104 ceph-mon[129021]: pgmap v56: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 344 KiB/s wr, 91 op/s 2023-07-21T22:23:16.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:15 smithi066 ceph-mon[126379]: pgmap v56: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 344 KiB/s wr, 91 op/s 2023-07-21T22:23:16.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:15 smithi149 ceph-mon[129268]: pgmap v56: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 344 KiB/s wr, 91 op/s 2023-07-21T22:23:16.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25597 3.61 MB/sec warmup 43 sec latency 1265.431 ms 2023-07-21T22:23:17.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25597 3.52 MB/sec warmup 44 sec latency 2265.528 ms 2023-07-21T22:23:18.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:17 smithi104 ceph-mon[129021]: pgmap v57: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 247 KiB/s wr, 55 op/s 2023-07-21T22:23:18.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:17 smithi066 ceph-mon[126379]: pgmap v57: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 247 KiB/s wr, 55 op/s 2023-07-21T22:23:18.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:17 smithi149 ceph-mon[129268]: pgmap v57: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 247 KiB/s wr, 55 op/s 2023-07-21T22:23:18.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25597 3.45 MB/sec warmup 45 sec latency 3265.668 ms 2023-07-21T22:23:19.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 25597 3.37 MB/sec warmup 46 sec latency 4265.803 ms 2023-07-21T22:23:20.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:19 smithi104 ceph-mon[129021]: pgmap v58: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 247 KiB/s wr, 55 op/s 2023-07-21T22:23:20.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:19 smithi066 ceph-mon[126379]: pgmap v58: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 247 KiB/s wr, 55 op/s 2023-07-21T22:23:20.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:19 smithi149 ceph-mon[129268]: pgmap v58: 129 pgs: 129 active+clean; 31 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 247 KiB/s wr, 55 op/s 2023-07-21T22:23:20.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 26309 3.40 MB/sec warmup 47 sec latency 4930.091 ms 2023-07-21T22:23:21.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 28683 3.62 MB/sec warmup 48 sec latency 10.159 ms 2023-07-21T22:23:22.119 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:21 smithi104 ceph-mon[129021]: pgmap v59: 129 pgs: 129 active+clean; 32 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 285 KiB/s wr, 62 op/s 2023-07-21T22:23:22.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:21 smithi066 ceph-mon[126379]: pgmap v59: 129 pgs: 129 active+clean; 32 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 285 KiB/s wr, 62 op/s 2023-07-21T22:23:22.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:21 smithi149 ceph-mon[129268]: pgmap v59: 129 pgs: 129 active+clean; 32 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 285 KiB/s wr, 62 op/s 2023-07-21T22:23:22.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29083 3.55 MB/sec warmup 49 sec latency 519.493 ms 2023-07-21T22:23:23.511 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29083 3.48 MB/sec warmup 50 sec latency 1519.582 ms 2023-07-21T22:23:24.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:23 smithi066 ceph-mon[126379]: pgmap v60: 129 pgs: 129 active+clean; 32 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 274 KiB/s wr, 58 op/s 2023-07-21T22:23:24.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:23 smithi149 ceph-mon[129268]: pgmap v60: 129 pgs: 129 active+clean; 32 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 274 KiB/s wr, 58 op/s 2023-07-21T22:23:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:23 smithi104 ceph-mon[129021]: pgmap v60: 129 pgs: 129 active+clean; 32 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 274 KiB/s wr, 58 op/s 2023-07-21T22:23:24.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29083 3.41 MB/sec warmup 51 sec latency 2519.722 ms 2023-07-21T22:23:25.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29207 3.35 MB/sec warmup 52 sec latency 3188.553 ms 2023-07-21T22:23:26.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:25 smithi066 ceph-mon[126379]: pgmap v61: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 467 KiB/s wr, 93 op/s 2023-07-21T22:23:26.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:25 smithi149 ceph-mon[129268]: pgmap v61: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 467 KiB/s wr, 93 op/s 2023-07-21T22:23:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:25 smithi104 ceph-mon[129021]: pgmap v61: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 467 KiB/s wr, 93 op/s 2023-07-21T22:23:26.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29207 3.29 MB/sec warmup 53 sec latency 1210.841 ms 2023-07-21T22:23:27.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29207 3.23 MB/sec warmup 54 sec latency 2210.971 ms 2023-07-21T22:23:28.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:27 smithi066 ceph-mon[126379]: pgmap v62: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 231 KiB/s wr, 42 op/s 2023-07-21T22:23:28.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:27 smithi149 ceph-mon[129268]: pgmap v62: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 231 KiB/s wr, 42 op/s 2023-07-21T22:23:28.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:27 smithi104 ceph-mon[129021]: pgmap v62: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 231 KiB/s wr, 42 op/s 2023-07-21T22:23:28.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29207 3.17 MB/sec warmup 55 sec latency 3211.121 ms 2023-07-21T22:23:29.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29207 3.11 MB/sec warmup 56 sec latency 4211.288 ms 2023-07-21T22:23:30.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:29 smithi066 ceph-mon[126379]: pgmap v63: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 231 KiB/s wr, 42 op/s 2023-07-21T22:23:30.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:29 smithi149 ceph-mon[129268]: pgmap v63: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 231 KiB/s wr, 42 op/s 2023-07-21T22:23:30.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:23:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:29 smithi104 ceph-mon[129021]: pgmap v63: 129 pgs: 129 active+clean; 33 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 231 KiB/s wr, 42 op/s 2023-07-21T22:23:30.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 29923 3.14 MB/sec warmup 57 sec latency 4874.560 ms 2023-07-21T22:23:30.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 -- 172.21.15.66:0/1318608765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5080731c0 msgr2=0x7ff50810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 --2- 172.21.15.66:0/1318608765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5080731c0 0x7ff50810bdd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009990 tx=0x7ff4f8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 -- 172.21.15.66:0/1318608765 shutdown_connections 2023-07-21T22:23:30.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 --2- 172.21.15.66:0/1318608765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff50810c310 0x7ff50810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 --2- 172.21.15.66:0/1318608765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5080728b0 0x7ff508072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 --2- 172.21.15.66:0/1318608765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5080731c0 0x7ff50810bdd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 -- 172.21.15.66:0/1318608765 >> 172.21.15.66:0/1318608765 conn(0x7ff5080fb610 msgr2=0x7ff5080fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 -- 172.21.15.66:0/1318608765 shutdown_connections 2023-07-21T22:23:30.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.499+0000 7ff50fdff700 1 -- 172.21.15.66:0/1318608765 wait complete. 2023-07-21T22:23:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.500+0000 7ff50fdff700 1 Processor -- start 2023-07-21T22:23:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.500+0000 7ff50fdff700 1 -- start start 2023-07-21T22:23:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50fdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 0x7ff508197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50fdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5080731c0 0x7ff508198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50fdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff50810c310 0x7ff50819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50fdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50819cdf0 con 0x7ff5080731c0 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50fdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50819cf60 con 0x7ff5080728b0 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50fdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50819d120 con 0x7ff50810c310 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 0x7ff508197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50d39a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5080731c0 0x7ff508198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 0x7ff508197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:44898/0 (socket says 172.21.15.66:44898) 2023-07-21T22:23:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.501+0000 7ff50db9b700 1 -- 172.21.15.66:0/2610930400 learned_addr learned my addr 172.21.15.66:0/2610930400 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff50db9b700 1 -- 172.21.15.66:0/2610930400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff50810c310 msgr2=0x7ff50819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff50810c310 0x7ff50819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff50db9b700 1 -- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5080731c0 msgr2=0x7ff508198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5080731c0 0x7ff508198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff50db9b700 1 -- 172.21.15.66:0/2610930400 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80095f0 con 0x7ff5080728b0 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 0x7ff508197e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e920 tx=0x7ff50400ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff4feffd700 1 -- 172.21.15.66:0/2610930400 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504009790 con 0x7ff5080728b0 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff4feffd700 1 -- 172.21.15.66:0/2610930400 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50400ee70 con 0x7ff5080728b0 2023-07-21T22:23:30.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.502+0000 7ff4feffd700 1 -- 172.21.15.66:0/2610930400 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504018680 con 0x7ff5080728b0 2023-07-21T22:23:30.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.503+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50819d400 con 0x7ff5080728b0 2023-07-21T22:23:30.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.503+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5081a4c40 con 0x7ff5080728b0 2023-07-21T22:23:30.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.506+0000 7ff4feffd700 1 -- 172.21.15.66:0/2610930400 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff50400cbd0 con 0x7ff5080728b0 2023-07-21T22:23:30.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 Processor -- start 2023-07-21T22:23:30.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 -- start start 2023-07-21T22:23:30.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff4feffd700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4f40a1340 0x7ff4f40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc7c8cb0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc81ba40 con 0x5623bc8c81b0 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.508+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc814ed0 con 0x5623bc8c3930 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff4feffd700 1 -- 172.21.15.66:0/2610930400 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff504014070 con 0x7ff5080728b0 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50e39c700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4f40a1340 0x7ff4f40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:44912/0 (socket says 172.21.15.66:44912) 2023-07-21T22:23:30.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 -- 172.21.15.66:0/2709265583 learned_addr learned my addr 172.21.15.66:0/2709265583 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:30.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50d39a700 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50e39c700 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 -- 172.21.15.66:0/2709265583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 msgr2=0x5623bc8c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 -- 172.21.15.66:0/2709265583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 msgr2=0x5623bc8c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.509+0000 7ff50db9b700 1 -- 172.21.15.66:0/2709265583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623bc642de0 con 0x5623bc8c81b0 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.510+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8c8580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5040a1ec0 tx=0x7ff504099b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c434b4e7afd3e6c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.510+0000 7ff50e39c700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4f40a1340 0x7ff4f40a37f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5000060b0 tx=0x7ff500013220 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.510+0000 7ff4f3fff700 1 -- 172.21.15.66:0/2709265583 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50409dfd0 con 0x5623bc8c81b0 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.510+0000 7ff4f3fff700 1 -- 172.21.15.66:0/2709265583 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5040a23f0 con 0x5623bc8c81b0 2023-07-21T22:23:30.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.510+0000 7ff4f3fff700 1 -- 172.21.15.66:0/2709265583 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5040b83e0 con 0x5623bc8c81b0 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2709265583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 msgr2=0x5623bc8c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8c8580 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5040a1ec0 tx=0x7ff504099b60 comp rx=0 tx=0).stop 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2709265583 shutdown_connections 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8c8580 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2709265583 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2709265583 >> 172.21.15.66:0/2709265583 conn(0x5623bc8b8ef0 msgr2=0x5623bc8be830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2709265583 shutdown_connections 2023-07-21T22:23:30.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.511+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2709265583 wait complete. 2023-07-21T22:23:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.512+0000 7ff5237d8b80 1 -- start start 2023-07-21T22:23:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.512+0000 7ff5237d8b80 1 Processor -- start 2023-07-21T22:23:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc7b4ac0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc7b47c0 con 0x5623bc8c81b0 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc698920 con 0x5623bc8c3930 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff5237d8b80 10 client.0 _ll_register_callbacks cb 0x5623bc7b2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50d39a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50d39a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:44928/0 (socket says 172.21.15.66:44928) 2023-07-21T22:23:30.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50d39a700 1 -- 172.21.15.66:0/2158748459 learned_addr learned my addr 172.21.15.66:0/2158748459 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50e39c700 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50db9b700 1 -- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 msgr2=0x5623bc8ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50db9b700 1 -- 172.21.15.66:0/2158748459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 msgr2=0x5623bc8cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8cd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.513+0000 7ff50db9b700 1 -- 172.21.15.66:0/2158748459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5081a4c40 con 0x5623bc8c3930 2023-07-21T22:23:30.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff50db9b700 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8cfc90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff504099f30 tx=0x7ff504096410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d7c4e4a6227c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff4f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff4f17fa700 1 -- 172.21.15.66:0/2158748459 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5040b82b0 con 0x5623bc8c3930 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623bc76e4f0 con 0x5623bc8c3930 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff4f17fa700 1 -- 172.21.15.66:0/2158748459 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff504099490 con 0x5623bc8c3930 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff4f17fa700 1 -- 172.21.15.66:0/2158748459 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff504099650 con 0x5623bc8c3930 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff5237d8b80 10 client.25930 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.514+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5623bc62bc60 con 0x5623bc8c3930 2023-07-21T22:23:30.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff4f17fa700 1 -- 172.21.15.66:0/2158748459 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5623bc76e4f0 con 0x5623bc8c3930 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff4f17fa700 1 -- 172.21.15.66:0/2158748459 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff504094cd0 con 0x5623bc8c3930 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff5237d8b80 10 client.25930 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff5237d8b80 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff5237d8b80 20 client.25930 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc8dd0e0 0x5623bc8df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff5237d8b80 4 client.25930 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623bc80deb0 con 0x5623bc8dd0e0 2023-07-21T22:23:30.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.515+0000 7ff50d39a700 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc8dd0e0 0x5623bc8df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.516+0000 7ff50d39a700 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc8dd0e0 0x5623bc8df4f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc17baea2385277a server_cookie=f5479d9e8c5f790c in_seq=0 out_seq=0 2023-07-21T22:23:30.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.516+0000 7ff4f17fa700 10 client.25930 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:30.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.516+0000 7ff4f17fa700 1 -- 172.21.15.66:0/2158748459 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5623bc80deb0 con 0x5623bc8dd0e0 2023-07-21T22:23:30.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.516+0000 7ff4f17fa700 10 client.25930 handle_command_reply: tid=0 2023-07-21T22:23:30.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.516+0000 7ff5237d8b80 1 client.25930 shutdown 2023-07-21T22:23:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 msgr2=0x5623bc8cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8cfc90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff504099f30 tx=0x7ff504096410 comp rx=0 tx=0).stop 2023-07-21T22:23:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 shutdown_connections 2023-07-21T22:23:30.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc8dd0e0 0x5623bc8df4f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c8ac0 0x5623bc8cd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8cfc90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/2158748459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c81b0 0x5623bc8ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 >> 172.21.15.66:0/2158748459 conn(0x5623bc8b8ef0 msgr2=0x5623bc8be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff4f17fa700 0 client.25930 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 shutdown_connections 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 1 -- 172.21.15.66:0/2158748459 wait complete. 2023-07-21T22:23:30.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.517+0000 7ff5237d8b80 20 client.25930 trim_cache size 0 max 16384 2023-07-21T22:23:30.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 Processor -- start 2023-07-21T22:23:30.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 -- start start 2023-07-21T22:23:30.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c3930 0x5623bc8d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8be2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 0x5623bc8be810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc8c4730 con 0x5623bc8c81b0 2023-07-21T22:23:30.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc8bf350 con 0x5623bc8c3930 2023-07-21T22:23:30.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc8d4ac0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.537+0000 7ff50e39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 0x5623bc8be810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c3930 0x5623bc8d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 0x5623bc8be810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47444/0 (socket says 172.21.15.66:47444) 2023-07-21T22:23:30.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 -- 172.21.15.66:0/1216864411 learned_addr learned my addr 172.21.15.66:0/1216864411 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50d39a700 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8be2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 -- 172.21.15.66:0/1216864411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c3930 msgr2=0x5623bc8d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c3930 0x5623bc8d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 -- 172.21.15.66:0/1216864411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 msgr2=0x5623bc8be2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8be2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 -- 172.21.15.66:0/1216864411 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623bc669250 con 0x5623bc8c8ac0 2023-07-21T22:23:30.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50db9b700 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c3930 0x5623bc8d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff50e39c700 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 0x5623bc8be810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff500017a10 tx=0x7ff50001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94621d15196743c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.538+0000 7ff4fcbff700 1 -- 172.21.15.66:0/1216864411 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500004db0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff4fcbff700 1 -- 172.21.15.66:0/1216864411 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50001a600 con 0x5623bc8c8ac0 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff4fcbff700 1 -- 172.21.15.66:0/1216864411 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500004280 con 0x5623bc8c8ac0 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 -- 172.21.15.66:0/1216864411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 msgr2=0x5623bc8be810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 0x5623bc8be810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff500017a10 tx=0x7ff50001e000 comp rx=0 tx=0).stop 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 -- 172.21.15.66:0/1216864411 shutdown_connections 2023-07-21T22:23:30.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c3930 0x5623bc8d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c8ac0 0x5623bc8be810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/1216864411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8be2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 -- 172.21.15.66:0/1216864411 >> 172.21.15.66:0/1216864411 conn(0x5623bc8b8ef0 msgr2=0x5623bc8d8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 -- 172.21.15.66:0/1216864411 shutdown_connections 2023-07-21T22:23:30.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.539+0000 7ff5237d8b80 1 -- 172.21.15.66:0/1216864411 wait complete. 2023-07-21T22:23:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.540+0000 7ff5237d8b80 1 -- start start 2023-07-21T22:23:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.540+0000 7ff5237d8b80 1 Processor -- start 2023-07-21T22:23:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c8ac0 0x5623bc8d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc8d48d0 con 0x5623bc8c81b0 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc930fb0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bc9310f0 con 0x5623bc8c3930 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff5237d8b80 10 client.0 _ll_register_callbacks cb 0x7ff504005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50e39c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c8ac0 0x5623bc8d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50db9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47450/0 (socket says 172.21.15.66:47450) 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50db9b700 1 -- 172.21.15.66:0/3833840528 learned_addr learned my addr 172.21.15.66:0/3833840528 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50d39a700 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50e39c700 1 -- 172.21.15.66:0/3833840528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 msgr2=0x5623bc8d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50e39c700 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50e39c700 1 -- 172.21.15.66:0/3833840528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 msgr2=0x5623bc8d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50e39c700 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.541+0000 7ff50e39c700 1 -- 172.21.15.66:0/3833840528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623bc62bc60 con 0x5623bc8c8ac0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff50e39c700 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c8ac0 0x5623bc8d3410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff500004b10 tx=0x7ff500004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3911b9da951549 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff4f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff4f17fa700 1 -- 172.21.15.66:0/3833840528 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50001ab80 con 0x5623bc8c8ac0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5040a23f0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff4f17fa700 1 -- 172.21.15.66:0/3833840528 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff500010050 con 0x5623bc8c8ac0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff4f17fa700 1 -- 172.21.15.66:0/3833840528 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff50001ace0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff5237d8b80 10 client.25958 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff500017780 con 0x5623bc8c8ac0 2023-07-21T22:23:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.542+0000 7ff4f17fa700 1 -- 172.21.15.66:0/3833840528 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5040a23f0 con 0x5623bc8c8ac0 2023-07-21T22:23:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff4f17fa700 1 -- 172.21.15.66:0/3833840528 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff500037070 con 0x5623bc8c8ac0 2023-07-21T22:23:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff5237d8b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff5237d8b80 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:30.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff5237d8b80 20 client.25958 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:30.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc932780 0x5623bc934b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff5237d8b80 4 client.25958 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5623bc7d0050 con 0x5623bc932780 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff50db9b700 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc932780 0x5623bc934b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff50db9b700 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc932780 0x5623bc934b90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a34a872e5949c8 server_cookie=31307dda886be66a in_seq=0 out_seq=0 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.543+0000 7ff4f17fa700 10 client.25958 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff4f17fa700 1 -- 172.21.15.66:0/3833840528 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5623bc7d0050 con 0x5623bc932780 2023-07-21T22:23:30.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff4f17fa700 10 client.25958 handle_command_reply: tid=0 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 client.25958 shutdown 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c8ac0 msgr2=0x5623bc8d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c8ac0 0x5623bc8d3410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff500004b10 tx=0x7ff500004e50 comp rx=0 tx=0).stop 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 shutdown_connections 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5623bc8c8ac0 0x5623bc8d3410 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5623bc932780 0x5623bc934b90 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5623bc8c3930 0x5623bc8d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 --2- 172.21.15.66:0/3833840528 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5623bc8c81b0 0x5623bc8d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff4f17fa700 0 client.25958 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 >> 172.21.15.66:0/3833840528 conn(0x5623bc8b8ef0 msgr2=0x5623bc8be2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 shutdown_connections 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 1 -- 172.21.15.66:0/3833840528 wait complete. 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.544+0000 7ff5237d8b80 20 client.25958 trim_cache size 0 max 16384 2023-07-21T22:23:30.598 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:23:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:23:30.599 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:23:30.599 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4f40a1340 msgr2=0x7ff4f40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4f40a1340 0x7ff4f40a37f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff5000060b0 tx=0x7ff500013220 comp rx=0 tx=0).stop 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 msgr2=0x7ff508197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 0x7ff508197e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e920 tx=0x7ff50400ec30 comp rx=0 tx=0).stop 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 shutdown_connections 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5080728b0 0x7ff508197e10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4f40a1340 0x7ff4f40a37f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff50810c310 0x7ff50819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 --2- 172.21.15.66:0/2610930400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff5080731c0 0x7ff508198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 >> 172.21.15.66:0/2610930400 conn(0x7ff5080fb610 msgr2=0x7ff50810e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 shutdown_connections 2023-07-21T22:23:30.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.545+0000 7ff50fdff700 1 -- 172.21.15.66:0/2610930400 wait complete. 2023-07-21T22:23:30.613 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:23:30.613 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:23:30.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 -- 172.21.15.66:0/3424604012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54101fa0 msgr2=0x7f4d54102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 --2- 172.21.15.66:0/3424604012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54101fa0 0x7f4d54102370 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009920 tx=0x7f4d44009c30 comp rx=0 tx=0).stop 2023-07-21T22:23:30.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 -- 172.21.15.66:0/3424604012 shutdown_connections 2023-07-21T22:23:30.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 --2- 172.21.15.66:0/3424604012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54106fe0 0x7f4d5410ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 --2- 172.21.15.66:0/3424604012 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54101fa0 0x7f4d54102370 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 --2- 172.21.15.66:0/3424604012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54102940 0x7f4d54106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 -- 172.21.15.66:0/3424604012 >> 172.21.15.66:0/3424604012 conn(0x7f4d540fd7d0 msgr2=0x7f4d540ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:30.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 -- 172.21.15.66:0/3424604012 shutdown_connections 2023-07-21T22:23:30.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.782+0000 7f4d589ff700 1 -- 172.21.15.66:0/3424604012 wait complete. 2023-07-21T22:23:30.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.783+0000 7f4d589ff700 1 Processor -- start 2023-07-21T22:23:30.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.783+0000 7f4d589ff700 1 -- start start 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d589ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54101fa0 0x7f4d54197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d589ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 0x7f4d541984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d589ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54106fe0 0x7f4d5419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d589ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5419cf40 con 0x7f4d54102940 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d589ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5419d0b0 con 0x7f4d54106fe0 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d589ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5419d270 con 0x7f4d54101fa0 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d51d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 0x7f4d541984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d51d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 0x7f4d541984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36254/0 (socket says 172.21.15.66:36254) 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d51d9b700 1 -- 172.21.15.66:0/1311784958 learned_addr learned my addr 172.21.15.66:0/1311784958 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d52d9d700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54106fe0 0x7f4d5419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.784+0000 7f4d5259c700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54101fa0 0x7f4d54197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d51d9b700 1 -- 172.21.15.66:0/1311784958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54101fa0 msgr2=0x7f4d54197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d51d9b700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54101fa0 0x7f4d54197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d51d9b700 1 -- 172.21.15.66:0/1311784958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54106fe0 msgr2=0x7f4d5419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:30.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d51d9b700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54106fe0 0x7f4d5419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:30.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d51d9b700 1 -- 172.21.15.66:0/1311784958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44009520 con 0x7f4d54102940 2023-07-21T22:23:30.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d51d9b700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 0x7f4d541984a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e940 tx=0x7f4d4800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48009730 con 0x7f4d54102940 2023-07-21T22:23:30.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5419d550 con 0x7f4d54102940 2023-07-21T22:23:30.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d48010430 con 0x7f4d54102940 2023-07-21T22:23:30.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.785+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800cea0 con 0x7f4d54102940 2023-07-21T22:23:30.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.786+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d541a4d90 con 0x7f4d54102940 2023-07-21T22:23:30.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.787+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d54067750 con 0x7f4d54102940 2023-07-21T22:23:30.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.789+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4d48010810 con 0x7f4d54102940 2023-07-21T22:23:30.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.790+0000 7f4d437fe700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4d3c0a1300 0x7f4d3c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:30.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.790+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4d48014070 con 0x7f4d54102940 2023-07-21T22:23:30.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.791+0000 7f4d5259c700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4d3c0a1300 0x7f4d3c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:30.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.791+0000 7f4d5259c700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4d3c0a1300 0x7f4d3c0a37b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c010 tx=0x7f4d44019040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:30.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:30.793+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4d48096450 con 0x7f4d54102940 2023-07-21T22:23:31.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.053+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d541a4f80 con 0x7f4d54102940 2023-07-21T22:23:31.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.054+0000 7f4d437fe700 1 -- 172.21.15.66:0/1311784958 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f4d48099a70 con 0x7f4d54102940 2023-07-21T22:23:31.100 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:23:31.100 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:23:07.967297+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:23:31.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4d3c0a1300 msgr2=0x7f4d3c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4d3c0a1300 0x7f4d3c0a37b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400c010 tx=0x7f4d44019040 comp rx=0 tx=0).stop 2023-07-21T22:23:31.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 msgr2=0x7f4d541984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 0x7f4d541984a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e940 tx=0x7f4d4800ec50 comp rx=0 tx=0).stop 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 shutdown_connections 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54106fe0 0x7f4d5419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4d3c0a1300 0x7f4d3c0a37b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4d54101fa0 0x7f4d54197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 --2- 172.21.15.66:0/1311784958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4d54102940 0x7f4d541984a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 >> 172.21.15.66:0/1311784958 conn(0x7f4d540fd7d0 msgr2=0x7f4d541058e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 shutdown_connections 2023-07-21T22:23:31.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.056+0000 7f4d589ff700 1 -- 172.21.15.66:0/1311784958 wait complete. 2023-07-21T22:23:31.104 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 27 2023-07-21T22:23:31.121 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:23:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 -- 172.21.15.66:0/819176213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950101f60 msgr2=0x7f1950102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 --2- 172.21.15.66:0/819176213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950101f60 0x7f1950102330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f194c0099c0 tx=0x7f194c009cd0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 -- 172.21.15.66:0/819176213 shutdown_connections 2023-07-21T22:23:31.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 --2- 172.21.15.66:0/819176213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950106fa0 0x7f195010eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 --2- 172.21.15.66:0/819176213 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950101f60 0x7f1950102330 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 --2- 172.21.15.66:0/819176213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 0x7f19501068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 -- 172.21.15.66:0/819176213 >> 172.21.15.66:0/819176213 conn(0x7f19500fd7d0 msgr2=0x7f19500ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 -- 172.21.15.66:0/819176213 shutdown_connections 2023-07-21T22:23:31.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.295+0000 7f1957fff700 1 -- 172.21.15.66:0/819176213 wait complete. 2023-07-21T22:23:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.296+0000 7f1957fff700 1 Processor -- start 2023-07-21T22:23:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.296+0000 7f1957fff700 1 -- start start 2023-07-21T22:23:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1957fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 0x7f1950197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1957fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 0x7f1950198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1957fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950106fa0 0x7f195019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1957fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195019cea0 con 0x7f1950106fa0 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1957fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195019d010 con 0x7f1950101f60 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1957fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195019d1d0 con 0x7f1950102900 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 0x7f1950197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f195559a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 0x7f1950198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 0x7f1950197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:44980/0 (socket says 172.21.15.66:44980) 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f195659c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950106fa0 0x7f195019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f1955d9b700 1 -- 172.21.15.66:0/3870450463 learned_addr learned my addr 172.21.15.66:0/3870450463 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f195559a700 1 -- 172.21.15.66:0/3870450463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 msgr2=0x7f1950197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f195559a700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 0x7f1950197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f195559a700 1 -- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950106fa0 msgr2=0x7f195019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.297+0000 7f195559a700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950106fa0 0x7f195019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f195559a700 1 -- 172.21.15.66:0/3870450463 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0095f0 con 0x7f1950102900 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f1955d9b700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 0x7f1950197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f195659c700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950106fa0 0x7f195019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f195559a700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 0x7f1950198400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1950102cd0 tx=0x7f194000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f1946ffd700 1 -- 172.21.15.66:0/3870450463 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940010900 con 0x7f1950102900 2023-07-21T22:23:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195019d4b0 con 0x7f1950102900 2023-07-21T22:23:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f1946ffd700 1 -- 172.21.15.66:0/3870450463 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194000c430 con 0x7f1950102900 2023-07-21T22:23:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f1946ffd700 1 -- 172.21.15.66:0/3870450463 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1940018680 con 0x7f1950102900 2023-07-21T22:23:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.298+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19501a4cf0 con 0x7f1950102900 2023-07-21T22:23:31.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.302+0000 7f1946ffd700 1 -- 172.21.15.66:0/3870450463 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f194000c5a0 con 0x7f1950102900 2023-07-21T22:23:31.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 Processor -- start 2023-07-21T22:23:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 -- start start 2023-07-21T22:23:31.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f1946ffd700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f193c0a12c0 0x7f193c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f1946ffd700 1 -- 172.21.15.66:0/3870450463 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1940014070 con 0x7f1950102900 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d73350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d73da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67c78cb0 con 0x55ec67d78ac0 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67ccba40 con 0x55ec67d73930 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67cc4ed0 con 0x55ec67d781b0 2023-07-21T22:23:31.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.303+0000 7f195659c700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f193c0a12c0 0x7f193c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d78580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47514/0 (socket says 172.21.15.66:47514) 2023-07-21T22:23:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 -- 172.21.15.66:0/3614278759 learned_addr learned my addr 172.21.15.66:0/3614278759 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f195559a700 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d73350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f195659c700 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d73da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 -- 172.21.15.66:0/3614278759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 msgr2=0x55ec67d73da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d73da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 -- 172.21.15.66:0/3614278759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 msgr2=0x55ec67d73350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d73350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 -- 172.21.15.66:0/3614278759 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec67af2de0 con 0x55ec67d781b0 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f195659c700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f193c0a12c0 0x7f193c0a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1948004230 tx=0x7f19480045f0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.304+0000 7f1955d9b700 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d78580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f194c005bc0 tx=0x7f194c005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54c5add5a919ddc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f193bfff700 1 -- 172.21.15.66:0/3614278759 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00bec0 con 0x55ec67d781b0 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f193bfff700 1 -- 172.21.15.66:0/3614278759 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c01e040 con 0x55ec67d781b0 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f193bfff700 1 -- 172.21.15.66:0/3614278759 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c003e70 con 0x55ec67d781b0 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f196b8f9b80 1 -- 172.21.15.66:0/3614278759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 msgr2=0x55ec67d78580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d78580 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f194c005bc0 tx=0x7f194c005bf0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f196b8f9b80 1 -- 172.21.15.66:0/3614278759 shutdown_connections 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d73da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d78580 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.305+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/3614278759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d73350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.306+0000 7f196b8f9b80 1 -- 172.21.15.66:0/3614278759 >> 172.21.15.66:0/3614278759 conn(0x55ec67d68ef0 msgr2=0x55ec67d6e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.306+0000 7f196b8f9b80 1 -- 172.21.15.66:0/3614278759 shutdown_connections 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.306+0000 7f196b8f9b80 1 -- 172.21.15.66:0/3614278759 wait complete. 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.306+0000 7f196b8f9b80 1 -- start start 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.306+0000 7f196b8f9b80 1 Processor -- start 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d7faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d7cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67c64ac0 con 0x55ec67d78ac0 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67c647c0 con 0x55ec67d73930 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67b48920 con 0x55ec67d781b0 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f196b8f9b80 10 client.0 _ll_register_callbacks cb 0x55ec67c62760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d7faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195559a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d7cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d7faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45008/0 (socket says 172.21.15.66:45008) 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f1955d9b700 1 -- 172.21.15.66:0/4048438980 learned_addr learned my addr 172.21.15.66:0/4048438980 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195659c700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195559a700 1 -- 172.21.15.66:0/4048438980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 msgr2=0x55ec67d7faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195559a700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d7faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195559a700 1 -- 172.21.15.66:0/4048438980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 msgr2=0x55ec67d7d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195559a700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d7d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195559a700 1 -- 172.21.15.66:0/4048438980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195010f3b0 con 0x55ec67d781b0 2023-07-21T22:23:31.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f1955d9b700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d7faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.307+0000 7f195659c700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f195559a700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d7cd70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19400ca530 tx=0x7f1940099eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98493b8a7ea57d49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f19397fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f19397fa700 1 -- 172.21.15.66:0/4048438980 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400b1070 con 0x55ec67d781b0 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f19397fa700 1 -- 172.21.15.66:0/4048438980 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f19400a2300 con 0x55ec67d781b0 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec67c1e4f0 con 0x55ec67d781b0 2023-07-21T22:23:31.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f19397fa700 1 -- 172.21.15.66:0/4048438980 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19400b8170 con 0x55ec67d781b0 2023-07-21T22:23:31.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f196b8f9b80 10 client.25954 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:31.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec67adbc60 con 0x55ec67d781b0 2023-07-21T22:23:31.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f19397fa700 1 -- 172.21.15.66:0/4048438980 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19400c8080 con 0x55ec67d781b0 2023-07-21T22:23:31.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.308+0000 7f19397fa700 1 -- 172.21.15.66:0/4048438980 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f19400a1f00 con 0x55ec67d781b0 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f196b8f9b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f196b8f9b80 10 client.25954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f196b8f9b80 20 client.25954 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67d8d370 0x55ec67d8f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f196b8f9b80 4 client.25954 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec67cbdeb0 con 0x55ec67d8d370 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f1955d9b700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67d8d370 0x55ec67d8f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f1955d9b700 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67d8d370 0x55ec67d8f780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48161b8a267ce46e server_cookie=c93b5b238acee1aa in_seq=0 out_seq=0 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.309+0000 7f19397fa700 10 client.25954 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f19397fa700 1 -- 172.21.15.66:0/4048438980 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec67cbdeb0 con 0x55ec67d8d370 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f19397fa700 10 client.25954 handle_command_reply: tid=0 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 client.25954 shutdown 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 msgr2=0x55ec67d7cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d7cd70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f19400ca530 tx=0x7f1940099eb0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 shutdown_connections 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67d8d370 0x55ec67d8f780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d7faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/4048438980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d7cd70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f19397fa700 0 client.25954 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 >> 172.21.15.66:0/4048438980 conn(0x55ec67d68ef0 msgr2=0x55ec67d6e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.310+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 shutdown_connections 2023-07-21T22:23:31.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.311+0000 7f196b8f9b80 1 -- 172.21.15.66:0/4048438980 wait complete. 2023-07-21T22:23:31.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.311+0000 7f196b8f9b80 20 client.25954 trim_cache size 0 max 16384 2023-07-21T22:23:31.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 Processor -- start 2023-07-21T22:23:31.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 -- start start 2023-07-21T22:23:31.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d84520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d88cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d72a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67d6b010 con 0x55ec67d78ac0 2023-07-21T22:23:31.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67d66050 con 0x55ec67d73930 2023-07-21T22:23:31.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67d72590 con 0x55ec67d781b0 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d84520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f195559a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d88cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d84520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45010/0 (socket says 172.21.15.66:45010) 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f1955d9b700 1 -- 172.21.15.66:0/1771650608 learned_addr learned my addr 172.21.15.66:0/1771650608 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.325+0000 7f195659c700 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d72a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195559a700 1 -- 172.21.15.66:0/1771650608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 msgr2=0x55ec67d84520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195559a700 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d84520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195559a700 1 -- 172.21.15.66:0/1771650608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 msgr2=0x55ec67d72a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195559a700 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d72a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195559a700 1 -- 172.21.15.66:0/1771650608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec67b19250 con 0x55ec67d781b0 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195659c700 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d72a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f195559a700 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d88cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19400963b0 tx=0x7f194000ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4999fabb11fa01a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f1944ff9700 1 -- 172.21.15.66:0/1771650608 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400b1070 con 0x55ec67d781b0 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f1944ff9700 1 -- 172.21.15.66:0/1771650608 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec67c1e4f0 con 0x55ec67d781b0 2023-07-21T22:23:31.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.326+0000 7f1944ff9700 1 -- 172.21.15.66:0/1771650608 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19400a20f0 con 0x55ec67d781b0 2023-07-21T22:23:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 -- 172.21.15.66:0/1771650608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 msgr2=0x55ec67d88cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d88cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19400963b0 tx=0x7f194000ca90 comp rx=0 tx=0).stop 2023-07-21T22:23:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 -- 172.21.15.66:0/1771650608 shutdown_connections 2023-07-21T22:23:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67d84520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d78ac0 0x55ec67d72a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/1771650608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d781b0 0x55ec67d88cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 -- 172.21.15.66:0/1771650608 >> 172.21.15.66:0/1771650608 conn(0x55ec67d68ef0 msgr2=0x55ec67d6cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 -- 172.21.15.66:0/1771650608 shutdown_connections 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 -- 172.21.15.66:0/1771650608 wait complete. 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 -- start start 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.327+0000 7f196b8f9b80 1 Processor -- start 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67dd9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d781b0 0x55ec67dd9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d78ac0 0x55ec67dda4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67ddabe0 con 0x55ec67d781b0 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67ddad50 con 0x55ec67d73930 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec67ddaf10 con 0x55ec67d78ac0 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f196b8f9b80 10 client.0 _ll_register_callbacks cb 0x7f19400999f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67dd9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f195559a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d781b0 0x55ec67dd9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f1955d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67dd9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45024/0 (socket says 172.21.15.66:45024) 2023-07-21T22:23:31.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f1955d9b700 1 -- 172.21.15.66:0/446785157 learned_addr learned my addr 172.21.15.66:0/446785157 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.328+0000 7f195659c700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d78ac0 0x55ec67dda4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f1955d9b700 1 -- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d78ac0 msgr2=0x55ec67dda4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f1955d9b700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d78ac0 0x55ec67dda4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f1955d9b700 1 -- 172.21.15.66:0/446785157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d781b0 msgr2=0x55ec67dd9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f1955d9b700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d781b0 0x55ec67dd9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f1955d9b700 1 -- 172.21.15.66:0/446785157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec67adbc60 con 0x55ec67d73930 2023-07-21T22:23:31.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f195559a700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d781b0 0x55ec67dd9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f1955d9b700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67dd9760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb3b8b5d97be80c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f19397fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f19397fa700 1 -- 172.21.15.66:0/446785157 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c031d90 con 0x55ec67d73930 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f19397fa700 1 -- 172.21.15.66:0/446785157 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f194c00b570 con 0x55ec67d73930 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.329+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f194c01e040 con 0x55ec67d73930 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f19397fa700 1 -- 172.21.15.66:0/446785157 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194c031150 con 0x55ec67d73930 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 10 client.25982 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:31.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ec67af12b0 con 0x55ec67d73930 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f19397fa700 1 -- 172.21.15.66:0/446785157 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c01e040 con 0x55ec67d73930 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f19397fa700 1 -- 172.21.15.66:0/446785157 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f194c00f460 con 0x55ec67d73930 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 10 client.25982 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 10 client.25982 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 20 client.25982 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67de2250 0x55ec67de4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 4 client.25982 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:23:31.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec67c80050 con 0x55ec67de2250 2023-07-21T22:23:31.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.330+0000 7f195559a700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67de2250 0x55ec67de4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.331+0000 7f195559a700 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67de2250 0x55ec67de4630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe279f8d3e1d5e3d server_cookie=3e446fb604f32fc7 in_seq=0 out_seq=0 2023-07-21T22:23:31.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.331+0000 7f19397fa700 10 client.25982 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:23:31.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f19397fa700 1 -- 172.21.15.66:0/446785157 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec67c80050 con 0x55ec67de2250 2023-07-21T22:23:31.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f19397fa700 10 client.25982 handle_command_reply: tid=0 2023-07-21T22:23:31.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 client.25982 shutdown 2023-07-21T22:23:31.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 msgr2=0x55ec67dd9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67dd9760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c01a040 comp rx=0 tx=0).stop 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 shutdown_connections 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55ec67de2250 0x55ec67de4630 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ec67d73930 0x55ec67dd9760 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ec67d78ac0 0x55ec67dda4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 --2- 172.21.15.66:0/446785157 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ec67d781b0 0x55ec67dd9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f19397fa700 0 client.25982 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:23:31.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 >> 172.21.15.66:0/446785157 conn(0x55ec67d68ef0 msgr2=0x55ec67d88cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 shutdown_connections 2023-07-21T22:23:31.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 1 -- 172.21.15.66:0/446785157 wait complete. 2023-07-21T22:23:31.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.332+0000 7f196b8f9b80 20 client.25982 trim_cache size 0 max 16384 2023-07-21T22:23:31.386 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:23:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f193c0a12c0 msgr2=0x7f193c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f193c0a12c0 0x7f193c0a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1948004230 tx=0x7f19480045f0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 msgr2=0x7f1950198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 0x7f1950198400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1950102cd0 tx=0x7f194000eca0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 shutdown_connections 2023-07-21T22:23:31.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1950101f60 0x7f1950197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f193c0a12c0 0x7f193c0a3770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1950106fa0 0x7f195019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 --2- 172.21.15.66:0/3870450463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1950102900 0x7f1950198400 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 >> 172.21.15.66:0/3870450463 conn(0x7f19500fd7d0 msgr2=0x7f19501058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 shutdown_connections 2023-07-21T22:23:31.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.333+0000 7f1957fff700 1 -- 172.21.15.66:0/3870450463 wait complete. 2023-07-21T22:23:31.396 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:23:31.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32234 3.32 MB/sec warmup 58 sec latency 12.722 ms 2023-07-21T22:23:31.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.575+0000 7f0375fff700 1 -- 172.21.15.66:0/1633097767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f037010c310 msgr2=0x7f037010e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.575+0000 7f0375fff700 1 --2- 172.21.15.66:0/1633097767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f037010c310 0x7f037010e6f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f036800b0a0 tx=0x7f036800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 -- 172.21.15.66:0/1633097767 shutdown_connections 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 --2- 172.21.15.66:0/1633097767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f037010c310 0x7f037010e6f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 --2- 172.21.15.66:0/1633097767 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 --2- 172.21.15.66:0/1633097767 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03700731c0 0x7f037010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 -- 172.21.15.66:0/1633097767 >> 172.21.15.66:0/1633097767 conn(0x7f03700fb5f0 msgr2=0x7f03700fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 -- 172.21.15.66:0/1633097767 shutdown_connections 2023-07-21T22:23:31.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.576+0000 7f0375fff700 1 -- 172.21.15.66:0/1633097767 wait complete. 2023-07-21T22:23:31.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.577+0000 7f0375fff700 1 Processor -- start 2023-07-21T22:23:31.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.577+0000 7f0375fff700 1 -- start start 2023-07-21T22:23:31.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f0375fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f0375fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 0x7f0370198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f0375fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f037010c310 0x7f037019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f0375fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019cea0 con 0x7f03700728b0 2023-07-21T22:23:31.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f0375fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019d010 con 0x7f03700731c0 2023-07-21T22:23:31.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f0375fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019d1d0 con 0x7f037010c310 2023-07-21T22:23:31.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f036f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f036effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 0x7f0370198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f036f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36300/0 (socket says 172.21.15.66:36300) 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f036f7fe700 1 -- 172.21.15.66:0/3981372592 learned_addr learned my addr 172.21.15.66:0/3981372592 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f036effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 0x7f0370198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45038/0 (socket says 172.21.15.66:45038) 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.578+0000 7f036ffff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f037010c310 0x7f037019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036f7fe700 1 -- 172.21.15.66:0/3981372592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f037010c310 msgr2=0x7f037019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036f7fe700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f037010c310 0x7f037019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036f7fe700 1 -- 172.21.15.66:0/3981372592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 msgr2=0x7f0370198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036f7fe700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 0x7f0370198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036f7fe700 1 -- 172.21.15.66:0/3981372592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0360009520 con 0x7f03700728b0 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036effd700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 0x7f0370198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036ffff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f037010c310 0x7f037019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036f7fe700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370197ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f036000eb30 tx=0x7f036000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036cff9700 1 -- 172.21.15.66:0/3981372592 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360009880 con 0x7f03700728b0 2023-07-21T22:23:31.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0368009d00 con 0x7f03700728b0 2023-07-21T22:23:31.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036cff9700 1 -- 172.21.15.66:0/3981372592 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03600044e0 con 0x7f03700728b0 2023-07-21T22:23:31.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f036cff9700 1 -- 172.21.15.66:0/3981372592 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360003d80 con 0x7f03700728b0 2023-07-21T22:23:31.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.579+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037019d6a0 con 0x7f03700728b0 2023-07-21T22:23:31.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.583+0000 7f036cff9700 1 -- 172.21.15.66:0/3981372592 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0360003f20 con 0x7f03700728b0 2023-07-21T22:23:31.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.584+0000 7f038996fb80 1 Processor -- start 2023-07-21T22:23:31.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.584+0000 7f038996fb80 1 -- start start 2023-07-21T22:23:31.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.584+0000 7f038996fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b61b0 0x5655425b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f038996fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b6ac0 0x5655425b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f038996fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f038996fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655424b6cb0 con 0x5655425b6ac0 2023-07-21T22:23:31.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f038996fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x565542509a40 con 0x5655425b61b0 2023-07-21T22:23:31.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f038996fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565542502ed0 con 0x5655425b1930 2023-07-21T22:23:31.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b61b0 0x5655425b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b61b0 0x5655425b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45042/0 (socket says 172.21.15.66:45042) 2023-07-21T22:23:31.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036ffff700 1 -- 172.21.15.66:0/3577565986 learned_addr learned my addr 172.21.15.66:0/3577565986 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036f7fe700 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b6ac0 0x5655425b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036effd700 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036cff9700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036effd700 1 -- 172.21.15.66:0/3577565986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b61b0 msgr2=0x5655425b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036effd700 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b61b0 0x5655425b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036effd700 1 -- 172.21.15.66:0/3577565986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b6ac0 msgr2=0x5655425b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036effd700 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b6ac0 0x5655425b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.585+0000 7f036effd700 1 -- 172.21.15.66:0/3577565986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565542330de0 con 0x5655425b1930 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.586+0000 7f036cff9700 1 -- 172.21.15.66:0/3981372592 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0360014070 con 0x7f03700728b0 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.586+0000 7f036ffff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.586+0000 7f036effd700 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425b1da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0364009fb0 tx=0x7f0364011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19aba0adbed5d848 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.586+0000 7f0355ffb700 1 -- 172.21.15.66:0/3577565986 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364004230 con 0x5655425b1930 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.586+0000 7f0355ffb700 1 -- 172.21.15.66:0/3577565986 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0364004cf0 con 0x5655425b1930 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.586+0000 7f0355ffb700 1 -- 172.21.15.66:0/3577565986 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364003680 con 0x5655425b1930 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f036ffff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f036800b0a0 tx=0x7f03680099e0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 -- 172.21.15.66:0/3577565986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 msgr2=0x5655425b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425b1da0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0364009fb0 tx=0x7f0364011720 comp rx=0 tx=0).stop 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 -- 172.21.15.66:0/3577565986 shutdown_connections 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b61b0 0x5655425b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425b1da0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 --2- 172.21.15.66:0/3577565986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b6ac0 0x5655425b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 -- 172.21.15.66:0/3577565986 >> 172.21.15.66:0/3577565986 conn(0x5655425a6ef0 msgr2=0x5655425ac830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 -- 172.21.15.66:0/3577565986 shutdown_connections 2023-07-21T22:23:31.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.587+0000 7f038996fb80 1 -- 172.21.15.66:0/3577565986 wait complete. 2023-07-21T22:23:31.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.588+0000 7f038996fb80 1 -- start start 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.588+0000 7f038996fb80 1 Processor -- start 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425bdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b61b0 0x5655425bae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b6ac0 0x5655425bb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655424a2ac0 con 0x5655425b61b0 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655424a27c0 con 0x5655425b6ac0 2023-07-21T22:23:31.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565542386920 con 0x5655425b1930 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f038996fb80 10 client.0 _ll_register_callbacks cb 0x5655424a0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f036f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425bdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f036f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425bdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47576/0 (socket says 172.21.15.66:47576) 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f036f7fe700 1 -- 172.21.15.66:0/1348096643 learned_addr learned my addr 172.21.15.66:0/1348096643 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f036effd700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b61b0 0x5655425bae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.589+0000 7f036ffff700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b6ac0 0x5655425bb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f036f7fe700 1 -- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b6ac0 msgr2=0x5655425bb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f036f7fe700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b6ac0 0x5655425bb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f036f7fe700 1 -- 172.21.15.66:0/1348096643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b61b0 msgr2=0x5655425bae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f036f7fe700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b61b0 0x5655425bae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f036f7fe700 1 -- 172.21.15.66:0/1348096643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0364009c90 con 0x5655425b1930 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f036f7fe700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425bdb90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f03600044b0 tx=0x7f0360099fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1236cc3f4e5bdd4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f034effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f034effd700 1 -- 172.21.15.66:0/1348096643 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036009a100 con 0x5655425b1930 2023-07-21T22:23:31.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f034effd700 1 -- 172.21.15.66:0/1348096643 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f036001e2e0 con 0x5655425b1930 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56554245c4f0 con 0x5655425b1930 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f034effd700 1 -- 172.21.15.66:0/1348096643 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f036000cbd0 con 0x5655425b1930 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f038996fb80 10 client.25972 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x565542319c60 con 0x5655425b1930 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.590+0000 7f034effd700 1 -- 172.21.15.66:0/1348096643 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56554245c4f0 con 0x5655425b1930 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f034effd700 1 -- 172.21.15.66:0/1348096643 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f03600cac30 con 0x5655425b1930 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f038996fb80 10 client.25972 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:31.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f038996fb80 10 client.25972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f038996fb80 20 client.25972 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f038996fb80 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425cb4e0 0x5655425cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f038996fb80 4 client.25972 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5655424fbeb0 con 0x5655425cb4e0 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.591+0000 7f036effd700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425cb4e0 0x5655425cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.592+0000 7f036effd700 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425cb4e0 0x5655425cd8c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94a00c615c9bdcae server_cookie=f0cea96b5c72c4b8 in_seq=0 out_seq=0 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.592+0000 7f034effd700 10 client.25972 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:23:31.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.592+0000 7f034effd700 1 -- 172.21.15.66:0/1348096643 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5655424fbeb0 con 0x5655425cb4e0 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.592+0000 7f034effd700 10 client.25972 handle_command_reply: tid=0 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.592+0000 7f038996fb80 1 client.25972 shutdown 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 msgr2=0x5655425bdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425bdb90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f03600044b0 tx=0x7f0360099fa0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 shutdown_connections 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425cb4e0 0x5655425cd8c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425b6ac0 0x5655425bb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f034effd700 0 client.25972 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b1930 0x5655425bdb90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 --2- 172.21.15.66:0/1348096643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b61b0 0x5655425bae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 >> 172.21.15.66:0/1348096643 conn(0x5655425a6ef0 msgr2=0x5655425ac420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 shutdown_connections 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 1 -- 172.21.15.66:0/1348096643 wait complete. 2023-07-21T22:23:31.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.593+0000 7f038996fb80 20 client.25972 trim_cache size 0 max 16384 2023-07-21T22:23:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.607+0000 7f038996fb80 1 Processor -- start 2023-07-21T22:23:31.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.607+0000 7f038996fb80 1 -- start start 2023-07-21T22:23:31.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f038996fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f038996fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425ba610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f038996fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f038996fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655425b24a0 con 0x5655425b1930 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f038996fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655425bc3f0 con 0x5655425a6ef0 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f038996fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655425b5e00 con 0x5655425b61b0 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425bab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47588/0 (socket says 172.21.15.66:47588) 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036ffff700 1 -- 172.21.15.66:0/2184509970 learned_addr learned my addr 172.21.15.66:0/2184509970 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425ba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36322/0 (socket says 172.21.15.66:36322) 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036f7fe700 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 -- 172.21.15.66:0/2184509970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 msgr2=0x5655425bab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425bab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 -- 172.21.15.66:0/2184509970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 msgr2=0x5655425bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425bd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.608+0000 7f036effd700 1 -- 172.21.15.66:0/2184509970 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565542357250 con 0x5655425b1930 2023-07-21T22:23:31.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.609+0000 7f036effd700 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425ba610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f0364004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e80a922a7c03b3f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.609+0000 7f03567fc700 1 -- 172.21.15.66:0/2184509970 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640050e0 con 0x5655425b1930 2023-07-21T22:23:31.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.609+0000 7f03567fc700 1 -- 172.21.15.66:0/2184509970 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03640054a0 con 0x5655425b1930 2023-07-21T22:23:31.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.609+0000 7f03567fc700 1 -- 172.21.15.66:0/2184509970 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364019690 con 0x5655425b1930 2023-07-21T22:23:31.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 -- 172.21.15.66:0/2184509970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 msgr2=0x5655425ba610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425ba610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0364000c00 tx=0x7f0364004e60 comp rx=0 tx=0).stop 2023-07-21T22:23:31.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 -- 172.21.15.66:0/2184509970 shutdown_connections 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425bab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 --2- 172.21.15.66:0/2184509970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425ba610 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 -- 172.21.15.66:0/2184509970 >> 172.21.15.66:0/2184509970 conn(0x5655425b2060 msgr2=0x5655425af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 -- 172.21.15.66:0/2184509970 shutdown_connections 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 -- 172.21.15.66:0/2184509970 wait complete. 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.610+0000 7f038996fb80 1 -- start start 2023-07-21T22:23:31.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 Processor -- start 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425c0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655425c10a0 con 0x5655425b1930 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655425c1210 con 0x5655425a6ef0 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655425c13d0 con 0x5655425b61b0 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f038996fb80 10 client.0 _ll_register_callbacks cb 0x7f0360003720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:31.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f036f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f036f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45072/0 (socket says 172.21.15.66:45072) 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f036f7fe700 1 -- 172.21.15.66:0/828998561 learned_addr learned my addr 172.21.15.66:0/828998561 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f036effd700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425c0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.611+0000 7f036ffff700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f036f7fe700 1 -- 172.21.15.66:0/828998561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 msgr2=0x5655425c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f036f7fe700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f036f7fe700 1 -- 172.21.15.66:0/828998561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 msgr2=0x5655425c0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f036f7fe700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425c0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f036f7fe700 1 -- 172.21.15.66:0/828998561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036401d040 con 0x5655425a6ef0 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f036f7fe700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425c2360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f036009a2c0 tx=0x7f0360005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240f6a0acfb8aed2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f034effd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f034effd700 1 -- 172.21.15.66:0/828998561 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600a24f0 con 0x5655425a6ef0 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f034effd700 1 -- 172.21.15.66:0/828998561 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f036001e2e0 con 0x5655425a6ef0 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0364004cf0 con 0x5655425a6ef0 2023-07-21T22:23:31.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f034effd700 1 -- 172.21.15.66:0/828998561 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0360099c90 con 0x5655425a6ef0 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f038996fb80 10 client.26000 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.612+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56554232f2b0 con 0x5655425a6ef0 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f034effd700 1 -- 172.21.15.66:0/828998561 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0364004cf0 con 0x5655425a6ef0 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f034effd700 1 -- 172.21.15.66:0/828998561 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0360099820 con 0x5655425a6ef0 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f038996fb80 10 client.26000 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f038996fb80 10 client.26000 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f038996fb80 20 client.26000 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:31.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f038996fb80 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425b6ac0 0x565542622270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f038996fb80 4 client.26000 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:23:31.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5655424be050 con 0x5655425b6ac0 2023-07-21T22:23:31.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.613+0000 7f036effd700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425b6ac0 0x565542622270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f036effd700 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425b6ac0 0x565542622270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b42a9101dc84b693 server_cookie=6ec1187fd35c4eda in_seq=0 out_seq=0 2023-07-21T22:23:31.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f034effd700 10 client.26000 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:23:31.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f034effd700 1 -- 172.21.15.66:0/828998561 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5655424be050 con 0x5655425b6ac0 2023-07-21T22:23:31.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f034effd700 10 client.26000 handle_command_reply: tid=0 2023-07-21T22:23:31.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 client.26000 shutdown 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 msgr2=0x5655425c2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425c2360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f036009a2c0 tx=0x7f0360005b40 comp rx=0 tx=0).stop 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 shutdown_connections 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5655425b6ac0 0x565542622270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5655425a6ef0 0x5655425c2360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5655425b61b0 0x5655425c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 --2- 172.21.15.66:0/828998561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5655425b1930 0x5655425c0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f034effd700 0 client.26000 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:23:31.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 >> 172.21.15.66:0/828998561 conn(0x5655425b2060 msgr2=0x5655425b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 shutdown_connections 2023-07-21T22:23:31.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.614+0000 7f038996fb80 1 -- 172.21.15.66:0/828998561 wait complete. 2023-07-21T22:23:31.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f038996fb80 20 client.26000 trim_cache size 0 max 16384 2023-07-21T22:23:31.668 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:23:31.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 msgr2=0x7f03580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f036800b0a0 tx=0x7f03680099e0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 msgr2=0x7f0370197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370197ec0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f036000eb30 tx=0x7f036000ee40 comp rx=0 tx=0).stop 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 shutdown_connections 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f037010c310 0x7f037019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700728b0 0x7f0370197ec0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 --2- 172.21.15.66:0/3981372592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700731c0 0x7f0370198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 >> 172.21.15.66:0/3981372592 conn(0x7f03700fb5f0 msgr2=0x7f0370103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 shutdown_connections 2023-07-21T22:23:31.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.615+0000 7f0375fff700 1 -- 172.21.15.66:0/3981372592 wait complete. 2023-07-21T22:23:31.679 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:23:31.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 -- 172.21.15.66:0/4159985896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84105ae0 msgr2=0x7f1a84105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 --2- 172.21.15.66:0/4159985896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84105ae0 0x7f1a84105eb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009990 tx=0x7f1a74009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 -- 172.21.15.66:0/4159985896 shutdown_connections 2023-07-21T22:23:31.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 --2- 172.21.15.66:0/4159985896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84105ae0 0x7f1a84105eb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 --2- 172.21.15.66:0/4159985896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a84100480 0x7f1a8410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 --2- 172.21.15.66:0/4159985896 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 0x7f1a840fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 -- 172.21.15.66:0/4159985896 >> 172.21.15.66:0/4159985896 conn(0x7f1a840fb5f0 msgr2=0x7f1a840fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 -- 172.21.15.66:0/4159985896 shutdown_connections 2023-07-21T22:23:31.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.848+0000 7f1a89bff700 1 -- 172.21.15.66:0/4159985896 wait complete. 2023-07-21T22:23:31.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.849+0000 7f1a89bff700 1 Processor -- start 2023-07-21T22:23:31.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.849+0000 7f1a89bff700 1 -- start start 2023-07-21T22:23:31.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a89bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 0x7f1a84072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a89bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 0x7f1a84072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a89bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a84105ae0 0x7f1a84073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a89bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841a8560 con 0x7f1a840ffad0 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a89bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841a86a0 con 0x7f1a84100480 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a89bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841a87e0 con 0x7f1a84105ae0 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 0x7f1a84072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 0x7f1a84072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 0x7f1a84072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45092/0 (socket says 172.21.15.66:45092) 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2066341941 learned_addr learned my addr 172.21.15.66:0/2066341941 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.850+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a84105ae0 0x7f1a84073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2066341941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a84105ae0 msgr2=0x7f1a84073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a84105ae0 0x7f1a84073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 msgr2=0x7f1a84072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 0x7f1a84072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2066341941 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a740095f0 con 0x7f1a84100480 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a837fe700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 0x7f1a84072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 0x7f1a84072d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e9a0 tx=0x7f1a7800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a80ff9700 1 -- 172.21.15.66:0/2066341941 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78009790 con 0x7f1a84100480 2023-07-21T22:23:31.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a80ff9700 1 -- 172.21.15.66:0/2066341941 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a78010430 con 0x7f1a84100480 2023-07-21T22:23:31.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a841a8ac0 con 0x7f1a84100480 2023-07-21T22:23:31.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a80ff9700 1 -- 172.21.15.66:0/2066341941 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a78018680 con 0x7f1a84100480 2023-07-21T22:23:31.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.851+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a841a9020 con 0x7f1a84100480 2023-07-21T22:23:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.855+0000 7f1a80ff9700 1 -- 172.21.15.66:0/2066341941 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1a780105a0 con 0x7f1a84100480 2023-07-21T22:23:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.856+0000 7f1a9d5e0b80 1 Processor -- start 2023-07-21T22:23:31.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.856+0000 7f1a9d5e0b80 1 -- start start 2023-07-21T22:23:31.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a80ff9700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1a6c0a1340 0x7f1a6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8a6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe7abcb0 con 0x561efe8ab1b0 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe7fea40 con 0x561efe8abac0 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe7f7ed0 con 0x561efe8a6930 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a80ff9700 1 -- 172.21.15.66:0/2066341941 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1a78014070 con 0x7f1a84100480 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1a6c0a1340 0x7f1a6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45108/0 (socket says 172.21.15.66:45108) 2023-07-21T22:23:31.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2628704643 learned_addr learned my addr 172.21.15.66:0/2628704643 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a837fe700 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8a6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2628704643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 msgr2=0x561efe8a6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.857+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8a6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2628704643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 msgr2=0x561efe8ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2628704643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efe625de0 con 0x561efe8abac0 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a837fe700 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a6350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a780a1e60 tx=0x7f1a78099e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25a85b3c0993ab4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1a6c0a1340 0x7f1a6c0a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009ec0 tx=0x7f1a7c009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a69ffb700 1 -- 172.21.15.66:0/2628704643 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7800b4a0 con 0x561efe8abac0 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a69ffb700 1 -- 172.21.15.66:0/2628704643 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7800bcf0 con 0x561efe8abac0 2023-07-21T22:23:31.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.858+0000 7f1a69ffb700 1 -- 172.21.15.66:0/2628704643 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a780ca840 con 0x561efe8abac0 2023-07-21T22:23:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2628704643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 msgr2=0x561efe8a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a6350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a780a1e60 tx=0x7f1a78099e20 comp rx=0 tx=0).stop 2023-07-21T22:23:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2628704643 shutdown_connections 2023-07-21T22:23:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8a6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2628704643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a6350 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2628704643 >> 172.21.15.66:0/2628704643 conn(0x561efe89bef0 msgr2=0x561efe8a1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2628704643 shutdown_connections 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2628704643 wait complete. 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.859+0000 7f1a9d5e0b80 1 -- start start 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 Processor -- start 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8a6930 0x561efe8afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8abac0 0x561efe8b0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe797ac0 con 0x561efe8a6930 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe7977c0 con 0x561efe8ab1b0 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe67b920 con 0x561efe8abac0 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.860+0000 7f1a9d5e0b80 10 client.0 _ll_register_callbacks cb 0x561efe795760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45110/0 (socket says 172.21.15.66:45110) 2023-07-21T22:23:31.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a82ffd700 1 -- 172.21.15.66:0/2472011991 learned_addr learned my addr 172.21.15.66:0/2472011991 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8abac0 0x561efe8b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a837fe700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8a6930 0x561efe8afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 -- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 msgr2=0x561efe8b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 -- 172.21.15.66:0/2472011991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8a6930 msgr2=0x561efe8afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8a6930 0x561efe8afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 -- 172.21.15.66:0/2472011991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a841a9020 con 0x561efe8abac0 2023-07-21T22:23:31.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a83fff700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8abac0 0x561efe8b0970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009fd0 tx=0x7f1a7c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853f872286860bb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.861+0000 7f1a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a62ffd700 1 -- 172.21.15.66:0/2472011991 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c020070 con 0x561efe8abac0 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561efe7514f0 con 0x561efe8abac0 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a62ffd700 1 -- 172.21.15.66:0/2472011991 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1a7c01a680 con 0x561efe8abac0 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a62ffd700 1 -- 172.21.15.66:0/2472011991 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a7c00e070 con 0x561efe8abac0 2023-07-21T22:23:31.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a9d5e0b80 10 client.25990 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561efe60ec60 con 0x561efe8abac0 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.862+0000 7f1a62ffd700 1 -- 172.21.15.66:0/2472011991 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561efe7514f0 con 0x561efe8abac0 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a62ffd700 1 -- 172.21.15.66:0/2472011991 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1a7c025a30 con 0x561efe8abac0 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a9d5e0b80 10 client.25990 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a9d5e0b80 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a9d5e0b80 20 client.25990 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe8c0680 0x561efe8c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a9d5e0b80 4 client.25990 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:31.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561efe7f0eb0 con 0x561efe8c0680 2023-07-21T22:23:31.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.863+0000 7f1a837fe700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe8c0680 0x561efe8c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.864+0000 7f1a62ffd700 10 client.25990 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:31.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.864+0000 7f1a837fe700 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe8c0680 0x561efe8c2a90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53fdf7c33d668a9 server_cookie=a3d02fe462b23920 in_seq=0 out_seq=0 2023-07-21T22:23:31.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a62ffd700 1 -- 172.21.15.66:0/2472011991 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561efe7f0eb0 con 0x561efe8c0680 2023-07-21T22:23:31.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a62ffd700 10 client.25990 handle_command_reply: tid=0 2023-07-21T22:23:31.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 client.25990 shutdown 2023-07-21T22:23:31.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8abac0 msgr2=0x561efe8b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8abac0 0x561efe8b0970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009fd0 tx=0x7f1a7c004b40 comp rx=0 tx=0).stop 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 shutdown_connections 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe8c0680 0x561efe8c2a90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8abac0 0x561efe8b0970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8a6930 0x561efe8afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a62ffd700 0 client.25990 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/2472011991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 >> 172.21.15.66:0/2472011991 conn(0x561efe89bef0 msgr2=0x561efe8a13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 shutdown_connections 2023-07-21T22:23:31.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/2472011991 wait complete. 2023-07-21T22:23:31.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.865+0000 7f1a9d5e0b80 20 client.25990 trim_cache size 0 max 16384 2023-07-21T22:23:31.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.879+0000 7f1a9d5e0b80 1 Processor -- start 2023-07-21T22:23:31.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.879+0000 7f1a9d5e0b80 1 -- start start 2023-07-21T22:23:31.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe8bdc30 con 0x561efe8ab1b0 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe89e010 con 0x561efe8abac0 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe8a76d0 con 0x561efe8a6930 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47648/0 (socket says 172.21.15.66:47648) 2023-07-21T22:23:31.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45126/0 (socket says 172.21.15.66:45126) 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a837fe700 1 -- 172.21.15.66:0/3570673929 learned_addr learned my addr 172.21.15.66:0/3570673929 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36364/0 (socket says 172.21.15.66:36364) 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 -- 172.21.15.66:0/3570673929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 msgr2=0x561efe8b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 -- 172.21.15.66:0/3570673929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 msgr2=0x561efe8a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a83fff700 1 -- 172.21.15.66:0/3570673929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561efe8afa60 con 0x561efe8abac0 2023-07-21T22:23:31.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.880+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.881+0000 7f1a83fff700 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a4fa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c02f040 tx=0x7f1a7c0092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c8f9f86603d742 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.881+0000 7f1a6a7fc700 1 -- 172.21.15.66:0/3570673929 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c020070 con 0x561efe8abac0 2023-07-21T22:23:31.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.881+0000 7f1a6a7fc700 1 -- 172.21.15.66:0/3570673929 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7c00f040 con 0x561efe8abac0 2023-07-21T22:23:31.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.881+0000 7f1a6a7fc700 1 -- 172.21.15.66:0/3570673929 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c01a940 con 0x561efe8abac0 2023-07-21T22:23:31.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.881+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/3570673929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 msgr2=0x561efe8a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.881+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a4fa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c02f040 tx=0x7f1a7c0092f0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/3570673929 shutdown_connections 2023-07-21T22:23:31.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8abac0 0x561efe8a4fa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/3570673929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8ab1b0 0x561efe8a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/3570673929 >> 172.21.15.66:0/3570673929 conn(0x561efe89bef0 msgr2=0x561efe8a6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/3570673929 shutdown_connections 2023-07-21T22:23:31.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/3570673929 wait complete. 2023-07-21T22:23:31.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 -- start start 2023-07-21T22:23:31.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.882+0000 7f1a9d5e0b80 1 Processor -- start 2023-07-21T22:23:31.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8abac0 0x561efe8b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe9136e0 con 0x561efe8abac0 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe913850 con 0x561efe8ab1b0 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561efe913a10 con 0x561efe8a6930 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a9d5e0b80 10 client.0 _ll_register_callbacks cb 0x7f1a7c008d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a82ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47658/0 (socket says 172.21.15.66:47658) 2023-07-21T22:23:31.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 -- 172.21.15.66:0/1043172495 learned_addr learned my addr 172.21.15.66:0/1043172495 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a83fff700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8abac0 0x561efe8b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 -- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 msgr2=0x561efe8b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 -- 172.21.15.66:0/1043172495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8abac0 msgr2=0x561efe8b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8abac0 0x561efe8b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a837fe700 1 -- 172.21.15.66:0/1043172495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561efe625de0 con 0x561efe8a6930 2023-07-21T22:23:31.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.883+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a837fe700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b3550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400be10 tx=0x7f1a7400be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc5c9240d19d5ec9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a62ffd700 1 -- 172.21.15.66:0/1043172495 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7401d040 con 0x561efe8a6930 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a62ffd700 1 -- 172.21.15.66:0/1043172495 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1a74028110 con 0x561efe8a6930 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a7800bcf0 con 0x561efe8a6930 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a62ffd700 1 -- 172.21.15.66:0/1043172495 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a740282d0 con 0x561efe8a6930 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a9d5e0b80 10 client.25996 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:31.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561efe60ec60 con 0x561efe8a6930 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.884+0000 7f1a62ffd700 1 -- 172.21.15.66:0/1043172495 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a7800bcf0 con 0x561efe8a6930 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a62ffd700 1 -- 172.21.15.66:0/1043172495 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1a74004860 con 0x561efe8a6930 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a9d5e0b80 10 client.25996 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a9d5e0b80 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a9d5e0b80 20 client.25996 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe9151f0 0x561efe917600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:31.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a9d5e0b80 4 client.25996 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561efe7b3050 con 0x561efe9151f0 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe9151f0 0x561efe917600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a62ffd700 10 client.25996 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.885+0000 7f1a82ffd700 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe9151f0 0x561efe917600 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9421bef018d9f4 server_cookie=3647b06f6002c9b5 in_seq=0 out_seq=0 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a62ffd700 1 -- 172.21.15.66:0/1043172495 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561efe7b3050 con 0x561efe9151f0 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a62ffd700 10 client.25996 handle_command_reply: tid=0 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 client.25996 shutdown 2023-07-21T22:23:31.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 msgr2=0x561efe8b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b3550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1a7400be10 tx=0x7f1a7400be40 comp rx=0 tx=0).stop 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 shutdown_connections 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561efe9151f0 0x561efe917600 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561efe8abac0 0x561efe8b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561efe8a6930 0x561efe8b3550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 --2- 172.21.15.66:0/1043172495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561efe8ab1b0 0x561efe8b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a62ffd700 0 client.25996 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:31.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 >> 172.21.15.66:0/1043172495 conn(0x561efe89bef0 msgr2=0x561efe8a4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 shutdown_connections 2023-07-21T22:23:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 1 -- 172.21.15.66:0/1043172495 wait complete. 2023-07-21T22:23:31.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.886+0000 7f1a9d5e0b80 20 client.25996 trim_cache size 0 max 16384 2023-07-21T22:23:31.942 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:23:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1a6c0a1340 msgr2=0x7f1a6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1a6c0a1340 0x7f1a6c0a37f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009ec0 tx=0x7f1a7c009920 comp rx=0 tx=0).stop 2023-07-21T22:23:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 msgr2=0x7f1a84072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:31.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 0x7f1a84072d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e9a0 tx=0x7f1a7800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 shutdown_connections 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1a6c0a1340 0x7f1a6c0a37f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1a84105ae0 0x7f1a84073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1a840ffad0 0x7f1a84072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 --2- 172.21.15.66:0/2066341941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1a84100480 0x7f1a84072d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 >> 172.21.15.66:0/2066341941 conn(0x7f1a840fb5f0 msgr2=0x7f1a840fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 shutdown_connections 2023-07-21T22:23:31.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:31.887+0000 7f1a89bff700 1 -- 172.21.15.66:0/2066341941 wait complete. 2023-07-21T22:23:31.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:31 smithi066 ceph-mon[126379]: pgmap v64: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 275 KiB/s wr, 53 op/s 2023-07-21T22:23:31.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:31 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1311784958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:23:32.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:31 smithi149 ceph-mon[129268]: pgmap v64: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 275 KiB/s wr, 53 op/s 2023-07-21T22:23:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:31 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1311784958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:23:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:31 smithi104 ceph-mon[129021]: pgmap v64: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 275 KiB/s wr, 53 op/s 2023-07-21T22:23:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:31 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1311784958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:23:32.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32631 3.27 MB/sec warmup 59 sec latency 512.972 ms 2023-07-21T22:23:32.951 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:23:32.951 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:23:33.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 -- 172.21.15.66:0/1606444452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 msgr2=0x7f64081105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 --2- 172.21.15.66:0/1606444452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f64081105a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009990 tx=0x7f63f4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 -- 172.21.15.66:0/1606444452 shutdown_connections 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 --2- 172.21.15.66:0/1606444452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6408110ae0 0x7f6408112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 --2- 172.21.15.66:0/1606444452 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64080728b0 0x7f6408072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 --2- 172.21.15.66:0/1606444452 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f64081105a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 -- 172.21.15.66:0/1606444452 >> 172.21.15.66:0/1606444452 conn(0x7f64080fb5f0 msgr2=0x7f64080fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 -- 172.21.15.66:0/1606444452 shutdown_connections 2023-07-21T22:23:33.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.129+0000 7f640ebff700 1 -- 172.21.15.66:0/1606444452 wait complete. 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.130+0000 7f640ebff700 1 Processor -- start 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.130+0000 7f640ebff700 1 -- start start 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640ebff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64080728b0 0x7f640819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640ebff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f640819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640ebff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6408110ae0 0x7f64081a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640ebff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081a12a0 con 0x7f64080731c0 2023-07-21T22:23:33.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640ebff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081a1410 con 0x7f64080728b0 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640ebff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081a15d0 con 0x7f6408110ae0 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f640819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f640819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51164/0 (socket says 172.21.15.66:51164) 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 -- 172.21.15.66:0/2642850031 learned_addr learned my addr 172.21.15.66:0/2642850031 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640c99b700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64080728b0 0x7f640819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f640d19c700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6408110ae0 0x7f64081a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 -- 172.21.15.66:0/2642850031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6408110ae0 msgr2=0x7f64081a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6408110ae0 0x7f64081a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 -- 172.21.15.66:0/2642850031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64080728b0 msgr2=0x7f640819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64080728b0 0x7f640819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.131+0000 7f6407fff700 1 -- 172.21.15.66:0/2642850031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40095f0 con 0x7f64080731c0 2023-07-21T22:23:33.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.132+0000 7f6407fff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f640819c800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005910 tx=0x7f63f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.132+0000 7f6405ffb700 1 -- 172.21.15.66:0/2642850031 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f401d070 con 0x7f64080731c0 2023-07-21T22:23:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.132+0000 7f6405ffb700 1 -- 172.21.15.66:0/2642850031 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f4022470 con 0x7f64080731c0 2023-07-21T22:23:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.132+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64081a1850 con 0x7f64080731c0 2023-07-21T22:23:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.132+0000 7f6405ffb700 1 -- 172.21.15.66:0/2642850031 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4017650 con 0x7f64080731c0 2023-07-21T22:23:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.132+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64081a90f0 con 0x7f64080731c0 2023-07-21T22:23:33.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.135+0000 7f6405ffb700 1 -- 172.21.15.66:0/2642850031 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f63f400fad0 con 0x7f64080731c0 2023-07-21T22:23:33.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 Processor -- start 2023-07-21T22:23:33.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 -- start start 2023-07-21T22:23:33.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f6405ffb700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63f80a1340 0x7f63f80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270230 0x55de2b270600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b26b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b26b9b0 0x55de2b26be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b170d30 con 0x55de2b26b9b0 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b1c3ac0 con 0x55de2b270b40 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f642251cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b1bcf50 con 0x55de2b270230 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.137+0000 7f6405ffb700 1 -- 172.21.15.66:0/2642850031 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f63f40ce460 con 0x7f64080731c0 2023-07-21T22:23:33.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f640c99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270230 0x55de2b270600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b26b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f640d19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b26b9b0 0x55de2b26be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b26b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:44784/0 (socket says 172.21.15.66:44784) 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 -- 172.21.15.66:0/1553421061 learned_addr learned my addr 172.21.15.66:0/1553421061 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f640d19c700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63f80a1340 0x7f63f80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 -- 172.21.15.66:0/1553421061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270230 msgr2=0x55de2b270600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270230 0x55de2b270600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 -- 172.21.15.66:0/1553421061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b26b9b0 msgr2=0x55de2b26be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b26b9b0 0x55de2b26be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 -- 172.21.15.66:0/1553421061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de2afeae60 con 0x55de2b270b40 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f640c99b700 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270230 0x55de2b270600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f6407fff700 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b26b3d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63f409b200 tx=0x7f63f409b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13d01b32b8134bb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.138+0000 7f640d19c700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63f80a1340 0x7f63f80a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6400009550 tx=0x7f6400009100 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f63f2ffd700 1 -- 172.21.15.66:0/1553421061 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40048c0 con 0x55de2b270b40 2023-07-21T22:23:33.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f63f2ffd700 1 -- 172.21.15.66:0/1553421061 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f40046e0 con 0x55de2b270b40 2023-07-21T22:23:33.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f63f2ffd700 1 -- 172.21.15.66:0/1553421061 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40a36c0 con 0x55de2b270b40 2023-07-21T22:23:33.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 -- 172.21.15.66:0/1553421061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 msgr2=0x55de2b26b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b26b3d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63f409b200 tx=0x7f63f409b040 comp rx=0 tx=0).stop 2023-07-21T22:23:33.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 -- 172.21.15.66:0/1553421061 shutdown_connections 2023-07-21T22:23:33.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b26b9b0 0x55de2b26be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270230 0x55de2b270600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 --2- 172.21.15.66:0/1553421061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b26b3d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 -- 172.21.15.66:0/1553421061 >> 172.21.15.66:0/1553421061 conn(0x55de2b260f70 msgr2=0x55de2b2668b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 -- 172.21.15.66:0/1553421061 shutdown_connections 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.139+0000 7f642251cb80 1 -- 172.21.15.66:0/1553421061 wait complete. 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.140+0000 7f642251cb80 1 -- start start 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.140+0000 7f642251cb80 1 Processor -- start 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 0x55de2b277b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b274e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270b40 0x55de2b275390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b15cb40 con 0x55de2b270230 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b15c840 con 0x55de2b26b9b0 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b0409a0 con 0x55de2b270b40 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f642251cb80 10 client.0 _ll_register_callbacks cb 0x55de2b15a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 0x55de2b277b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 0x55de2b277b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:44788/0 (socket says 172.21.15.66:44788) 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 -- 172.21.15.66:0/4224906789 learned_addr learned my addr 172.21.15.66:0/4224906789 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f6407fff700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b274e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640d19c700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270b40 0x55de2b275390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 -- 172.21.15.66:0/4224906789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270b40 msgr2=0x55de2b275390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270b40 0x55de2b275390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 -- 172.21.15.66:0/4224906789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 msgr2=0x55de2b274e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b274e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.141+0000 7f640c99b700 1 -- 172.21.15.66:0/4224906789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc0098e0 con 0x55de2b26b9b0 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f640c99b700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 0x55de2b277b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63fc012cf0 tx=0x7f63fc009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adaa6cece90382d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f63e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f63e7fff700 1 -- 172.21.15.66:0/4224906789 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc017070 con 0x55de2b26b9b0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de2b116570 con 0x55de2b26b9b0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f63e7fff700 1 -- 172.21.15.66:0/4224906789 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f63fc004020 con 0x55de2b26b9b0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f63e7fff700 1 -- 172.21.15.66:0/4224906789 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63fc009e90 con 0x55de2b26b9b0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f642251cb80 10 client.26030 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.142+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f64081a90f0 con 0x55de2b26b9b0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f63e7fff700 1 -- 172.21.15.66:0/4224906789 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de2b116570 con 0x55de2b26b9b0 2023-07-21T22:23:33.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f63e7fff700 1 -- 172.21.15.66:0/4224906789 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f63fc003680 con 0x55de2b26b9b0 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f642251cb80 10 client.26030 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f642251cb80 10 client.26030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f642251cb80 20 client.26030 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f642251cb80 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2851a0 0x55de2b2875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f642251cb80 4 client.26030 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de2b1b5f30 con 0x55de2b2851a0 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.143+0000 7f6407fff700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2851a0 0x55de2b2875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.144+0000 7f6407fff700 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2851a0 0x55de2b2875b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b4ab0cc5c1080bb server_cookie=a6bf4d7697f2bffc in_seq=0 out_seq=0 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.144+0000 7f63e7fff700 10 client.26030 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.144+0000 7f63e7fff700 1 -- 172.21.15.66:0/4224906789 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de2b1b5f30 con 0x55de2b2851a0 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.144+0000 7f63e7fff700 10 client.26030 handle_command_reply: tid=0 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.144+0000 7f642251cb80 1 client.26030 shutdown 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 msgr2=0x55de2b277b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 0x55de2b277b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f63fc012cf0 tx=0x7f63fc009d00 comp rx=0 tx=0).stop 2023-07-21T22:23:33.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 shutdown_connections 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b26b9b0 0x55de2b277b20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2851a0 0x55de2b2875b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b270b40 0x55de2b275390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 --2- 172.21.15.66:0/4224906789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b274e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 >> 172.21.15.66:0/4224906789 conn(0x55de2b260f70 msgr2=0x55de2b266440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f63e7fff700 0 client.26030 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 shutdown_connections 2023-07-21T22:23:33.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 1 -- 172.21.15.66:0/4224906789 wait complete. 2023-07-21T22:23:33.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.145+0000 7f642251cb80 20 client.26030 trim_cache size 0 max 16384 2023-07-21T22:23:33.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.163+0000 7f642251cb80 1 Processor -- start 2023-07-21T22:23:33.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.163+0000 7f642251cb80 1 -- start start 2023-07-21T22:23:33.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.163+0000 7f642251cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b26b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.163+0000 7f642251cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b2776c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f642251cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b274660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f642251cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b2763e0 con 0x55de2b270230 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f642251cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b265690 con 0x55de2b270b40 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f642251cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b282880 con 0x55de2b26b9b0 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b2776c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f640d19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b274660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b2776c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51190/0 (socket says 172.21.15.66:51190) 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 -- 172.21.15.66:0/552098750 learned_addr learned my addr 172.21.15.66:0/552098750 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f640c99b700 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b26b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 -- 172.21.15.66:0/552098750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 msgr2=0x55de2b26b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b26b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 -- 172.21.15.66:0/552098750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 msgr2=0x55de2b274660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b274660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 -- 172.21.15.66:0/552098750 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de2afd3ce0 con 0x55de2b270230 2023-07-21T22:23:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.164+0000 7f6407fff700 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b2776c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f63f409af70 tx=0x7f63f4022c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b5a30e96f2ef48 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f63f37fe700 1 -- 172.21.15.66:0/552098750 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4004580 con 0x55de2b270230 2023-07-21T22:23:33.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f63f37fe700 1 -- 172.21.15.66:0/552098750 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f409b270 con 0x55de2b270230 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f63f37fe700 1 -- 172.21.15.66:0/552098750 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4099610 con 0x55de2b270230 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 -- 172.21.15.66:0/552098750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 msgr2=0x55de2b2776c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b2776c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f63f409af70 tx=0x7f63f4022c20 comp rx=0 tx=0).stop 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 -- 172.21.15.66:0/552098750 shutdown_connections 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b274660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b26b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 --2- 172.21.15.66:0/552098750 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b2776c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 -- 172.21.15.66:0/552098750 >> 172.21.15.66:0/552098750 conn(0x55de2b260f70 msgr2=0x55de2b26a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 -- 172.21.15.66:0/552098750 shutdown_connections 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.165+0000 7f642251cb80 1 -- 172.21.15.66:0/552098750 wait complete. 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.166+0000 7f642251cb80 1 -- start start 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.166+0000 7f642251cb80 1 Processor -- start 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b27b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b27b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b2d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b27be30 con 0x55de2b270230 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b2d95b0 con 0x55de2b270b40 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de2b2d9770 con 0x55de2b26b9b0 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f642251cb80 10 client.0 _ll_register_callbacks cb 0x7f63fc0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b27b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f640c99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b27b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b27b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51194/0 (socket says 172.21.15.66:51194) 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 -- 172.21.15.66:0/2789045658 learned_addr learned my addr 172.21.15.66:0/2789045658 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f640d19c700 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b2d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 -- 172.21.15.66:0/2789045658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 msgr2=0x55de2b27b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b27b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 -- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 msgr2=0x55de2b2d9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b2d9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.167+0000 7f6407fff700 1 -- 172.21.15.66:0/2789045658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de2afeae60 con 0x55de2b270230 2023-07-21T22:23:33.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f6407fff700 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b27b6a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006010 tx=0x7f63f4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee92edac261b7e92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f63e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f63e7fff700 1 -- 172.21.15.66:0/2789045658 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4019040 con 0x55de2b270230 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f63e7fff700 1 -- 172.21.15.66:0/2789045658 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f63f400fd80 con 0x55de2b270230 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63f40046e0 con 0x55de2b270230 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f63e7fff700 1 -- 172.21.15.66:0/2789045658 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f4099610 con 0x55de2b270230 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f642251cb80 10 client.15987 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55de2b0112d0 con 0x55de2b270230 2023-07-21T22:23:33.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f63e7fff700 1 -- 172.21.15.66:0/2789045658 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63f40046e0 con 0x55de2b270230 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f63e7fff700 1 -- 172.21.15.66:0/2789045658 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f63f409b4d0 con 0x55de2b270230 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f642251cb80 10 client.15987 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f642251cb80 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.168+0000 7f642251cb80 20 client.15987 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.169+0000 7f642251cb80 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2dabf0 0x55de2b2dd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.169+0000 7f642251cb80 4 client.15987 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:23:33.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.169+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55de2b1780d0 con 0x55de2b2dabf0 2023-07-21T22:23:33.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.169+0000 7f640c99b700 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2dabf0 0x55de2b2dd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.169+0000 7f640c99b700 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2dabf0 0x55de2b2dd000 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2fad1b7f5f79d7 server_cookie=70c73b9c8d1a0caa in_seq=0 out_seq=0 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.169+0000 7f63e7fff700 10 client.15987 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.170+0000 7f63e7fff700 1 -- 172.21.15.66:0/2789045658 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de2b1780d0 con 0x55de2b2dabf0 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.170+0000 7f63e7fff700 10 client.15987 handle_command_reply: tid=0 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.170+0000 7f642251cb80 1 client.15987 shutdown 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 msgr2=0x55de2b27b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b27b6a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f63f4006010 tx=0x7f63f4004580 comp rx=0 tx=0).stop 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 shutdown_connections 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55de2b270b40 0x55de2b2d9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55de2b2dabf0 0x55de2b2dd000 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55de2b26b9b0 0x55de2b27b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 --2- 172.21.15.66:0/2789045658 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55de2b270230 0x55de2b27b6a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f63e7fff700 0 client.15987 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 >> 172.21.15.66:0/2789045658 conn(0x55de2b260f70 msgr2=0x55de2b26b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 shutdown_connections 2023-07-21T22:23:33.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 1 -- 172.21.15.66:0/2789045658 wait complete. 2023-07-21T22:23:33.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.171+0000 7f642251cb80 20 client.15987 trim_cache size 0 max 16384 2023-07-21T22:23:33.224 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:23:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:23:33.224 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "6b224387-14a8-49ee-8822-8727fcd9d90d", 2023-07-21T22:23:33.225 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:23:33.225 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:23:33.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63f80a1340 msgr2=0x7f63f80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63f80a1340 0x7f63f80a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6400009550 tx=0x7f6400009100 comp rx=0 tx=0).stop 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 msgr2=0x7f640819c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f640819c800 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005910 tx=0x7f63f4005740 comp rx=0 tx=0).stop 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 shutdown_connections 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64080728b0 0x7f640819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63f80a1340 0x7f63f80a37f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6408110ae0 0x7f64081a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 --2- 172.21.15.66:0/2642850031 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f64080731c0 0x7f640819c800 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 >> 172.21.15.66:0/2642850031 conn(0x7f64080fb5f0 msgr2=0x7f64080fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 shutdown_connections 2023-07-21T22:23:33.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.172+0000 7f640ebff700 1 -- 172.21.15.66:0/2642850031 wait complete. 2023-07-21T22:23:33.236 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:23:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 -- 172.21.15.66:0/4202286704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8940ffad0 msgr2=0x7fa8940fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/4202286704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8940ffad0 0x7fa8940fff40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009990 tx=0x7fa87c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 -- 172.21.15.66:0/4202286704 shutdown_connections 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/4202286704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 0x7fa89410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/4202286704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa894105ae0 0x7fa894105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/4202286704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa8940ffad0 0x7fa8940fff40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 -- 172.21.15.66:0/4202286704 >> 172.21.15.66:0/4202286704 conn(0x7fa8940fb5f0 msgr2=0x7fa8940fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 -- 172.21.15.66:0/4202286704 shutdown_connections 2023-07-21T22:23:33.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.406+0000 7fa89a5ff700 1 -- 172.21.15.66:0/4202286704 wait complete. 2023-07-21T22:23:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.407+0000 7fa89a5ff700 1 Processor -- start 2023-07-21T22:23:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 -- start start 2023-07-21T22:23:33.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 0x7fa894197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 0x7fa894198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa894105ae0 0x7fa89419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419ce00 con 0x7fa894105ae0 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419cf70 con 0x7fa894100480 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa89a5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419d130 con 0x7fa8940ffad0 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa893fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 0x7fa894197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa893fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 0x7fa894197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37158/0 (socket says 172.21.15.66:37158) 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa893fff700 1 -- 172.21.15.66:0/1331795665 learned_addr learned my addr 172.21.15.66:0/1331795665 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 0x7fa894198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.408+0000 7fa898b9c700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa894105ae0 0x7fa89419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa893fff700 1 -- 172.21.15.66:0/1331795665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 msgr2=0x7fa894198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa893fff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 0x7fa894198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa893fff700 1 -- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa894105ae0 msgr2=0x7fa89419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa893fff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa894105ae0 0x7fa89419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa893fff700 1 -- 172.21.15.66:0/1331795665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0095f0 con 0x7fa8940ffad0 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa88bfff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 0x7fa894198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa898b9c700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa894105ae0 0x7fa89419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa893fff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 0x7fa894197e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa88400e9a0 tx=0x7fa88400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa891ffb700 1 -- 172.21.15.66:0/1331795665 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884009790 con 0x7fa8940ffad0 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89419d410 con 0x7fa8940ffad0 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa891ffb700 1 -- 172.21.15.66:0/1331795665 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa884010430 con 0x7fa8940ffad0 2023-07-21T22:23:33.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.409+0000 7fa891ffb700 1 -- 172.21.15.66:0/1331795665 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884018680 con 0x7fa8940ffad0 2023-07-21T22:23:33.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.410+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8941a4c50 con 0x7fa8940ffad0 2023-07-21T22:23:33.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.413+0000 7fa891ffb700 1 -- 172.21.15.66:0/1331795665 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa8840105a0 con 0x7fa8940ffad0 2023-07-21T22:23:33.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.414+0000 7fa8adfc3b80 1 Processor -- start 2023-07-21T22:23:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.414+0000 7fa8adfc3b80 1 -- start start 2023-07-21T22:23:33.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa891ffb700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa8800a12c0 0x7fa8800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa891ffb700 1 -- 172.21.15.66:0/1331795665 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa884014070 con 0x7fa8940ffad0 2023-07-21T22:23:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 0x55738d014580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa898b9c700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa8800a12c0 0x7fa8800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d014ac0 0x55738d00f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d00fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf14cb0 con 0x55738d00f930 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf67a40 con 0x55738d014ac0 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf60ed0 con 0x55738d0141b0 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa893fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 0x55738d014580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa898b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d00fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d014ac0 0x55738d00f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa893fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 0x55738d014580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37164/0 (socket says 172.21.15.66:37164) 2023-07-21T22:23:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa893fff700 1 -- 172.21.15.66:0/2225462771 learned_addr learned my addr 172.21.15.66:0/2225462771 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.415+0000 7fa898b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d00fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51232/0 (socket says 172.21.15.66:51232) 2023-07-21T22:23:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88bfff700 1 -- 172.21.15.66:0/2225462771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 msgr2=0x55738d014580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88bfff700 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 0x55738d014580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88bfff700 1 -- 172.21.15.66:0/2225462771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 msgr2=0x55738d00fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88bfff700 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d00fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88bfff700 1 -- 172.21.15.66:0/2225462771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55738cd8ede0 con 0x55738d014ac0 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa893fff700 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 0x55738d014580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa898b9c700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa8800a12c0 0x7fa8800a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa88c004230 tx=0x7fa88c0045f0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88bfff700 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d014ac0 0x55738d00f350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0059d0 tx=0x7fa87c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=643b6ce761ecb45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88a7fc700 1 -- 172.21.15.66:0/2225462771 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c01d030 con 0x55738d014ac0 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88a7fc700 1 -- 172.21.15.66:0/2225462771 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87c01e040 con 0x55738d014ac0 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.416+0000 7fa88a7fc700 1 -- 172.21.15.66:0/2225462771 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c004c40 con 0x55738d014ac0 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2225462771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d014ac0 msgr2=0x55738d00f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d014ac0 0x55738d00f350 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0059d0 tx=0x7fa87c00b610 comp rx=0 tx=0).stop 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2225462771 shutdown_connections 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d00fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d0141b0 0x55738d014580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2225462771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d014ac0 0x55738d00f350 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2225462771 >> 172.21.15.66:0/2225462771 conn(0x55738d004ef0 msgr2=0x55738d00a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2225462771 shutdown_connections 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.417+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2225462771 wait complete. 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 -- start start 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 Processor -- start 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d018ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d0141b0 0x55738d0193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d014ac0 0x55738d019920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf00ac0 con 0x55738d00f930 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cf007c0 con 0x55738d0141b0 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.418+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738cde4920 con 0x55738d014ac0 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa8adfc3b80 10 client.0 _ll_register_callbacks cb 0x55738cefe760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d018ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d018ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51240/0 (socket says 172.21.15.66:51240) 2023-07-21T22:23:33.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 -- 172.21.15.66:0/3443051536 learned_addr learned my addr 172.21.15.66:0/3443051536 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa88bfff700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d0141b0 0x55738d0193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa898b9c700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d014ac0 0x55738d019920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 -- 172.21.15.66:0/3443051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d014ac0 msgr2=0x55738d019920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d014ac0 0x55738d019920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 -- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d0141b0 msgr2=0x55738d0193e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d0141b0 0x55738d0193e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 -- 172.21.15.66:0/3443051536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8941a4c50 con 0x55738d00f930 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa893fff700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d018ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa884096420 tx=0x7fa8840984d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad7a279fa7ab8a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.419+0000 7fa8737fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8737fe700 1 -- 172.21.15.66:0/3443051536 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88409a380 con 0x55738d00f930 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55738ceba4f0 con 0x55738d00f930 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8737fe700 1 -- 172.21.15.66:0/3443051536 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa88409df50 con 0x55738d00f930 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8737fe700 1 -- 172.21.15.66:0/3443051536 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8840a2370 con 0x55738d00f930 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 10 client.15999 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:33.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa89410d210 con 0x55738d00f930 2023-07-21T22:23:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8737fe700 1 -- 172.21.15.66:0/3443051536 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55738ceba4f0 con 0x55738d00f930 2023-07-21T22:23:33.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8737fe700 1 -- 172.21.15.66:0/3443051536 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa8840a1ee0 con 0x55738d00f930 2023-07-21T22:23:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 20 client.15999 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0294e0 0x55738d02b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 4 client.15999 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:23:33.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.420+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55738cf59eb0 con 0x55738d0294e0 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.421+0000 7fa88bfff700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0294e0 0x55738d02b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.421+0000 7fa8737fe700 10 client.15999 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.421+0000 7fa88bfff700 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0294e0 0x55738d02b8f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e085cd615ed0a60 server_cookie=88d09f9ccdfd909e in_seq=0 out_seq=0 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.422+0000 7fa8737fe700 1 -- 172.21.15.66:0/3443051536 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55738cf59eb0 con 0x55738d0294e0 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.422+0000 7fa8737fe700 10 client.15999 handle_command_reply: tid=0 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.422+0000 7fa8adfc3b80 1 client.15999 shutdown 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 msgr2=0x55738d018ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d018ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa884096420 tx=0x7fa8840984d0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 shutdown_connections 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0294e0 0x55738d02b8f0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d014ac0 0x55738d019920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d00f930 0x55738d018ea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8737fe700 0 client.15999 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/3443051536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d0141b0 0x55738d0193e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 >> 172.21.15.66:0/3443051536 conn(0x55738d004ef0 msgr2=0x55738d00a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 shutdown_connections 2023-07-21T22:23:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/3443051536 wait complete. 2023-07-21T22:23:33.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.423+0000 7fa8adfc3b80 20 client.15999 trim_cache size 0 max 16384 2023-07-21T22:23:33.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 Processor -- start 2023-07-21T22:23:33.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 -- start start 2023-07-21T22:23:33.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 0x55738d00f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d004ef0 0x55738d019750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d017d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738d01bf50 con 0x55738d025af0 2023-07-21T22:23:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738d026b90 con 0x55738d004ef0 2023-07-21T22:23:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738d007010 con 0x55738d00f930 2023-07-21T22:23:33.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa893fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 0x55738d00f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa893fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 0x55738d00f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51244/0 (socket says 172.21.15.66:51244) 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.441+0000 7fa893fff700 1 -- 172.21.15.66:0/1241610844 learned_addr learned my addr 172.21.15.66:0/1241610844 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa88bfff700 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d004ef0 0x55738d019750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa898b9c700 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d017d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa893fff700 1 -- 172.21.15.66:0/1241610844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 msgr2=0x55738d017d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa893fff700 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d017d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa893fff700 1 -- 172.21.15.66:0/1241610844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d004ef0 msgr2=0x55738d019750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa893fff700 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d004ef0 0x55738d019750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa893fff700 1 -- 172.21.15.66:0/1241610844 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55738d025ec0 con 0x55738d025af0 2023-07-21T22:23:33.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa893fff700 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 0x55738d00f2b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa88409a0e0 tx=0x7fa884099fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481fb67093879e99 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa88affd700 1 -- 172.21.15.66:0/1241610844 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884098ed0 con 0x55738d025af0 2023-07-21T22:23:33.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa88affd700 1 -- 172.21.15.66:0/1241610844 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa88409df50 con 0x55738d025af0 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.442+0000 7fa88affd700 1 -- 172.21.15.66:0/1241610844 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840b8350 con 0x55738d025af0 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/1241610844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 msgr2=0x55738d00f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 0x55738d00f2b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa88409a0e0 tx=0x7fa884099fa0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/1241610844 shutdown_connections 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d017d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d025af0 0x55738d00f2b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/1241610844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d004ef0 0x55738d019750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/1241610844 >> 172.21.15.66:0/1241610844 conn(0x55738d025020 msgr2=0x55738d017670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/1241610844 shutdown_connections 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/1241610844 wait complete. 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.443+0000 7fa8adfc3b80 1 -- start start 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 Processor -- start 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 0x55738d01ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d01f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d025af0 0x55738d01f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738d07d400 con 0x55738d004ef0 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738d07d540 con 0x55738d025af0 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55738d07d700 con 0x55738d00f930 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa8adfc3b80 10 client.0 _ll_register_callbacks cb 0x7fa884099a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa893fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 0x55738d01ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa893fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 0x55738d01ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51254/0 (socket says 172.21.15.66:51254) 2023-07-21T22:23:33.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa893fff700 1 -- 172.21.15.66:0/2765605045 learned_addr learned my addr 172.21.15.66:0/2765605045 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.444+0000 7fa898b9c700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d025af0 0x55738d01f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa88bfff700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d01f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa893fff700 1 -- 172.21.15.66:0/2765605045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 msgr2=0x55738d01f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa893fff700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d01f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa893fff700 1 -- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d025af0 msgr2=0x55738d01f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa893fff700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d025af0 0x55738d01f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa893fff700 1 -- 172.21.15.66:0/2765605045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89410d210 con 0x55738d004ef0 2023-07-21T22:23:33.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa893fff700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 0x55738d01ecc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa88400e970 tx=0x7fa884099cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369dd71f0eb5c748 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8737fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8737fe700 1 -- 172.21.15.66:0/2765605045 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840b8370 con 0x55738d004ef0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8737fe700 1 -- 172.21.15.66:0/2765605045 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa8840a2370 con 0x55738d004ef0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa87c01e040 con 0x55738d004ef0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8737fe700 1 -- 172.21.15.66:0/2765605045 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8840187e0 con 0x55738d004ef0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8adfc3b80 10 client.16005 fetch_fsmap learned FSMap version 27 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.445+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55738d009230 con 0x55738d004ef0 2023-07-21T22:23:33.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8737fe700 1 -- 172.21.15.66:0/2765605045 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa87c01e040 con 0x55738d004ef0 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8737fe700 1 -- 172.21.15.66:0/2765605045 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 27) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa8840990e0 con 0x55738d004ef0 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8adfc3b80 10 client.16005 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8adfc3b80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8adfc3b80 20 client.16005 populate_metadata read hostname 'smithi066' 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0141b0 0x55738d0809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8adfc3b80 4 client.16005 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:23:33.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55738cf1c050 con 0x55738d0141b0 2023-07-21T22:23:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.446+0000 7fa88bfff700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0141b0 0x55738d0809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:23:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa88bfff700 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0141b0 0x55738d0809d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b5e956eafe5189 server_cookie=66d7995a0dcd13c0 in_seq=0 out_seq=0 2023-07-21T22:23:33.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8737fe700 10 client.16005 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8737fe700 1 -- 172.21.15.66:0/2765605045 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55738cf1c050 con 0x55738d0141b0 2023-07-21T22:23:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8737fe700 10 client.16005 handle_command_reply: tid=0 2023-07-21T22:23:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 client.16005 shutdown 2023-07-21T22:23:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 msgr2=0x55738d01ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 0x55738d01ecc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa88400e970 tx=0x7fa884099cf0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 shutdown_connections 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55738d025af0 0x55738d01f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55738d0141b0 0x55738d0809d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55738d004ef0 0x55738d01ecc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 --2- 172.21.15.66:0/2765605045 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55738d00f930 0x55738d01f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8737fe700 0 client.16005 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 >> 172.21.15.66:0/2765605045 conn(0x55738d025020 msgr2=0x55738d00f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.447+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 shutdown_connections 2023-07-21T22:23:33.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa8adfc3b80 1 -- 172.21.15.66:0/2765605045 wait complete. 2023-07-21T22:23:33.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa8adfc3b80 20 client.16005 trim_cache size 0 max 16384 2023-07-21T22:23:33.502 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:23:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:23:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:23:33.502 INFO:teuthology.orchestra.run.smithi066.stdout: "6b224387-14a8-49ee-8822-8727fcd9d90d": { 2023-07-21T22:23:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:23:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "6b224387-14a8-49ee-8822-8727fcd9d90d", 2023-07-21T22:23:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:23:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:23:33.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:23:33.503 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:23:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa8800a12c0 msgr2=0x7fa8800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa8800a12c0 0x7fa8800a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa88c004230 tx=0x7fa88c0045f0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 msgr2=0x7fa894197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:23:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 0x7fa894197e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa88400e9a0 tx=0x7fa88400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 shutdown_connections 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa8800a12c0 0x7fa8800a3770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa894105ae0 0x7fa89419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa8940ffad0 0x7fa894197e20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 --2- 172.21.15.66:0/1331795665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa894100480 0x7fa894198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 >> 172.21.15.66:0/1331795665 conn(0x7fa8940fb5f0 msgr2=0x7fa89410af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 shutdown_connections 2023-07-21T22:23:33.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:23:33.448+0000 7fa89a5ff700 1 -- 172.21.15.66:0/1331795665 wait complete. 2023-07-21T22:23:33.512 INFO:tasks.cephfs.filesystem:scrub status for tag:6b224387-14a8-49ee-8822-8727fcd9d90d - {'path': '/', 'tag': '6b224387-14a8-49ee-8822-8727fcd9d90d', 'options': 'recursive,force'} 2023-07-21T22:23:33.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32687 3.22 MB/sec warmup 60 sec latency 1241.839 ms 2023-07-21T22:23:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: pgmap v65: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 238 KiB/s wr, 45 op/s 2023-07-21T22:23:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:23:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:33 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: pgmap v65: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 238 KiB/s wr, 45 op/s 2023-07-21T22:23:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:23:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:34.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:34.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:34.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:33 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: pgmap v65: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 238 KiB/s wr, 45 op/s 2023-07-21T22:23:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:23:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:33 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:34.512 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32687 3.17 MB/sec warmup 61 sec latency 1223.167 ms 2023-07-21T22:23:35.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:34 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:23:35.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:34 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:35.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:34 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:23:35.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:34 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:35.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:34 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:23:35.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:34 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:23:35.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32755 3.12 MB/sec warmup 62 sec latency 1884.760 ms 2023-07-21T22:23:36.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:35 smithi066 ceph-mon[126379]: pgmap v66: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 587 KiB/s wr, 87 op/s 2023-07-21T22:23:36.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:35 smithi149 ceph-mon[129268]: pgmap v66: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 587 KiB/s wr, 87 op/s 2023-07-21T22:23:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:35 smithi104 ceph-mon[129021]: pgmap v66: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 587 KiB/s wr, 87 op/s 2023-07-21T22:23:36.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32878 3.08 MB/sec warmup 63 sec latency 687.212 ms 2023-07-21T22:23:37.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:36 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:37.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:36 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:37.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:36 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:23:37.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32878 3.03 MB/sec warmup 64 sec latency 1487.961 ms 2023-07-21T22:23:38.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:37 smithi066 ceph-mon[126379]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 393 KiB/s wr, 51 op/s 2023-07-21T22:23:38.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:37 smithi149 ceph-mon[129268]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 393 KiB/s wr, 51 op/s 2023-07-21T22:23:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:37 smithi104 ceph-mon[129021]: pgmap v67: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 393 KiB/s wr, 51 op/s 2023-07-21T22:23:38.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32944 2.98 MB/sec warmup 65 sec latency 1502.433 ms 2023-07-21T22:23:39.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:38 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:39.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:38 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:38 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:23:39.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32944 2.94 MB/sec warmup 66 sec latency 1931.091 ms 2023-07-21T22:23:40.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 393 KiB/s wr, 51 op/s 2023-07-21T22:23:40.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:40.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:40.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 393 KiB/s wr, 51 op/s 2023-07-21T22:23:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:40.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: pgmap v68: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 393 KiB/s wr, 51 op/s 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:23:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:23:40.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32944 2.89 MB/sec warmup 67 sec latency 2931.134 ms 2023-07-21T22:23:41.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:40 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:23:41.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:40 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:23:41.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:40 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:23:41.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32944 2.85 MB/sec warmup 68 sec latency 3931.258 ms 2023-07-21T22:23:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:41 smithi066 ceph-mon[126379]: pgmap v69: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 453 KiB/s wr, 63 op/s 2023-07-21T22:23:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:41 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:23:42.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:41 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:23:42.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:41 smithi149 ceph-mon[129268]: pgmap v69: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 453 KiB/s wr, 63 op/s 2023-07-21T22:23:42.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:41 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:23:42.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:41 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:23:42.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:41 smithi104 ceph-mon[129021]: pgmap v69: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 453 KiB/s wr, 63 op/s 2023-07-21T22:23:42.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:41 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:23:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:41 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:23:42.513 INFO:tasks.workunit.client.0.smithi066.stdout: 1 32944 2.81 MB/sec warmup 69 sec latency 4931.471 ms 2023-07-21T22:23:43.514 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34724 2.92 MB/sec warmup 70 sec latency 4942.472 ms 2023-07-21T22:23:44.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:43 smithi066 ceph-mon[126379]: pgmap v70: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 408 KiB/s wr, 53 op/s 2023-07-21T22:23:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:43 smithi149 ceph-mon[129268]: pgmap v70: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 408 KiB/s wr, 53 op/s 2023-07-21T22:23:44.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:43 smithi104 ceph-mon[129021]: pgmap v70: 129 pgs: 129 active+clean; 37 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 408 KiB/s wr, 53 op/s 2023-07-21T22:23:44.514 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34724 2.87 MB/sec warmup 71 sec latency 1345.840 ms 2023-07-21T22:23:45.514 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34724 2.83 MB/sec warmup 72 sec latency 2345.913 ms 2023-07-21T22:23:46.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:45 smithi066 ceph-mon[126379]: pgmap v71: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 486 KiB/s wr, 70 op/s 2023-07-21T22:23:46.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:45 smithi149 ceph-mon[129268]: pgmap v71: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 486 KiB/s wr, 70 op/s 2023-07-21T22:23:46.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:45 smithi104 ceph-mon[129021]: pgmap v71: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 486 KiB/s wr, 70 op/s 2023-07-21T22:23:46.514 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34864 2.81 MB/sec warmup 73 sec latency 2538.737 ms 2023-07-21T22:23:47.514 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34864 2.77 MB/sec warmup 74 sec latency 1665.518 ms 2023-07-21T22:23:48.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:47 smithi066 ceph-mon[126379]: pgmap v72: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-07-21T22:23:48.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:47 smithi149 ceph-mon[129268]: pgmap v72: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-07-21T22:23:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:47 smithi104 ceph-mon[129021]: pgmap v72: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-07-21T22:23:48.514 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34864 2.73 MB/sec warmup 75 sec latency 2665.653 ms 2023-07-21T22:23:49.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34864 2.70 MB/sec warmup 76 sec latency 3665.819 ms 2023-07-21T22:23:50.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:49 smithi066 ceph-mon[126379]: pgmap v73: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-07-21T22:23:50.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:49 smithi149 ceph-mon[129268]: pgmap v73: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-07-21T22:23:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:49 smithi104 ceph-mon[129021]: pgmap v73: 129 pgs: 129 active+clean; 38 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 137 KiB/s wr, 28 op/s 2023-07-21T22:23:50.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34937 2.67 MB/sec warmup 77 sec latency 4323.578 ms 2023-07-21T22:23:51.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34974 2.64 MB/sec warmup 78 sec latency 637.767 ms 2023-07-21T22:23:52.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:51 smithi066 ceph-mon[126379]: pgmap v74: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.0 MiB/s wr, 67 op/s 2023-07-21T22:23:52.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:51 smithi149 ceph-mon[129268]: pgmap v74: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.0 MiB/s wr, 67 op/s 2023-07-21T22:23:52.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:51 smithi104 ceph-mon[129021]: pgmap v74: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.0 MiB/s wr, 67 op/s 2023-07-21T22:23:52.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 34974 2.60 MB/sec warmup 79 sec latency 1637.881 ms 2023-07-21T22:23:53.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35138 2.58 MB/sec warmup 80 sec latency 1650.708 ms 2023-07-21T22:23:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:53 smithi149 ceph-mon[129268]: pgmap v75: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 988 KiB/s wr, 55 op/s 2023-07-21T22:23:54.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:53 smithi104 ceph-mon[129021]: pgmap v75: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 988 KiB/s wr, 55 op/s 2023-07-21T22:23:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:53 smithi066 ceph-mon[126379]: pgmap v75: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 988 KiB/s wr, 55 op/s 2023-07-21T22:23:54.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35138 2.55 MB/sec warmup 81 sec latency 1951.633 ms 2023-07-21T22:23:55.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35138 2.52 MB/sec warmup 82 sec latency 2951.772 ms 2023-07-21T22:23:56.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:55 smithi149 ceph-mon[129268]: pgmap v76: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1013 KiB/s wr, 62 op/s 2023-07-21T22:23:56.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:55 smithi104 ceph-mon[129021]: pgmap v76: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1013 KiB/s wr, 62 op/s 2023-07-21T22:23:56.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:55 smithi066 ceph-mon[126379]: pgmap v76: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1013 KiB/s wr, 62 op/s 2023-07-21T22:23:56.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35171 2.49 MB/sec warmup 83 sec latency 3272.739 ms 2023-07-21T22:23:57.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35171 2.46 MB/sec warmup 84 sec latency 1652.171 ms 2023-07-21T22:23:58.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:57 smithi149 ceph-mon[129268]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 935 KiB/s wr, 45 op/s 2023-07-21T22:23:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:57 smithi104 ceph-mon[129021]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 935 KiB/s wr, 45 op/s 2023-07-21T22:23:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:57 smithi066 ceph-mon[126379]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 935 KiB/s wr, 45 op/s 2023-07-21T22:23:58.515 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35171 2.43 MB/sec warmup 85 sec latency 2652.315 ms 2023-07-21T22:23:59.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35171 2.40 MB/sec warmup 86 sec latency 3652.386 ms 2023-07-21T22:24:00.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:23:59 smithi149 ceph-mon[129268]: pgmap v78: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 936 KiB/s wr, 45 op/s 2023-07-21T22:24:00.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:23:59 smithi104 ceph-mon[129021]: pgmap v78: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 936 KiB/s wr, 45 op/s 2023-07-21T22:24:00.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:23:59 smithi066 ceph-mon[126379]: pgmap v78: 129 pgs: 129 active+clean; 37 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 936 KiB/s wr, 45 op/s 2023-07-21T22:24:00.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 35171 2.37 MB/sec warmup 87 sec latency 4652.465 ms 2023-07-21T22:24:01.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 36174 2.41 MB/sec warmup 88 sec latency 4735.317 ms 2023-07-21T22:24:02.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:01 smithi149 ceph-mon[129268]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 941 KiB/s wr, 49 op/s 2023-07-21T22:24:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:01 smithi104 ceph-mon[129021]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 941 KiB/s wr, 49 op/s 2023-07-21T22:24:02.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:01 smithi066 ceph-mon[126379]: pgmap v79: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 941 KiB/s wr, 49 op/s 2023-07-21T22:24:02.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 36174 2.39 MB/sec warmup 89 sec latency 1071.586 ms 2023-07-21T22:24:03.513 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:24:03.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 36298 2.36 MB/sec warmup 90 sec latency 1080.448 ms 2023-07-21T22:24:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 -- 172.21.15.66:0/2220355424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 msgr2=0x7f205010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 --2- 172.21.15.66:0/2220355424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 0x7f205010c910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2048009f90 tx=0x7f204800b330 comp rx=0 tx=0).stop 2023-07-21T22:24:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 -- 172.21.15.66:0/2220355424 shutdown_connections 2023-07-21T22:24:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 --2- 172.21.15.66:0/2220355424 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 0x7f205010c910 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 --2- 172.21.15.66:0/2220355424 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050105ae0 0x7f2050105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 --2- 172.21.15.66:0/2220355424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f20500fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 -- 172.21.15.66:0/2220355424 >> 172.21.15.66:0/2220355424 conn(0x7f20500fb5f0 msgr2=0x7f20500fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:03.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 -- 172.21.15.66:0/2220355424 shutdown_connections 2023-07-21T22:24:03.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.689+0000 7f20567ff700 1 -- 172.21.15.66:0/2220355424 wait complete. 2023-07-21T22:24:03.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 Processor -- start 2023-07-21T22:24:03.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 -- start start 2023-07-21T22:24:03.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f2050197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 0x7f2050198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050105ae0 0x7f205019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205019cee0 con 0x7f2050100480 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205019d050 con 0x7f20500ffad0 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.690+0000 7f20567ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205019d210 con 0x7f2050105ae0 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f2050197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f2050197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53502/0 (socket says 172.21.15.66:53502) 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 -- 172.21.15.66:0/2683865704 learned_addr learned my addr 172.21.15.66:0/2683865704 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204f7fe700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 0x7f2050198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f2054d9c700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050105ae0 0x7f205019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 -- 172.21.15.66:0/2683865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050105ae0 msgr2=0x7f205019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050105ae0 0x7f205019c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 -- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 msgr2=0x7f2050198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 0x7f2050198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 -- 172.21.15.66:0/2683865704 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440095f0 con 0x7f20500ffad0 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204ffff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f2050197f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f204000e9a0 tx=0x7f204000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.691+0000 7f204d7fa700 1 -- 172.21.15.66:0/2683865704 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040009790 con 0x7f20500ffad0 2023-07-21T22:24:03.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.692+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2048009c40 con 0x7f20500ffad0 2023-07-21T22:24:03.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.692+0000 7f204d7fa700 1 -- 172.21.15.66:0/2683865704 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f204000cbd0 con 0x7f20500ffad0 2023-07-21T22:24:03.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.692+0000 7f204d7fa700 1 -- 172.21.15.66:0/2683865704 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040018680 con 0x7f20500ffad0 2023-07-21T22:24:03.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.692+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a4b50 con 0x7f20500ffad0 2023-07-21T22:24:03.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.695+0000 7f204d7fa700 1 -- 172.21.15.66:0/2683865704 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f204000cd40 con 0x7f20500ffad0 2023-07-21T22:24:03.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.696+0000 7f206a1c6b80 1 Processor -- start 2023-07-21T22:24:03.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.696+0000 7f206a1c6b80 1 -- start start 2023-07-21T22:24:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318b1b0 0x563b6318b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 0x563b63186350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b63186930 0x563b63186da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b6308bcb0 con 0x563b6318bac0 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b630dea40 con 0x563b6318b1b0 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b630d7ed0 con 0x563b63186930 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f204d7fa700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f20380a12c0 0x7f20380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f204ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 0x563b63186350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f2054d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318b1b0 0x563b6318b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f204ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 0x563b63186350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59716/0 (socket says 172.21.15.66:59716) 2023-07-21T22:24:03.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f2054d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318b1b0 0x563b6318b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53506/0 (socket says 172.21.15.66:53506) 2023-07-21T22:24:03.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f204ffff700 1 -- 172.21.15.66:0/590387866 learned_addr learned my addr 172.21.15.66:0/590387866 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:03.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f204f7fe700 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b63186930 0x563b63186da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f204d7fa700 1 -- 172.21.15.66:0/2683865704 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2040014070 con 0x7f20500ffad0 2023-07-21T22:24:03.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.697+0000 7f2054d9c700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f20380a12c0 0x7f20380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f204ffff700 1 -- 172.21.15.66:0/590387866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b63186930 msgr2=0x563b63186da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f204ffff700 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b63186930 0x563b63186da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f204ffff700 1 -- 172.21.15.66:0/590387866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318b1b0 msgr2=0x563b6318b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f204ffff700 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318b1b0 0x563b6318b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f204ffff700 1 -- 172.21.15.66:0/590387866 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b62f05de0 con 0x563b6318bac0 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f204ffff700 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 0x563b63186350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f204000c5b0 tx=0x7f204006c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f262b89e953da9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f2054d9c700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f20380a12c0 0x7f20380a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20480060b0 tx=0x7f2048006040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:03.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f20367fc700 1 -- 172.21.15.66:0/590387866 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20400cd120 con 0x563b6318bac0 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f20367fc700 1 -- 172.21.15.66:0/590387866 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f204009a040 con 0x563b6318bac0 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.698+0000 7f20367fc700 1 -- 172.21.15.66:0/590387866 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040065500 con 0x563b6318bac0 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 -- 172.21.15.66:0/590387866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 msgr2=0x563b63186350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 0x563b63186350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f204000c5b0 tx=0x7f204006c0b0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 -- 172.21.15.66:0/590387866 shutdown_connections 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318b1b0 0x563b6318b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b63186930 0x563b63186da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/590387866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318bac0 0x563b63186350 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 -- 172.21.15.66:0/590387866 >> 172.21.15.66:0/590387866 conn(0x563b6317bef0 msgr2=0x563b63181830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 -- 172.21.15.66:0/590387866 shutdown_connections 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.699+0000 7f206a1c6b80 1 -- 172.21.15.66:0/590387866 wait complete. 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 -- start start 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 Processor -- start 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 0x563b6318fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318b1b0 0x563b63190430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318bac0 0x563b63190970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b63077ac0 con 0x563b63186930 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b630777c0 con 0x563b6318bac0 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b62f5b920 con 0x563b6318b1b0 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.700+0000 7f206a1c6b80 10 client.0 _ll_register_callbacks cb 0x563b63075760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 0x563b6318fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 0x563b6318fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59730/0 (socket says 172.21.15.66:59730) 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 -- 172.21.15.66:0/4146665288 learned_addr learned my addr 172.21.15.66:0/4146665288 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:03.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f2054d9c700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318bac0 0x563b63190970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204f7fe700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318b1b0 0x563b63190430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 -- 172.21.15.66:0/4146665288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318b1b0 msgr2=0x563b63190430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318b1b0 0x563b63190430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 -- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318bac0 msgr2=0x563b63190970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318bac0 0x563b63190970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 -- 172.21.15.66:0/4146665288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20501a4b50 con 0x563b63186930 2023-07-21T22:24:03.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204f7fe700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318b1b0 0x563b63190430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f204ffff700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 0x563b6318fef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2040009230 tx=0x7f20400ca660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ba2ac237b4a945 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.701+0000 7f202f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f202f7fe700 1 -- 172.21.15.66:0/4146665288 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2040099530 con 0x563b63186930 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f202f7fe700 1 -- 172.21.15.66:0/4146665288 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2040099690 con 0x563b63186930 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b630314f0 con 0x563b63186930 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f202f7fe700 1 -- 172.21.15.66:0/4146665288 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20400a2000 con 0x563b63186930 2023-07-21T22:24:03.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 10 client.16017 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:03.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563b62eeec60 con 0x563b63186930 2023-07-21T22:24:03.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f202f7fe700 1 -- 172.21.15.66:0/4146665288 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20400b6080 con 0x563b63186930 2023-07-21T22:24:03.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f202f7fe700 1 -- 172.21.15.66:0/4146665288 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f204009ae20 con 0x563b63186930 2023-07-21T22:24:03.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:03.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 20 client.16017 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631a02a0 0x563b631a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 4 client.16017 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.702+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b630d0eb0 con 0x563b631a02a0 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.703+0000 7f204f7fe700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631a02a0 0x563b631a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.703+0000 7f204f7fe700 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631a02a0 0x563b631a26b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e229b07cb560739c server_cookie=e69feb847fc2a080 in_seq=0 out_seq=0 2023-07-21T22:24:03.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.703+0000 7f202f7fe700 10 client.16017 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f202f7fe700 1 -- 172.21.15.66:0/4146665288 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b630d0eb0 con 0x563b631a02a0 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f202f7fe700 10 client.16017 handle_command_reply: tid=0 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 client.16017 shutdown 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 msgr2=0x563b6318fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 0x563b6318fef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2040009230 tx=0x7f20400ca660 comp rx=0 tx=0).stop 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 shutdown_connections 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6318bac0 0x563b63190970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631a02a0 0x563b631a26b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b63186930 0x563b6318fef0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/4146665288 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318b1b0 0x563b63190430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f202f7fe700 0 client.16017 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 >> 172.21.15.66:0/4146665288 conn(0x563b6317bef0 msgr2=0x563b631813c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 shutdown_connections 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.704+0000 7f206a1c6b80 1 -- 172.21.15.66:0/4146665288 wait complete. 2023-07-21T22:24:03.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.705+0000 7f206a1c6b80 20 client.16017 trim_cache size 0 max 16384 2023-07-21T22:24:03.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.722+0000 7f206a1c6b80 1 Processor -- start 2023-07-21T22:24:03.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.722+0000 7f206a1c6b80 1 -- start start 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b6318f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b631860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b63181d70 con 0x563b6318b1b0 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b631876b0 con 0x563b63186930 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b6317d900 con 0x563b6318bac0 2023-07-21T22:24:03.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59736/0 (socket says 172.21.15.66:59736) 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 -- 172.21.15.66:0/1422797483 learned_addr learned my addr 172.21.15.66:0/1422797483 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204ffff700 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b6318f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f2054d9c700 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b631860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 -- 172.21.15.66:0/1422797483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 msgr2=0x563b631860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b631860f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 -- 172.21.15.66:0/1422797483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 msgr2=0x563b6318f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b6318f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.723+0000 7f204f7fe700 1 -- 172.21.15.66:0/1422797483 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b62f2c250 con 0x563b6318b1b0 2023-07-21T22:24:03.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.724+0000 7f204f7fe700 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631974b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x563b631915a0 tx=0x7f2044009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd71d4aa02be4d3f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:03.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.724+0000 7f2036ffd700 1 -- 172.21.15.66:0/1422797483 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044011cc0 con 0x563b6318b1b0 2023-07-21T22:24:03.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.724+0000 7f2036ffd700 1 -- 172.21.15.66:0/1422797483 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2044018040 con 0x563b6318b1b0 2023-07-21T22:24:03.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.724+0000 7f2036ffd700 1 -- 172.21.15.66:0/1422797483 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044005020 con 0x563b6318b1b0 2023-07-21T22:24:03.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.724+0000 7f206a1c6b80 1 -- 172.21.15.66:0/1422797483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 msgr2=0x563b631974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631974b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x563b631915a0 tx=0x7f2044009310 comp rx=0 tx=0).stop 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 -- 172.21.15.66:0/1422797483 shutdown_connections 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b6318f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b631860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/1422797483 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631974b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 -- 172.21.15.66:0/1422797483 >> 172.21.15.66:0/1422797483 conn(0x563b6317bef0 msgr2=0x563b631859e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 -- 172.21.15.66:0/1422797483 shutdown_connections 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 -- 172.21.15.66:0/1422797483 wait complete. 2023-07-21T22:24:03.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 -- start start 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.725+0000 7f206a1c6b80 1 Processor -- start 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b63197400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b63195c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b631961d0 con 0x563b6318b1b0 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b63196310 con 0x563b63186930 2023-07-21T22:24:03.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b631964d0 con 0x563b6318bac0 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f206a1c6b80 10 client.0 _ll_register_callbacks cb 0x7f2040005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59750/0 (socket says 172.21.15.66:59750) 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f204f7fe700 1 -- 172.21.15.66:0/57692969 learned_addr learned my addr 172.21.15.66:0/57692969 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f204ffff700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b63197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.726+0000 7f2054d9c700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b63195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204f7fe700 1 -- 172.21.15.66:0/57692969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 msgr2=0x563b63195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204f7fe700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b63195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204f7fe700 1 -- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 msgr2=0x563b63197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204f7fe700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b63197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204f7fe700 1 -- 172.21.15.66:0/57692969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204401fa40 con 0x563b6318b1b0 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204ffff700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b63197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f204f7fe700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631956c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2044009a60 tx=0x7f2044004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a71523c220f40d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f202f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f202f7fe700 1 -- 172.21.15.66:0/57692969 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044011d80 con 0x563b6318b1b0 2023-07-21T22:24:03.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f204009a040 con 0x563b6318b1b0 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.727+0000 7f202f7fe700 1 -- 172.21.15.66:0/57692969 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f204401f050 con 0x563b6318b1b0 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f202f7fe700 1 -- 172.21.15.66:0/57692969 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f204401f210 con 0x563b6318b1b0 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 10 client.16023 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563b62f042b0 con 0x563b6318b1b0 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f202f7fe700 1 -- 172.21.15.66:0/57692969 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f204009a040 con 0x563b6318b1b0 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f202f7fe700 1 -- 172.21.15.66:0/57692969 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f204402e070 con 0x563b6318b1b0 2023-07-21T22:24:03.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 20 client.16023 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631f5670 0x563b631f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 4 client.16023 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563b63093050 con 0x563b631f5670 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.728+0000 7f204ffff700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631f5670 0x563b631f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f204ffff700 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631f5670 0x563b631f7a80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fdfcd571cc73f45 server_cookie=26ddcb103b068dca in_seq=0 out_seq=0 2023-07-21T22:24:03.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f202f7fe700 10 client.16023 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f202f7fe700 1 -- 172.21.15.66:0/57692969 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563b63093050 con 0x563b631f5670 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f202f7fe700 10 client.16023 handle_command_reply: tid=0 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 client.16023 shutdown 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 msgr2=0x563b631956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631956c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2044009a60 tx=0x7f2044004ae0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 shutdown_connections 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b63186930 0x563b63197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563b631f5670 0x563b631f7a80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563b6318bac0 0x563b63195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 --2- 172.21.15.66:0/57692969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563b6318b1b0 0x563b631956c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f202f7fe700 0 client.16023 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.729+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 >> 172.21.15.66:0/57692969 conn(0x563b6317bef0 msgr2=0x563b63189cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 shutdown_connections 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f206a1c6b80 1 -- 172.21.15.66:0/57692969 wait complete. 2023-07-21T22:24:03.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f206a1c6b80 20 client.16023 trim_cache size 0 max 16384 2023-07-21T22:24:03.784 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:24:03.784 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:24:03.784 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:24:03.784 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:24:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f20380a12c0 msgr2=0x7f20380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f20380a12c0 0x7f20380a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f20480060b0 tx=0x7f2048006040 comp rx=0 tx=0).stop 2023-07-21T22:24:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 msgr2=0x7f2050197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:03.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f2050197f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f204000e9a0 tx=0x7f204000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 shutdown_connections 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20500ffad0 0x7f2050197f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f20380a12c0 0x7f20380a3770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2050105ae0 0x7f205019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 --2- 172.21.15.66:0/2683865704 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2050100480 0x7f2050198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 >> 172.21.15.66:0/2683865704 conn(0x7f20500fb5f0 msgr2=0x7f205010ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.730+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 shutdown_connections 2023-07-21T22:24:03.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.731+0000 7f20567ff700 1 -- 172.21.15.66:0/2683865704 wait complete. 2023-07-21T22:24:03.794 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:24:03.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:24:04.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 -- 172.21.15.66:0/902962838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278105ae0 msgr2=0x7f9278105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 --2- 172.21.15.66:0/902962838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278105ae0 0x7f9278105eb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f92680098f0 tx=0x7f9268009c00 comp rx=0 tx=0).stop 2023-07-21T22:24:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 -- 172.21.15.66:0/902962838 shutdown_connections 2023-07-21T22:24:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 --2- 172.21.15.66:0/902962838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278105ae0 0x7f9278105eb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 --2- 172.21.15.66:0/902962838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278100480 0x7f927810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 --2- 172.21.15.66:0/902962838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f92780fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 -- 172.21.15.66:0/902962838 >> 172.21.15.66:0/902962838 conn(0x7f92780fb590 msgr2=0x7f92780fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.957+0000 7f927e7ff700 1 -- 172.21.15.66:0/902962838 shutdown_connections 2023-07-21T22:24:04.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.958+0000 7f927e7ff700 1 -- 172.21.15.66:0/902962838 wait complete. 2023-07-21T22:24:04.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.958+0000 7f927e7ff700 1 Processor -- start 2023-07-21T22:24:04.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 -- start start 2023-07-21T22:24:04.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f9278072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278100480 0x7f9278072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278105ae0 0x7f92781a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9278073580 con 0x7f92780ffad0 2023-07-21T22:24:04.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92781a4310 con 0x7f9278100480 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f927e7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92781a4450 con 0x7f9278105ae0 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f9277fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f9278072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f9277fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f9278072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59770/0 (socket says 172.21.15.66:59770) 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f9277fff700 1 -- 172.21.15.66:0/2822357022 learned_addr learned my addr 172.21.15.66:0/2822357022 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.959+0000 7f92777fe700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278100480 0x7f9278072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f927cd9c700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278105ae0 0x7f92781a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f9277fff700 1 -- 172.21.15.66:0/2822357022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278105ae0 msgr2=0x7f92781a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f9277fff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278105ae0 0x7f92781a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f9277fff700 1 -- 172.21.15.66:0/2822357022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278100480 msgr2=0x7f9278072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f9277fff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278100480 0x7f9278072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f9277fff700 1 -- 172.21.15.66:0/2822357022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0095f0 con 0x7f92780ffad0 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f9277fff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f9278072810 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9268009810 tx=0x7f926800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926801c070 con 0x7f92780ffad0 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9268009520 con 0x7f92780ffad0 2023-07-21T22:24:04.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f926800bb20 con 0x7f92780ffad0 2023-07-21T22:24:04.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9268004930 con 0x7f92780ffad0 2023-07-21T22:24:04.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.960+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92781a4940 con 0x7f92780ffad0 2023-07-21T22:24:04.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.962+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9278067300 con 0x7f92780ffad0 2023-07-21T22:24:04.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.963+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9268020070 con 0x7f92780ffad0 2023-07-21T22:24:04.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.965+0000 7f92757fa700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f92600a1300 0x7f92600a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.965+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f92680cdd80 con 0x7f92780ffad0 2023-07-21T22:24:04.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.965+0000 7f92777fe700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f92600a1300 0x7f92600a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.966+0000 7f92777fe700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f92600a1300 0x7f92600a37b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9278074000 tx=0x7f926c009310 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:03.969+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9268096e70 con 0x7f92780ffad0 2023-07-21T22:24:04.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:03 smithi066 ceph-mon[126379]: pgmap v80: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-07-21T22:24:04.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:03 smithi149 ceph-mon[129268]: pgmap v80: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-07-21T22:24:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.228+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92781a4cb0 con 0x7f92780ffad0 2023-07-21T22:24:04.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.229+0000 7f92757fa700 1 -- 172.21.15.66:0/2822357022 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f9268005140 con 0x7f92780ffad0 2023-07-21T22:24:04.275 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:24:04.275 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:23:35.850636+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:24:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.231+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f92600a1300 msgr2=0x7f92600a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.231+0000 7f927e7ff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f92600a1300 0x7f92600a37b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9278074000 tx=0x7f926c009310 comp rx=0 tx=0).stop 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.231+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 msgr2=0x7f9278072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.231+0000 7f927e7ff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f9278072810 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9268009810 tx=0x7f926800fa00 comp rx=0 tx=0).stop 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 shutdown_connections 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f92600a1300 0x7f92600a37b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9278105ae0 0x7f92781a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f92780ffad0 0x7f9278072810 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 --2- 172.21.15.66:0/2822357022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9278100480 0x7f9278072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 >> 172.21.15.66:0/2822357022 conn(0x7f92780fb590 msgr2=0x7f92780fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 shutdown_connections 2023-07-21T22:24:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.232+0000 7f927e7ff700 1 -- 172.21.15.66:0/2822357022 wait complete. 2023-07-21T22:24:04.279 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 28 2023-07-21T22:24:04.296 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:24:04.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:03 smithi104 ceph-mon[129021]: pgmap v80: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 KiB/s wr, 11 op/s 2023-07-21T22:24:04.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 -- 172.21.15.66:0/3226112877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c105ae0 msgr2=0x7f9d6c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 --2- 172.21.15.66:0/3226112877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c105ae0 0x7f9d6c105eb0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009990 tx=0x7f9d5c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 -- 172.21.15.66:0/3226112877 shutdown_connections 2023-07-21T22:24:04.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 --2- 172.21.15.66:0/3226112877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d6c100480 0x7f9d6c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 --2- 172.21.15.66:0/3226112877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c105ae0 0x7f9d6c105eb0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 --2- 172.21.15.66:0/3226112877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d6c0ffad0 0x7f9d6c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 -- 172.21.15.66:0/3226112877 >> 172.21.15.66:0/3226112877 conn(0x7f9d6c0fb5f0 msgr2=0x7f9d6c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 -- 172.21.15.66:0/3226112877 shutdown_connections 2023-07-21T22:24:04.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.461+0000 7f9d729ff700 1 -- 172.21.15.66:0/3226112877 wait complete. 2023-07-21T22:24:04.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.462+0000 7f9d729ff700 1 Processor -- start 2023-07-21T22:24:04.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.462+0000 7f9d729ff700 1 -- start start 2023-07-21T22:24:04.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d729ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d6c0ffad0 0x7f9d6c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d729ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 0x7f9d6c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d729ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d6c105ae0 0x7f9d6c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d729ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19cdf0 con 0x7f9d6c100480 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d729ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19cf60 con 0x7f9d6c105ae0 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d729ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19d120 con 0x7f9d6c0ffad0 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d6b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 0x7f9d6c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d6b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 0x7f9d6c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59790/0 (socket says 172.21.15.66:59790) 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d6c0ffad0 0x7f9d6c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/1136127879 learned_addr learned my addr 172.21.15.66:0/1136127879 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d6c105ae0 0x7f9d6c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/1136127879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d6c0ffad0 msgr2=0x7f9d6c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.463+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d6c0ffad0 0x7f9d6c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/1136127879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d6c105ae0 msgr2=0x7f9d6c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d6c105ae0 0x7f9d6c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/1136127879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d60009520 con 0x7f9d6c100480 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 0x7f9d6c198350 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000eb30 tx=0x7f9d6000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d697fa700 1 -- 172.21.15.66:0/1136127879 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60009880 con 0x7f9d6c100480 2023-07-21T22:24:04.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c0095f0 con 0x7f9d6c100480 2023-07-21T22:24:04.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d697fa700 1 -- 172.21.15.66:0/1136127879 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d600044e0 con 0x7f9d6c100480 2023-07-21T22:24:04.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d697fa700 1 -- 172.21.15.66:0/1136127879 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60003d80 con 0x7f9d6c100480 2023-07-21T22:24:04.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.464+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c1a4a60 con 0x7f9d6c100480 2023-07-21T22:24:04.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.467+0000 7f9d697fa700 1 -- 172.21.15.66:0/1136127879 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9d60003f20 con 0x7f9d6c100480 2023-07-21T22:24:04.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.469+0000 7f9d86399b80 1 Processor -- start 2023-07-21T22:24:04.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.469+0000 7f9d697fa700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9d540a12c0 0x7f9d540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.469+0000 7f9d86399b80 1 -- start start 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.469+0000 7f9d697fa700 1 -- 172.21.15.66:0/1136127879 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9d60014070 con 0x7f9d6c100480 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.469+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9d540a12c0 0x7f9d540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d65302580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d652fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d652fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d86399b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d65202cb0 con 0x559d652fd930 2023-07-21T22:24:04.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d86399b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d65255a40 con 0x559d653021b0 2023-07-21T22:24:04.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d86399b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6524eed0 con 0x559d65302ac0 2023-07-21T22:24:04.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d652fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d70f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d652fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d652fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49626/0 (socket says 172.21.15.66:49626) 2023-07-21T22:24:04.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/597102782 learned_addr learned my addr 172.21.15.66:0/597102782 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d65302580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 36298 2.34 MB/sec warmup 91 sec latency 1851.550 ms 2023-07-21T22:24:04.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9d540a12c0 0x7f9d540a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400f590 tx=0x7f9d64009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/597102782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 msgr2=0x559d65302580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d65302580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/597102782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 msgr2=0x559d652fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d652fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.470+0000 7f9d6b7fe700 1 -- 172.21.15.66:0/597102782 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6507cde0 con 0x559d65302ac0 2023-07-21T22:24:04.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.471+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d652fd350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d600044b0 tx=0x7f9d6009a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fd9a5d794c9e7f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.471+0000 7f9d527fc700 1 -- 172.21.15.66:0/597102782 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600b1070 con 0x559d65302ac0 2023-07-21T22:24:04.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.471+0000 7f9d527fc700 1 -- 172.21.15.66:0/597102782 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6009e170 con 0x559d65302ac0 2023-07-21T22:24:04.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.471+0000 7f9d527fc700 1 -- 172.21.15.66:0/597102782 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d600986d0 con 0x559d65302ac0 2023-07-21T22:24:04.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 -- 172.21.15.66:0/597102782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 msgr2=0x559d652fd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d652fd350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9d600044b0 tx=0x7f9d6009a230 comp rx=0 tx=0).stop 2023-07-21T22:24:04.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 -- 172.21.15.66:0/597102782 shutdown_connections 2023-07-21T22:24:04.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d65302580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d652fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 --2- 172.21.15.66:0/597102782 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d652fd350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 -- 172.21.15.66:0/597102782 >> 172.21.15.66:0/597102782 conn(0x559d652f2ef0 msgr2=0x559d652f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 -- 172.21.15.66:0/597102782 shutdown_connections 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 -- 172.21.15.66:0/597102782 wait complete. 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.472+0000 7f9d86399b80 1 -- start start 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.473+0000 7f9d86399b80 1 Processor -- start 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d65306f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d653074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d65307a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d651eeac0 con 0x559d652fd930 2023-07-21T22:24:04.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d651ee7c0 con 0x559d653021b0 2023-07-21T22:24:04.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d650d2920 con 0x559d65302ac0 2023-07-21T22:24:04.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d86399b80 10 client.0 _ll_register_callbacks cb 0x559d651ec760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:04.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d65306f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d65306f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59812/0 (socket says 172.21.15.66:59812) 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3527415530 learned_addr learned my addr 172.21.15.66:0/3527415530 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d653074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d65307a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 msgr2=0x559d65307a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d65307a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3527415530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 msgr2=0x559d653074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d653074d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.474+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3527415530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c1a4a60 con 0x559d652fd930 2023-07-21T22:24:04.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d65306f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c010 tx=0x7f9d5c005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd06b201b120f1c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3527415530 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c01d070 con 0x559d652fd930 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3527415530 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9d5c004500 con 0x559d652fd930 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d651a84f0 con 0x559d652fd930 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3527415530 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5c0046c0 con 0x559d652fd930 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d86399b80 10 client.16041 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:04.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.475+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x559d65065c60 con 0x559d652fd930 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3527415530 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559d651a84f0 con 0x559d652fd930 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3527415530 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9d5c00f930 con 0x559d652fd930 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d86399b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d86399b80 10 client.16041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d86399b80 20 client.16041 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65317640 0x559d65319a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d86399b80 4 client.16041 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:04.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d65247eb0 con 0x559d65317640 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.476+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65317640 0x559d65319a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.477+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65317640 0x559d65319a50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99957d6f6468b9d0 server_cookie=bb36adc818bea2af in_seq=0 out_seq=0 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.477+0000 7f9d4b7fe700 10 client.16041 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3527415530 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559d65247eb0 con 0x559d65317640 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d4b7fe700 10 client.16041 handle_command_reply: tid=0 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 client.16041 shutdown 2023-07-21T22:24:04.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 msgr2=0x559d65306f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d65306f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00c010 tx=0x7f9d5c005ad0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 shutdown_connections 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65317640 0x559d65319a50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d65302ac0 0x559d65307a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d652fd930 0x559d65306f90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d4b7fe700 0 client.16041 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3527415530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d653021b0 0x559d653074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 >> 172.21.15.66:0/3527415530 conn(0x559d652f2ef0 msgr2=0x559d652f8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 shutdown_connections 2023-07-21T22:24:04.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 1 -- 172.21.15.66:0/3527415530 wait complete. 2023-07-21T22:24:04.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.478+0000 7f9d86399b80 20 client.16041 trim_cache size 0 max 16384 2023-07-21T22:24:04.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.492+0000 7f9d86399b80 1 Processor -- start 2023-07-21T22:24:04.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.492+0000 7f9d86399b80 1 -- start start 2023-07-21T22:24:04.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652f2ef0 0x559d652fd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652fd930 0x559d6530e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d65304760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d86399b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d652f5010 con 0x559d653021b0 2023-07-21T22:24:04.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d86399b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d652f4900 con 0x559d652fd930 2023-07-21T22:24:04.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d86399b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d652f9320 con 0x559d652f2ef0 2023-07-21T22:24:04.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652f2ef0 0x559d652fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d65304760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652f2ef0 0x559d652fd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49632/0 (socket says 172.21.15.66:49632) 2023-07-21T22:24:04.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d65304760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59816/0 (socket says 172.21.15.66:59816) 2023-07-21T22:24:04.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d6bfff700 1 -- 172.21.15.66:0/1848435440 learned_addr learned my addr 172.21.15.66:0/1848435440 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652fd930 0x559d6530e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 -- 172.21.15.66:0/1848435440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652f2ef0 msgr2=0x559d652fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652f2ef0 0x559d652fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 -- 172.21.15.66:0/1848435440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652fd930 msgr2=0x559d6530e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652fd930 0x559d6530e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d70f9c700 1 -- 172.21.15.66:0/1848435440 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d6531e680 con 0x559d653021b0 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.493+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652fd930 0x559d6530e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.494+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d65304760 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d6401b000 tx=0x7f9d6401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e948501b6fe8fbe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.494+0000 7f9d52ffd700 1 -- 172.21.15.66:0/1848435440 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400e040 con 0x559d653021b0 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.494+0000 7f9d52ffd700 1 -- 172.21.15.66:0/1848435440 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6400f040 con 0x559d653021b0 2023-07-21T22:24:04.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.494+0000 7f9d52ffd700 1 -- 172.21.15.66:0/1848435440 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640176a0 con 0x559d653021b0 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.494+0000 7f9d86399b80 1 -- 172.21.15.66:0/1848435440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 msgr2=0x559d65304760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.494+0000 7f9d86399b80 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d65304760 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9d6401b000 tx=0x7f9d6401c000 comp rx=0 tx=0).stop 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 -- 172.21.15.66:0/1848435440 shutdown_connections 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d65304760 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652f2ef0 0x559d652fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 --2- 172.21.15.66:0/1848435440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652fd930 0x559d6530e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 -- 172.21.15.66:0/1848435440 >> 172.21.15.66:0/1848435440 conn(0x559d652fe060 msgr2=0x559d65304050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 -- 172.21.15.66:0/1848435440 shutdown_connections 2023-07-21T22:24:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 -- 172.21.15.66:0/1848435440 wait complete. 2023-07-21T22:24:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 -- start start 2023-07-21T22:24:04.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.495+0000 7f9d86399b80 1 Processor -- start 2023-07-21T22:24:04.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 0x559d6530abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652fd930 0x559d6530b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d6530b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6536b500 con 0x559d653021b0 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6536b640 con 0x559d652f2ef0 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d6536b7d0 con 0x559d652fd930 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d86399b80 10 client.0 _ll_register_callbacks cb 0x7f9d5c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:04.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 0x559d6530abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.496+0000 7f9d70f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d6530b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 0x559d6530abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53584/0 (socket says 172.21.15.66:53584) 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3477728446 learned_addr learned my addr 172.21.15.66:0/3477728446 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652fd930 0x559d6530b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652fd930 msgr2=0x559d6530b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652fd930 0x559d6530b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3477728446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 msgr2=0x559d6530b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d6530b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 -- 172.21.15.66:0/3477728446 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d6507cde0 con 0x559d652f2ef0 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d70f9c700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d6530b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652fd930 0x559d6530b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d6bfff700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 0x559d6530abb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00fd60 tx=0x7f9d5c005360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e1a35b57d15deb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3477728446 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c01d070 con 0x559d652f2ef0 2023-07-21T22:24:04.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.497+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9d6009e170 con 0x559d652f2ef0 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3477728446 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9d5c000e40 con 0x559d652f2ef0 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3477728446 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d5c00f460 con 0x559d652f2ef0 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 10 client.26096 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f9d64019480 con 0x559d652f2ef0 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3477728446 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d6009e170 con 0x559d652f2ef0 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3477728446 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9d5c004500 con 0x559d652f2ef0 2023-07-21T22:24:04.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 10 client.26096 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 10 client.26096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 20 client.26096 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65302ac0 0x559d6536eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 4 client.26096 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.498+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559d6520a050 con 0x559d65302ac0 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.499+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65302ac0 0x559d6536eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.499+0000 7f9d4b7fe700 10 client.26096 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:04.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.499+0000 7f9d6b7fe700 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65302ac0 0x559d6536eb30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5ddc4cc6dd34689 server_cookie=28c3dfad0414dc1f in_seq=0 out_seq=0 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d4b7fe700 1 -- 172.21.15.66:0/3477728446 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559d6520a050 con 0x559d65302ac0 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d4b7fe700 10 client.26096 handle_command_reply: tid=0 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 client.26096 shutdown 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 msgr2=0x559d6530abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 0x559d6530abb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c00fd60 tx=0x7f9d5c005360 comp rx=0 tx=0).stop 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 shutdown_connections 2023-07-21T22:24:04.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x559d65302ac0 0x559d6536eb30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d652f2ef0 0x559d6530abb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d4b7fe700 0 client.26096 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559d653021b0 0x559d6530b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 --2- 172.21.15.66:0/3477728446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559d652fd930 0x559d6530b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 >> 172.21.15.66:0/3477728446 conn(0x559d652fe060 msgr2=0x559d653048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 shutdown_connections 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 1 -- 172.21.15.66:0/3477728446 wait complete. 2023-07-21T22:24:04.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.500+0000 7f9d86399b80 20 client.26096 trim_cache size 0 max 16384 2023-07-21T22:24:04.554 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:24:04.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9d540a12c0 msgr2=0x7f9d540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9d540a12c0 0x7f9d540a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400f590 tx=0x7f9d64009920 comp rx=0 tx=0).stop 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 msgr2=0x7f9d6c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 0x7f9d6c198350 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000eb30 tx=0x7f9d6000ee40 comp rx=0 tx=0).stop 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 shutdown_connections 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d6c105ae0 0x7f9d6c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9d540a12c0 0x7f9d540a3770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9d6c0ffad0 0x7f9d6c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 --2- 172.21.15.66:0/1136127879 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9d6c100480 0x7f9d6c198350 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 >> 172.21.15.66:0/1136127879 conn(0x7f9d6c0fb5f0 msgr2=0x7f9d6c10ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 shutdown_connections 2023-07-21T22:24:04.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.501+0000 7f9d729ff700 1 -- 172.21.15.66:0/1136127879 wait complete. 2023-07-21T22:24:04.565 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:24:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/610944436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 msgr2=0x7fdf040fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/610944436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 0x7fdf040fff40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdef40098f0 tx=0x7fdef4009c00 comp rx=0 tx=0).stop 2023-07-21T22:24:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/610944436 shutdown_connections 2023-07-21T22:24:04.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/610944436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04100480 0x7fdf0410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/610944436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04105ae0 0x7fdf04105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/610944436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 0x7fdf040fff40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/610944436 >> 172.21.15.66:0/610944436 conn(0x7fdf040fb5f0 msgr2=0x7fdf040fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/610944436 shutdown_connections 2023-07-21T22:24:04.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.729+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/610944436 wait complete. 2023-07-21T22:24:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.730+0000 7fdf0b5ff700 1 Processor -- start 2023-07-21T22:24:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.730+0000 7fdf0b5ff700 1 -- start start 2023-07-21T22:24:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0b5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 0x7fdf0419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0b5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 0x7fdf0419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0b5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04105ae0 0x7fdf04195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0b5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04196100 con 0x7fdf04100480 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0b5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04196270 con 0x7fdf040ffad0 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0b5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04196430 con 0x7fdf04105ae0 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 0x7fdf0419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 0x7fdf0419bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59832/0 (socket says 172.21.15.66:59832) 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 -- 172.21.15.66:0/3546435107 learned_addr learned my addr 172.21.15.66:0/3546435107 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf0939b700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 0x7fdf0419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04105ae0 0x7fdf04195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 -- 172.21.15.66:0/3546435107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04105ae0 msgr2=0x7fdf04195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04105ae0 0x7fdf04195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 -- 172.21.15.66:0/3546435107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 msgr2=0x7fdf0419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 0x7fdf0419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf08b9a700 1 -- 172.21.15.66:0/3546435107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf000095f0 con 0x7fdf04100480 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.731+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04105ae0 0x7fdf04195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.732+0000 7fdf08b9a700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 0x7fdf0419bf60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009810 tx=0x7fdef400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.732+0000 7fdefa7fc700 1 -- 172.21.15.66:0/3546435107 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef401c070 con 0x7fdf04100480 2023-07-21T22:24:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.732+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef4009520 con 0x7fdf04100480 2023-07-21T22:24:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.732+0000 7fdefa7fc700 1 -- 172.21.15.66:0/3546435107 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef400bbd0 con 0x7fdf04100480 2023-07-21T22:24:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.732+0000 7fdefa7fc700 1 -- 172.21.15.66:0/3546435107 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40049c0 con 0x7fdf04100480 2023-07-21T22:24:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.732+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf041a2830 con 0x7fdf04100480 2023-07-21T22:24:04.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.735+0000 7fdefa7fc700 1 -- 172.21.15.66:0/3546435107 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdef4020070 con 0x7fdf04100480 2023-07-21T22:24:04.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.736+0000 7fdf1ee43b80 1 Processor -- start 2023-07-21T22:24:04.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 -- start start 2023-07-21T22:24:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de1044580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de1044ac0 0x555de103f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de103f930 0x555de103fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdefa7fc700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdef00a12c0 0x7fdef00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0f44cb0 con 0x555de1044ac0 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0f97a40 con 0x555de10441b0 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0f90ed0 con 0x555de103f930 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdefa7fc700 1 -- 172.21.15.66:0/3546435107 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdef40cdd10 con 0x7fdf04100480 2023-07-21T22:24:04.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.737+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de1044580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de1044580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53598/0 (socket says 172.21.15.66:53598) 2023-07-21T22:24:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf09b9c700 1 -- 172.21.15.66:0/1612945889 learned_addr learned my addr 172.21.15.66:0/1612945889 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf08b9a700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdef00a12c0 0x7fdef00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de1044ac0 0x555de103f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de103f930 0x555de103fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 -- 172.21.15.66:0/1612945889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de103f930 msgr2=0x555de103fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de103f930 0x555de103fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 -- 172.21.15.66:0/1612945889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 msgr2=0x555de1044580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de1044580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 -- 172.21.15.66:0/1612945889 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555de0dbede0 con 0x555de1044ac0 2023-07-21T22:24:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf08b9a700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdef00a12c0 0x7fdef00a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdef409aab0 tx=0x7fdef4009c00 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.738+0000 7fdf0939b700 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de1044ac0 0x555de103f350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009f20 tx=0x7fdf00011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedc177b2ba681b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.739+0000 7fdeef7fe700 1 -- 172.21.15.66:0/1612945889 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004310 con 0x555de1044ac0 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.739+0000 7fdeef7fe700 1 -- 172.21.15.66:0/1612945889 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf00004470 con 0x555de1044ac0 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.739+0000 7fdeef7fe700 1 -- 172.21.15.66:0/1612945889 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00019840 con 0x555de1044ac0 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.739+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/1612945889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de1044ac0 msgr2=0x555de103f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.739+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de1044ac0 0x555de103f350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009f20 tx=0x7fdf00011720 comp rx=0 tx=0).stop 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.739+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/1612945889 shutdown_connections 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de1044580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de103f930 0x555de103fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/1612945889 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de1044ac0 0x555de103f350 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/1612945889 >> 172.21.15.66:0/1612945889 conn(0x555de1034ef0 msgr2=0x555de103a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/1612945889 shutdown_connections 2023-07-21T22:24:04.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/1612945889 wait complete. 2023-07-21T22:24:04.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 -- start start 2023-07-21T22:24:04.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.740+0000 7fdf1ee43b80 1 Processor -- start 2023-07-21T22:24:04.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de104bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de10441b0 0x555de1048e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de1044ac0 0x555de10495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0f30ac0 con 0x555de103f930 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0f307c0 con 0x555de1044ac0 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de0e14920 con 0x555de10441b0 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf1ee43b80 10 client.0 _ll_register_callbacks cb 0x555de0f2e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de104bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de104bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59842/0 (socket says 172.21.15.66:59842) 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf09b9c700 1 -- 172.21.15.66:0/3403645799 learned_addr learned my addr 172.21.15.66:0/3403645799 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.741+0000 7fdf0939b700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de10441b0 0x555de1048e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de1044ac0 0x555de10495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 -- 172.21.15.66:0/3403645799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de10441b0 msgr2=0x555de1048e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de10441b0 0x555de1048e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 -- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de1044ac0 msgr2=0x555de10495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de1044ac0 0x555de10495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 -- 172.21.15.66:0/3403645799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf00009b80 con 0x555de103f930 2023-07-21T22:24:04.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de104bc00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdefc007e50 tx=0x7fdefc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f95101f28578c20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdeecff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:24:04.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdeecff9700 1 -- 172.21.15.66:0/3403645799 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00e070 con 0x555de103f930 2023-07-21T22:24:04.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555de0eea4f0 con 0x555de103f930 2023-07-21T22:24:04.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.742+0000 7fdeecff9700 1 -- 172.21.15.66:0/3403645799 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdefc004bb0 con 0x555de103f930 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdeecff9700 1 -- 172.21.15.66:0/3403645799 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc004d70 con 0x555de103f930 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 10 client.16059 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555de0da7c60 con 0x555de103f930 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdeecff9700 1 -- 172.21.15.66:0/3403645799 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555de0eea4f0 con 0x555de103f930 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdeecff9700 1 -- 172.21.15.66:0/3403645799 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdefc004620 con 0x555de103f930 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 10 client.16059 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:04.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 10 client.16059 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 20 client.16059 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de1059540 0x555de105b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 4 client.16059 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.743+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555de0f89eb0 con 0x555de1059540 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.744+0000 7fdf0939b700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de1059540 0x555de105b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.744+0000 7fdf0939b700 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de1059540 0x555de105b950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1745b91ce7e1b2e8 server_cookie=5c8fd57bdbd38f63 in_seq=0 out_seq=0 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.744+0000 7fdeecff9700 10 client.16059 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:04.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.745+0000 7fdeecff9700 1 -- 172.21.15.66:0/3403645799 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555de0f89eb0 con 0x555de1059540 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.745+0000 7fdeecff9700 10 client.16059 handle_command_reply: tid=0 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.745+0000 7fdf1ee43b80 1 client.16059 shutdown 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.745+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 msgr2=0x555de104bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.745+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de104bc00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdefc007e50 tx=0x7fdefc00b040 comp rx=0 tx=0).stop 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 shutdown_connections 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de1059540 0x555de105b950 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de1044ac0 0x555de10495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de104bc00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/3403645799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de10441b0 0x555de1048e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 >> 172.21.15.66:0/3403645799 conn(0x555de1034ef0 msgr2=0x555de103a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdeecff9700 0 client.16059 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:04.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 shutdown_connections 2023-07-21T22:24:04.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/3403645799 wait complete. 2023-07-21T22:24:04.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.746+0000 7fdf1ee43b80 20 client.16059 trim_cache size 0 max 16384 2023-07-21T22:24:04.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.760+0000 7fdf1ee43b80 1 Processor -- start 2023-07-21T22:24:04.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.760+0000 7fdf1ee43b80 1 -- start start 2023-07-21T22:24:04.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de1050580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de103daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de1056b40 con 0x555de103f930 2023-07-21T22:24:04.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de10406d0 con 0x555de10441b0 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de1049ba0 con 0x555de1044ac0 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de1050580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf0939b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de1050580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59852/0 (socket says 172.21.15.66:59852) 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf09b9c700 1 -- 172.21.15.66:0/2699572648 learned_addr learned my addr 172.21.15.66:0/2699572648 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.761+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de103daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdf0939b700 1 -- 172.21.15.66:0/2699572648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 msgr2=0x555de103daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdf0939b700 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de103daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdf0939b700 1 -- 172.21.15.66:0/2699572648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 msgr2=0x555de1050580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdf0939b700 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de1050580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdf0939b700 1 -- 172.21.15.66:0/2699572648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555de0de5250 con 0x555de10441b0 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdf0939b700 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104b730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf00014040 tx=0x7fdf00009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9adfb1ae3c981259 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdeeffff700 1 -- 172.21.15.66:0/2699572648 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00003ed0 con 0x555de10441b0 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdeeffff700 1 -- 172.21.15.66:0/2699572648 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf00005510 con 0x555de10441b0 2023-07-21T22:24:04.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.762+0000 7fdeeffff700 1 -- 172.21.15.66:0/2699572648 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00003c80 con 0x555de10441b0 2023-07-21T22:24:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/2699572648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 msgr2=0x555de104b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104b730 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf00014040 tx=0x7fdf00009a10 comp rx=0 tx=0).stop 2023-07-21T22:24:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/2699572648 shutdown_connections 2023-07-21T22:24:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de103daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de1050580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/2699572648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104b730 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/2699572648 >> 172.21.15.66:0/2699572648 conn(0x555de1034ef0 msgr2=0x555de1054d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/2699572648 shutdown_connections 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.763+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/2699572648 wait complete. 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.764+0000 7fdf1ee43b80 1 -- start start 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.764+0000 7fdf1ee43b80 1 Processor -- start 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de10502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de104eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de104efe0 con 0x555de103f930 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de104f150 con 0x555de10441b0 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555de104f310 con 0x555de1044ac0 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf1ee43b80 10 client.0 _ll_register_callbacks cb 0x555de0f2e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de10502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf09b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de10502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59866/0 (socket says 172.21.15.66:59866) 2023-07-21T22:24:04.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf09b9c700 1 -- 172.21.15.66:0/888360638 learned_addr learned my addr 172.21.15.66:0/888360638 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf0939b700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de104eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf0939b700 1 -- 172.21.15.66:0/888360638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 msgr2=0x555de104eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf0939b700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de104eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf0939b700 1 -- 172.21.15.66:0/888360638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 msgr2=0x555de10502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf0939b700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de10502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf0939b700 1 -- 172.21.15.66:0/888360638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0001e040 con 0x555de10441b0 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.765+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de10502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdf0939b700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104e560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x555de0f89ed0 tx=0x7fdf00009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08efc44346f808b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdeecff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdeecff9700 1 -- 172.21.15.66:0/888360638 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004110 con 0x555de10441b0 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf00004470 con 0x555de10441b0 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdeecff9700 1 -- 172.21.15.66:0/888360638 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdf00004270 con 0x555de10441b0 2023-07-21T22:24:04.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdeecff9700 1 -- 172.21.15.66:0/888360638 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00003680 con 0x555de10441b0 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdf1ee43b80 10 client.26114 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.766+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555de0dbd2b0 con 0x555de10441b0 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdeecff9700 1 -- 172.21.15.66:0/888360638 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf00004470 con 0x555de10441b0 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdeecff9700 1 -- 172.21.15.66:0/888360638 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdf00039a40 con 0x555de10441b0 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf1ee43b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf1ee43b80 10 client.26114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:24:04.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf1ee43b80 20 client.26114 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:04.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de10b27d0 0x555de10b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:04.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf1ee43b80 4 client.26114 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:24:04.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555de0f4c050 con 0x555de10b27d0 2023-07-21T22:24:04.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.767+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de10b27d0 0x555de10b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:04.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf09b9c700 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de10b27d0 0x555de10b4be0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6703a34b239e4697 server_cookie=bc5f5288baccc504 in_seq=0 out_seq=0 2023-07-21T22:24:04.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdeecff9700 10 client.26114 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:04.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdeecff9700 1 -- 172.21.15.66:0/888360638 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555de0f4c050 con 0x555de10b27d0 2023-07-21T22:24:04.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdeecff9700 10 client.26114 handle_command_reply: tid=0 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 client.26114 shutdown 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 msgr2=0x555de104e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104e560 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x555de0f89ed0 tx=0x7fdf00009e50 comp rx=0 tx=0).stop 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 shutdown_connections 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555de1044ac0 0x555de104eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555de103f930 0x555de10502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x555de10b27d0 0x555de10b4be0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 --2- 172.21.15.66:0/888360638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555de10441b0 0x555de104e560 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 >> 172.21.15.66:0/888360638 conn(0x555de1034ef0 msgr2=0x555de103daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdeecff9700 0 client.26114 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 shutdown_connections 2023-07-21T22:24:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.768+0000 7fdf1ee43b80 1 -- 172.21.15.66:0/888360638 wait complete. 2023-07-21T22:24:04.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf1ee43b80 20 client.26114 trim_cache size 0 max 16384 2023-07-21T22:24:04.823 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:24:04.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdef00a12c0 msgr2=0x7fdef00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdef00a12c0 0x7fdef00a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdef409aab0 tx=0x7fdef4009c00 comp rx=0 tx=0).stop 2023-07-21T22:24:04.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 msgr2=0x7fdf0419bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:04.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 0x7fdf0419bf60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009810 tx=0x7fdef400fa00 comp rx=0 tx=0).stop 2023-07-21T22:24:04.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 shutdown_connections 2023-07-21T22:24:04.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdf040ffad0 0x7fdf0419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdef00a12c0 0x7fdef00a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdf04105ae0 0x7fdf04195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 --2- 172.21.15.66:0/3546435107 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdf04100480 0x7fdf0419bf60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:04.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 >> 172.21.15.66:0/3546435107 conn(0x7fdf040fb5f0 msgr2=0x7fdf040fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:04.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.769+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 shutdown_connections 2023-07-21T22:24:04.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.770+0000 7fdf0b5ff700 1 -- 172.21.15.66:0/3546435107 wait complete. 2023-07-21T22:24:04.833 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:24:05.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3599581966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc0ffae0 msgr2=0x7ff2bc0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3599581966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc0ffae0 0x7ff2bc0fff50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009990 tx=0x7ff2b0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3599581966 shutdown_connections 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3599581966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2bc105af0 0x7ff2bc105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3599581966 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2bc100490 0x7ff2bc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3599581966 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc0ffae0 0x7ff2bc0fff50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3599581966 >> 172.21.15.66:0/3599581966 conn(0x7ff2bc0fb5e0 msgr2=0x7ff2bc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:05.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3599581966 shutdown_connections 2023-07-21T22:24:05.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.997+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3599581966 wait complete. 2023-07-21T22:24:05.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.998+0000 7ff2c09ff700 1 Processor -- start 2023-07-21T22:24:05.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 -- start start 2023-07-21T22:24:05.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2bc0ffae0 0x7ff2bc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 0x7ff2bc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2bc105af0 0x7ff2bc1a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc073580 con 0x7ff2bc100490 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1a42f0 con 0x7ff2bc0ffae0 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2c09ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2bc1a4430 con 0x7ff2bc105af0 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2b9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 0x7ff2bc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2b9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 0x7ff2bc072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59890/0 (socket says 172.21.15.66:59890) 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/3159545428 learned_addr learned my addr 172.21.15.66:0/3159545428 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:05.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2bc0ffae0 0x7ff2bc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:04.999+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2bc105af0 0x7ff2bc1a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/3159545428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2bc105af0 msgr2=0x7ff2bc1a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2bc105af0 0x7ff2bc1a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/3159545428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2bc0ffae0 msgr2=0x7ff2bc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2bc0ffae0 0x7ff2bc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/3159545428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00095f0 con 0x7ff2bc100490 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 0x7ff2bc072d50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0000c00 tx=0x7ff2b0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:05.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2ab7fe700 1 -- 172.21.15.66:0/3159545428 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b001d070 con 0x7ff2bc100490 2023-07-21T22:24:05.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2ab7fe700 1 -- 172.21.15.66:0/3159545428 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b0021410 con 0x7ff2bc100490 2023-07-21T22:24:05.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2ab7fe700 1 -- 172.21.15.66:0/3159545428 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0017650 con 0x7ff2bc100490 2023-07-21T22:24:05.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2bc1a4630 con 0x7ff2bc100490 2023-07-21T22:24:05.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.000+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1a4a60 con 0x7ff2bc100490 2023-07-21T22:24:05.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.004+0000 7ff2ab7fe700 1 -- 172.21.15.66:0/3159545428 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff2b0021580 con 0x7ff2bc100490 2023-07-21T22:24:05.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.005+0000 7ff2d4246b80 1 Processor -- start 2023-07-21T22:24:05.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.005+0000 7ff2ab7fe700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff2a40a1340 0x7ff2a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.005+0000 7ff2d4246b80 1 -- start start 2023-07-21T22:24:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.005+0000 7ff2ab7fe700 1 -- 172.21.15.66:0/3159545428 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff2b00ce2d0 con 0x7ff2bc100490 2023-07-21T22:24:05.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.005+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff2a40a1340 0x7ff2a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 0x5620429bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bdac0 0x5620429b8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620428bdcb0 con 0x5620429b8930 2023-07-21T22:24:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562042910a40 con 0x5620429bdac0 2023-07-21T22:24:05.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562042909ed0 con 0x5620429bd1b0 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 0x5620429bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 0x5620429bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49726/0 (socket says 172.21.15.66:49726) 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2bad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429b8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1776586722 learned_addr learned my addr 172.21.15.66:0/1776586722 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bdac0 0x5620429b8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff2a40a1340 0x7ff2a40a37f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc074920 tx=0x7ff2b4009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.006+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1776586722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bdac0 msgr2=0x5620429b8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bdac0 0x5620429b8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1776586722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 msgr2=0x5620429b8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429b8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1776586722 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562042737de0 con 0x5620429bd1b0 2023-07-21T22:24:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bdac0 0x5620429b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:05.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:05.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 0x5620429bd580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009d00 tx=0x7ff2ac011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96f15f8125a7464f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:05.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2a8ff9700 1 -- 172.21.15.66:0/1776586722 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac004230 con 0x5620429bd1b0 2023-07-21T22:24:05.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2a8ff9700 1 -- 172.21.15.66:0/1776586722 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac004390 con 0x5620429bd1b0 2023-07-21T22:24:05.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.007+0000 7ff2a8ff9700 1 -- 172.21.15.66:0/1776586722 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac019710 con 0x5620429bd1b0 2023-07-21T22:24:05.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1776586722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 msgr2=0x5620429bd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 0x5620429bd580 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009d00 tx=0x7ff2ac011720 comp rx=0 tx=0).stop 2023-07-21T22:24:05.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1776586722 shutdown_connections 2023-07-21T22:24:05.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429b8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bd1b0 0x5620429bd580 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1776586722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bdac0 0x5620429b8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1776586722 >> 172.21.15.66:0/1776586722 conn(0x5620429adef0 msgr2=0x5620429b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1776586722 shutdown_connections 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.008+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1776586722 wait complete. 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 -- start start 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 Processor -- start 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 0x5620429c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bd1b0 0x5620429c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620428a9ac0 con 0x5620429bd1b0 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620428a97c0 con 0x5620429b8930 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56204278d920 con 0x5620429bdac0 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.009+0000 7ff2d4246b80 10 client.0 _ll_register_callbacks cb 0x5620428a7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 0x5620429c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 0x5620429c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53656/0 (socket says 172.21.15.66:53656) 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1993240013 learned_addr learned my addr 172.21.15.66:0/1993240013 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:05.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bd1b0 0x5620429c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1993240013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 msgr2=0x5620429c23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1993240013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bd1b0 msgr2=0x5620429c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bd1b0 0x5620429c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 -- 172.21.15.66:0/1993240013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b4019460 con 0x5620429b8930 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bd1b0 0x5620429c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:05.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 0x5620429c4bb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac000c00 tx=0x7ff2ac0038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a214c4177a5b68bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.010+0000 7ff2a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/1993240013 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac005040 con 0x5620429b8930 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620428634f0 con 0x5620429b8930 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/1993240013 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff2ac0051a0 con 0x5620429b8930 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/1993240013 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2ac005360 con 0x5620429b8930 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2d4246b80 10 client.26126 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:05.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7ff2bc1a4a60 con 0x5620429b8930 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.011+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/1993240013 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5620428634f0 con 0x5620429b8930 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/1993240013 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff2ac028020 con 0x5620429b8930 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2d4246b80 10 client.26126 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2d4246b80 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2d4246b80 20 client.26126 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5620429d23b0 0x5620429d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2d4246b80 4 client.26126 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:05.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562042902eb0 con 0x5620429d23b0 2023-07-21T22:24:05.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.012+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5620429d23b0 0x5620429d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.013+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5620429d23b0 0x5620429d47c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886bfd9db2013f81 server_cookie=dcf5ab4315aa7f27 in_seq=0 out_seq=0 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.013+0000 7ff2a1ffb700 10 client.26126 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.013+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/1993240013 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562042902eb0 con 0x5620429d23b0 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.013+0000 7ff2a1ffb700 10 client.26126 handle_command_reply: tid=0 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.013+0000 7ff2d4246b80 1 client.26126 shutdown 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 msgr2=0x5620429c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 0x5620429c4bb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac000c00 tx=0x7ff2ac0038b0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 shutdown_connections 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429b8930 0x5620429c4bb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5620429d23b0 0x5620429d47c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/1993240013 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bd1b0 0x5620429c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2a1ffb700 0 client.26126 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 >> 172.21.15.66:0/1993240013 conn(0x5620429adef0 msgr2=0x5620429b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:05.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 shutdown_connections 2023-07-21T22:24:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 1 -- 172.21.15.66:0/1993240013 wait complete. 2023-07-21T22:24:05.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.014+0000 7ff2d4246b80 20 client.26126 trim_cache size 0 max 16384 2023-07-21T22:24:05.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.027+0000 7ff2d4246b80 1 Processor -- start 2023-07-21T22:24:05.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 -- start start 2023-07-21T22:24:05.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 0x5620429c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bdac0 0x5620429c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620429b43b0 con 0x5620429bdac0 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620429b7320 con 0x5620429bd1b0 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620429c0350 con 0x5620429b8930 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2bad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bdac0 0x5620429c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 0x5620429c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2b9d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 0x5620429c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49742/0 (socket says 172.21.15.66:49742) 2023-07-21T22:24:05.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2b9d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53660/0 (socket says 172.21.15.66:53660) 2023-07-21T22:24:05.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.028+0000 7ff2ba59c700 1 -- 172.21.15.66:0/4081974255 learned_addr learned my addr 172.21.15.66:0/4081974255 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:05.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/4081974255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 msgr2=0x5620429c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 0x5620429c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/4081974255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bdac0 msgr2=0x5620429c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bdac0 0x5620429c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2b9d9b700 1 -- 172.21.15.66:0/4081974255 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac009960 con 0x5620429bd1b0 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 0x5620429c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c1660 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00cb670 tx=0x7ff2b0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75fe2d7cacc1de69 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:05.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2a97fa700 1 -- 172.21.15.66:0/4081974255 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b009b080 con 0x5620429bd1b0 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2a97fa700 1 -- 172.21.15.66:0/4081974255 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2b009f0b0 con 0x5620429bd1b0 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.029+0000 7ff2a97fa700 1 -- 172.21.15.66:0/4081974255 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b009b2e0 con 0x5620429bd1b0 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 -- 172.21.15.66:0/4081974255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 msgr2=0x5620429c1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c1660 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00cb670 tx=0x7ff2b0009310 comp rx=0 tx=0).stop 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 -- 172.21.15.66:0/4081974255 shutdown_connections 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429bdac0 0x5620429c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429b8930 0x5620429c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/4081974255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c1660 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 -- 172.21.15.66:0/4081974255 >> 172.21.15.66:0/4081974255 conn(0x5620429adef0 msgr2=0x5620429b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 -- 172.21.15.66:0/4081974255 shutdown_connections 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 -- 172.21.15.66:0/4081974255 wait complete. 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.030+0000 7ff2d4246b80 1 -- start start 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.031+0000 7ff2d4246b80 1 Processor -- start 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.031+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.080 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2d4246b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620429c9bc0 con 0x5620429b8930 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620429c8060 con 0x5620429bd1b0 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2d4246b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620429c81d0 con 0x5620429bdac0 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2d4246b80 10 client.0 _ll_register_callbacks cb 0x7ff2b400f330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2bad9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.081 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2ba59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59914/0 (socket says 172.21.15.66:59914) 2023-07-21T22:24:05.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2ba59c700 1 -- 172.21.15.66:0/254872887 learned_addr learned my addr 172.21.15.66:0/254872887 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:05.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2b9d9b700 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2bad9d700 1 -- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 msgr2=0x5620429c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2bad9d700 1 -- 172.21.15.66:0/254872887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 msgr2=0x5620429c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:05.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.032+0000 7ff2bad9d700 1 -- 172.21.15.66:0/254872887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2bc1a4a60 con 0x5620429bdac0 2023-07-21T22:24:05.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2bad9d700 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c7b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009fd0 tx=0x7ff2b4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc86672765a66a33 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:05.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2a1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/254872887 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4028070 con 0x5620429bdac0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2ac004390 con 0x5620429bdac0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/254872887 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff2b4017e40 con 0x5620429bdac0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/254872887 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2b400e040 con 0x5620429bdac0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2d4246b80 10 client.26110 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.033+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7ff2bc1a4630 con 0x5620429bdac0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/254872887 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2ac004390 con 0x5620429bdac0 2023-07-21T22:24:05.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/254872887 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff2b4023910 con 0x5620429bdac0 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2d4246b80 10 client.26110 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2d4246b80 10 client.26110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2d4246b80 20 client.26110 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562042a276b0 0x562042a29ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2d4246b80 4 client.26110 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5620428c5050 con 0x562042a276b0 2023-07-21T22:24:05.086 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.034+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562042a276b0 0x562042a29ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2a1ffb700 10 client.26110 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2ba59c700 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562042a276b0 0x562042a29ac0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca4c2b5f745c66b server_cookie=d2caa41d2091b017 in_seq=0 out_seq=0 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2a1ffb700 1 -- 172.21.15.66:0/254872887 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5620428c5050 con 0x562042a276b0 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2a1ffb700 10 client.26110 handle_command_reply: tid=0 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 client.26110 shutdown 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 msgr2=0x5620429c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c7b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009fd0 tx=0x7ff2b4004b40 comp rx=0 tx=0).stop 2023-07-21T22:24:05.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 shutdown_connections 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562042a276b0 0x562042a29ac0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5620429bdac0 0x5620429c7b20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5620429b8930 0x5620429c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2a1ffb700 0 client.26110 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 --2- 172.21.15.66:0/254872887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5620429bd1b0 0x5620429c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 >> 172.21.15.66:0/254872887 conn(0x5620429adef0 msgr2=0x5620429b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:05.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 shutdown_connections 2023-07-21T22:24:05.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 1 -- 172.21.15.66:0/254872887 wait complete. 2023-07-21T22:24:05.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.035+0000 7ff2d4246b80 20 client.26110 trim_cache size 0 max 16384 2023-07-21T22:24:05.089 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:24:05.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff2a40a1340 msgr2=0x7ff2a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff2a40a1340 0x7ff2a40a37f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc074920 tx=0x7ff2b4009920 comp rx=0 tx=0).stop 2023-07-21T22:24:05.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 msgr2=0x7ff2bc072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:05.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 0x7ff2bc072d50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0000c00 tx=0x7ff2b0004790 comp rx=0 tx=0).stop 2023-07-21T22:24:05.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 shutdown_connections 2023-07-21T22:24:05.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff2bc0ffae0 0x7ff2bc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff2a40a1340 0x7ff2a40a37f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2bc105af0 0x7ff2bc1a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 --2- 172.21.15.66:0/3159545428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff2bc100490 0x7ff2bc072d50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:05.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 >> 172.21.15.66:0/3159545428 conn(0x7ff2bc0fb5e0 msgr2=0x7ff2bc0fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:05.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 shutdown_connections 2023-07-21T22:24:05.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:05.036+0000 7ff2c09ff700 1 -- 172.21.15.66:0/3159545428 wait complete. 2023-07-21T22:24:05.098 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:04 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2822357022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:24:05.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:04 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2822357022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:24:05.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:04 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2822357022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:24:05.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 36298 2.31 MB/sec warmup 92 sec latency 2851.683 ms 2023-07-21T22:24:06.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:24:06.101 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:24:06.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:05 smithi149 ceph-mon[129268]: pgmap v81: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 148 KiB/s wr, 41 op/s 2023-07-21T22:24:06.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 -- 172.21.15.66:0/3570349633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c105ae0 msgr2=0x7f636c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 --2- 172.21.15.66:0/3570349633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c105ae0 0x7f636c105eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f635c009990 tx=0x7f635c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 -- 172.21.15.66:0/3570349633 shutdown_connections 2023-07-21T22:24:06.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 --2- 172.21.15.66:0/3570349633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c105ae0 0x7f636c105eb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 --2- 172.21.15.66:0/3570349633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636c100480 0x7f636c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 --2- 172.21.15.66:0/3570349633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c0ffad0 0x7f636c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 -- 172.21.15.66:0/3570349633 >> 172.21.15.66:0/3570349633 conn(0x7f636c0fb5f0 msgr2=0x7f636c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 -- 172.21.15.66:0/3570349633 shutdown_connections 2023-07-21T22:24:06.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.271+0000 7f63719ff700 1 -- 172.21.15.66:0/3570349633 wait complete. 2023-07-21T22:24:06.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.272+0000 7f63719ff700 1 Processor -- start 2023-07-21T22:24:06.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.272+0000 7f63719ff700 1 -- start start 2023-07-21T22:24:06.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f63719ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c0ffad0 0x7f636c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f63719ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 0x7f636c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f63719ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636c105ae0 0x7f636c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f63719ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19cdb0 con 0x7f636c100480 2023-07-21T22:24:06.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f63719ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19cf20 con 0x7f636c0ffad0 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f63719ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c19d0e0 con 0x7f636c105ae0 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f6363fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 0x7f636c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f6363fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 0x7f636c198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59936/0 (socket says 172.21.15.66:59936) 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f6363fff700 1 -- 172.21.15.66:0/1081102914 learned_addr learned my addr 172.21.15.66:0/1081102914 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f636affd700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c0ffad0 0x7f636c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.273+0000 7f636b7fe700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636c105ae0 0x7f636c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f6363fff700 1 -- 172.21.15.66:0/1081102914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636c105ae0 msgr2=0x7f636c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f6363fff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636c105ae0 0x7f636c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f6363fff700 1 -- 172.21.15.66:0/1081102914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c0ffad0 msgr2=0x7f636c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f6363fff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c0ffad0 0x7f636c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f6363fff700 1 -- 172.21.15.66:0/1081102914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c0095f0 con 0x7f636c100480 2023-07-21T22:24:06.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f6363fff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 0x7f636c198310 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9a0 tx=0x7f635400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f63709fd700 1 -- 172.21.15.66:0/1081102914 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354009790 con 0x7f636c100480 2023-07-21T22:24:06.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c19d3c0 con 0x7f636c100480 2023-07-21T22:24:06.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f63709fd700 1 -- 172.21.15.66:0/1081102914 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6354010430 con 0x7f636c100480 2023-07-21T22:24:06.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f63709fd700 1 -- 172.21.15.66:0/1081102914 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6354018680 con 0x7f636c100480 2023-07-21T22:24:06.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.274+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c1a4c00 con 0x7f636c100480 2023-07-21T22:24:06.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.278+0000 7f63709fd700 1 -- 172.21.15.66:0/1081102914 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f63540105a0 con 0x7f636c100480 2023-07-21T22:24:06.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.279+0000 7f6385388b80 1 Processor -- start 2023-07-21T22:24:06.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.279+0000 7f6385388b80 1 -- start start 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6385388b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369768600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6385388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768b40 0x55a3697633d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6385388b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3697639b0 0x55a369763e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6385388b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a369668d30 con 0x55a3697639b0 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6385388b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3696bbac0 con 0x55a369768230 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6385388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3696b4f50 con 0x55a369768b40 2023-07-21T22:24:06.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f63709fd700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63580a1340 0x7f63580a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f63709fd700 1 -- 172.21.15.66:0/1081102914 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6354014070 con 0x7f636c100480 2023-07-21T22:24:06.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6363fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3697639b0 0x55a369763e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369768600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6363fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3697639b0 0x55a369763e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59942/0 (socket says 172.21.15.66:59942) 2023-07-21T22:24:06.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f6363fff700 1 -- 172.21.15.66:0/3628368105 learned_addr learned my addr 172.21.15.66:0/3628368105 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f636affd700 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768b40 0x55a3697633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.280+0000 7f636b7fe700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63580a1340 0x7f63580a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 -- 172.21.15.66:0/3628368105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768b40 msgr2=0x55a3697633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768b40 0x55a3697633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 -- 172.21.15.66:0/3628368105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3697639b0 msgr2=0x55a369763e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3697639b0 0x55a369763e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 -- 172.21.15.66:0/3628368105 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3694e2e60 con 0x55a369768230 2023-07-21T22:24:06.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369768600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6364008e60 tx=0x7f636401a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5cfa89bb1518a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f636b7fe700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63580a1340 0x7f63580a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f636401aa10 tx=0x7f6364018000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f6361ffb700 1 -- 172.21.15.66:0/3628368105 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400e040 con 0x55a369768230 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f6361ffb700 1 -- 172.21.15.66:0/3628368105 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f636400f040 con 0x55a369768230 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.281+0000 7f6361ffb700 1 -- 172.21.15.66:0/3628368105 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364007c20 con 0x55a369768230 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 -- 172.21.15.66:0/3628368105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 msgr2=0x55a369768600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369768600 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6364008e60 tx=0x7f636401a6c0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 -- 172.21.15.66:0/3628368105 shutdown_connections 2023-07-21T22:24:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369768600 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a3697639b0 0x55a369763e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 --2- 172.21.15.66:0/3628368105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768b40 0x55a3697633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 -- 172.21.15.66:0/3628368105 >> 172.21.15.66:0/3628368105 conn(0x55a369758f70 msgr2=0x55a36975e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 -- 172.21.15.66:0/3628368105 shutdown_connections 2023-07-21T22:24:06.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.282+0000 7f6385388b80 1 -- 172.21.15.66:0/3628368105 wait complete. 2023-07-21T22:24:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.283+0000 7f6385388b80 1 -- start start 2023-07-21T22:24:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.283+0000 7f6385388b80 1 Processor -- start 2023-07-21T22:24:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.283+0000 7f6385388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a36976cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.283+0000 7f6385388b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a36976d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.283+0000 7f6385388b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a36976da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f6385388b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a369654b40 con 0x55a369768b40 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f6385388b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a369654840 con 0x55a369768230 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f6385388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3695389a0 con 0x55a3697639b0 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f6385388b80 10 client.0 _ll_register_callbacks cb 0x55a3696527e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a36976da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a36976cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f6363fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a36976d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a36976cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49790/0 (socket says 172.21.15.66:49790) 2023-07-21T22:24:06.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636affd700 1 -- 172.21.15.66:0/1003125098 learned_addr learned my addr 172.21.15.66:0/1003125098 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636b7fe700 1 -- 172.21.15.66:0/1003125098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 msgr2=0x55a36976cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636b7fe700 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a36976cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636b7fe700 1 -- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 msgr2=0x55a36976d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636b7fe700 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a36976d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.284+0000 7f636b7fe700 1 -- 172.21.15.66:0/1003125098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6364008a40 con 0x55a369768b40 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f636b7fe700 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a36976da30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6364025040 tx=0x7f6364003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05e5b81a41c699f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f634affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f634affd700 1 -- 172.21.15.66:0/1003125098 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364030070 con 0x55a369768b40 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a36960e570 con 0x55a369768b40 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f634affd700 1 -- 172.21.15.66:0/1003125098 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6364007710 con 0x55a369768b40 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f634affd700 1 -- 172.21.15.66:0/1003125098 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636400e050 con 0x55a369768b40 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f6385388b80 10 client.16095 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55a3694cbce0 con 0x55a369768b40 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.285+0000 7f634affd700 1 -- 172.21.15.66:0/1003125098 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a36960e570 con 0x55a369768b40 2023-07-21T22:24:06.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f634affd700 1 -- 172.21.15.66:0/1003125098 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6364033020 con 0x55a369768b40 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f6385388b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f6385388b80 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f6385388b80 20 client.16095 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f6385388b80 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a36977d520 0x55a36977f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f6385388b80 4 client.16095 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3696adf30 con 0x55a36977d520 2023-07-21T22:24:06.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.286+0000 7f636affd700 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a36977d520 0x55a36977f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.287+0000 7f634affd700 10 client.16095 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.287+0000 7f636affd700 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a36977d520 0x55a36977f930 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6685fc16b2bc1369 server_cookie=7a06e48cccf43ee in_seq=0 out_seq=0 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.287+0000 7f634affd700 1 -- 172.21.15.66:0/1003125098 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a3696adf30 con 0x55a36977d520 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.287+0000 7f634affd700 10 client.16095 handle_command_reply: tid=0 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 client.16095 shutdown 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 msgr2=0x55a36976da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a36976da30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6364025040 tx=0x7f6364003c30 comp rx=0 tx=0).stop 2023-07-21T22:24:06.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 shutdown_connections 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a36977d520 0x55a36977f930 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a36976da30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a36976cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 --2- 172.21.15.66:0/1003125098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a36976d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f634affd700 0 client.16095 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 >> 172.21.15.66:0/1003125098 conn(0x55a369758f70 msgr2=0x55a36975e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 shutdown_connections 2023-07-21T22:24:06.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 1 -- 172.21.15.66:0/1003125098 wait complete. 2023-07-21T22:24:06.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.288+0000 7f6385388b80 20 client.16095 trim_cache size 0 max 16384 2023-07-21T22:24:06.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.306+0000 7f6385388b80 1 Processor -- start 2023-07-21T22:24:06.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.306+0000 7f6385388b80 1 -- start start 2023-07-21T22:24:06.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6385388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a369774530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6385388b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369762aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6385388b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a369762fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6385388b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a369755fb0 con 0x55a369768b40 2023-07-21T22:24:06.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6385388b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36975a9b0 con 0x55a369768230 2023-07-21T22:24:06.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6385388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3697702a0 con 0x55a3697639b0 2023-07-21T22:24:06.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a369762fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f636affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a369774530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a369762fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:59964/0 (socket says 172.21.15.66:59964) 2023-07-21T22:24:06.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f636b7fe700 1 -- 172.21.15.66:0/2835024296 learned_addr learned my addr 172.21.15.66:0/2835024296 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.307+0000 7f6363fff700 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369762aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636affd700 1 -- 172.21.15.66:0/2835024296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 msgr2=0x55a369762aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636affd700 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369762aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636affd700 1 -- 172.21.15.66:0/2835024296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 msgr2=0x55a369762fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636affd700 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a369762fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636affd700 1 -- 172.21.15.66:0/2835024296 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3695092d0 con 0x55a3697639b0 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636b7fe700 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a369762fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f636affd700 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a369774530 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55a369780210 tx=0x7f635c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc37ae05864d2bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f63627fc700 1 -- 172.21.15.66:0/2835024296 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c01d040 con 0x55a3697639b0 2023-07-21T22:24:06.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f63627fc700 1 -- 172.21.15.66:0/2835024296 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f635c01e040 con 0x55a3697639b0 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.308+0000 7f63627fc700 1 -- 172.21.15.66:0/2835024296 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c005040 con 0x55a3697639b0 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 -- 172.21.15.66:0/2835024296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 msgr2=0x55a369774530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a369774530 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55a369780210 tx=0x7f635c005fd0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 -- 172.21.15.66:0/2835024296 shutdown_connections 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a369762fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a3697639b0 0x55a369774530 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 --2- 172.21.15.66:0/2835024296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a369768230 0x55a369762aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 -- 172.21.15.66:0/2835024296 >> 172.21.15.66:0/2835024296 conn(0x55a369758f70 msgr2=0x55a369761b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 -- 172.21.15.66:0/2835024296 shutdown_connections 2023-07-21T22:24:06.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 -- 172.21.15.66:0/2835024296 wait complete. 2023-07-21T22:24:06.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.309+0000 7f6385388b80 1 -- start start 2023-07-21T22:24:06.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.310+0000 7f6385388b80 1 Processor -- start 2023-07-21T22:24:06.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.310+0000 7f6385388b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3697639b0 0x55a36976ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.310+0000 7f6385388b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 0x55a36976f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6385388b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a3697ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6385388b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3697ca950 con 0x55a369768b40 2023-07-21T22:24:06.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6385388b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3697caac0 con 0x55a3697639b0 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6385388b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3697cac80 con 0x55a369768230 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6385388b80 10 client.0 _ll_register_callbacks cb 0x7f6354010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 0x55a36976f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a3697ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f636affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3697639b0 0x55a36976ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 0x55a36976f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49806/0 (socket says 172.21.15.66:49806) 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 -- 172.21.15.66:0/115478851 learned_addr learned my addr 172.21.15.66:0/115478851 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 -- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3697639b0 msgr2=0x55a36976ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3697639b0 0x55a36976ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 -- 172.21.15.66:0/115478851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 msgr2=0x55a3697ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a3697ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f6363fff700 1 -- 172.21.15.66:0/115478851 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3694cbce0 con 0x55a369768230 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.311+0000 7f636b7fe700 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a3697ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f6363fff700 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 0x55a36976f130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6354005bf0 tx=0x7f63540984f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e045c7af095dfdc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f634affd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:06.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f634affd700 1 -- 172.21.15.66:0/115478851 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400da60 con 0x55a369768230 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f636400f040 con 0x55a369768230 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f634affd700 1 -- 172.21.15.66:0/115478851 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f635409df70 con 0x55a369768230 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f634affd700 1 -- 172.21.15.66:0/115478851 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63540a2390 con 0x55a369768230 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f6385388b80 10 client.26128 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.312+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f636c10d210 con 0x55a369768230 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f634affd700 1 -- 172.21.15.66:0/115478851 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f636400f040 con 0x55a369768230 2023-07-21T22:24:06.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f634affd700 1 -- 172.21.15.66:0/115478851 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6354099430 con 0x55a369768230 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f6385388b80 10 client.26128 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f6385388b80 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f6385388b80 20 client.26128 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f6385388b80 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a3697d36a0 0x55a3697d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f6385388b80 4 client.26128 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a3696700d0 con 0x55a3697d36a0 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.313+0000 7f636affd700 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a3697d36a0 0x55a3697d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.314+0000 7f634affd700 10 client.26128 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.314+0000 7f636affd700 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a3697d36a0 0x55a3697d5ab0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2309521ce5b6f8f9 server_cookie=3d08cc402510ce0d in_seq=0 out_seq=0 2023-07-21T22:24:06.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f634affd700 1 -- 172.21.15.66:0/115478851 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a3696700d0 con 0x55a3697d36a0 2023-07-21T22:24:06.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f634affd700 10 client.26128 handle_command_reply: tid=0 2023-07-21T22:24:06.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 client.26128 shutdown 2023-07-21T22:24:06.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 msgr2=0x55a36976f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 0x55a36976f130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6354005bf0 tx=0x7f63540984f0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 shutdown_connections 2023-07-21T22:24:06.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3697639b0 0x55a36976ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a3697d36a0 0x55a3697d5ab0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a369768b40 0x55a3697ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 --2- 172.21.15.66:0/115478851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a369768230 0x55a36976f130 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f634affd700 0 client.26128 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 >> 172.21.15.66:0/115478851 conn(0x55a369758f70 msgr2=0x55a369762aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 shutdown_connections 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 1 -- 172.21.15.66:0/115478851 wait complete. 2023-07-21T22:24:06.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.315+0000 7f6385388b80 20 client.26128 trim_cache size 0 max 16384 2023-07-21T22:24:06.368 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:24:06.368 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:24:06.368 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "f02df1f3-3f71-4097-ad40-a9c505a197ae", 2023-07-21T22:24:06.368 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:24:06.368 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:24:06.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63580a1340 msgr2=0x7f63580a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63580a1340 0x7f63580a37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f636401aa10 tx=0x7f6364018000 comp rx=0 tx=0).stop 2023-07-21T22:24:06.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 msgr2=0x7f636c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 0x7f636c198310 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f635400e9a0 tx=0x7f635400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 shutdown_connections 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f636c0ffad0 0x7f636c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f63580a1340 0x7f63580a37f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f636c105ae0 0x7f636c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 --2- 172.21.15.66:0/1081102914 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f636c100480 0x7f636c198310 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 >> 172.21.15.66:0/1081102914 conn(0x7f636c0fb5f0 msgr2=0x7f636c10ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 shutdown_connections 2023-07-21T22:24:06.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.316+0000 7f63719ff700 1 -- 172.21.15.66:0/1081102914 wait complete. 2023-07-21T22:24:06.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:05 smithi104 ceph-mon[129021]: pgmap v81: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 148 KiB/s wr, 41 op/s 2023-07-21T22:24:06.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:05 smithi066 ceph-mon[126379]: pgmap v81: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 148 KiB/s wr, 41 op/s 2023-07-21T22:24:06.380 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:24:06.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 36458 2.29 MB/sec warmup 93 sec latency 3721.627 ms 2023-07-21T22:24:06.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.558+0000 7f1dd4dff700 1 -- 172.21.15.66:0/2311332885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 msgr2=0x7f1dd010bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.558+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/2311332885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 0x7f1dd010bdd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009990 tx=0x7f1dc4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 -- 172.21.15.66:0/2311332885 shutdown_connections 2023-07-21T22:24:06.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/2311332885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd010c310 0x7f1dd010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/2311332885 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd00728b0 0x7f1dd0072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/2311332885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 0x7f1dd010bdd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 -- 172.21.15.66:0/2311332885 >> 172.21.15.66:0/2311332885 conn(0x7f1dd00fb5f0 msgr2=0x7f1dd00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 -- 172.21.15.66:0/2311332885 shutdown_connections 2023-07-21T22:24:06.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.559+0000 7f1dd4dff700 1 -- 172.21.15.66:0/2311332885 wait complete. 2023-07-21T22:24:06.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 Processor -- start 2023-07-21T22:24:06.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 -- start start 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 0x7f1dd0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 0x7f1dd0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd010c310 0x7f1dd019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019cdf0 con 0x7f1dd010c310 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019cf60 con 0x7f1dd00728b0 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.560+0000 7f1dd4dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019d120 con 0x7f1dd00731c0 2023-07-21T22:24:06.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 0x7f1dd0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 0x7f1dd0197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53728/0 (socket says 172.21.15.66:53728) 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 -- 172.21.15.66:0/3681533138 learned_addr learned my addr 172.21.15.66:0/3681533138 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dced9d700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd010c310 0x7f1dd019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 0x7f1dd0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 -- 172.21.15.66:0/3681533138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 msgr2=0x7f1dd0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 0x7f1dd0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 -- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd010c310 msgr2=0x7f1dd019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd010c310 0x7f1dd019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 -- 172.21.15.66:0/3681533138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc8009c40 con 0x7f1dd00728b0 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 0x7f1dd0197e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e9a0 tx=0x7f1dc000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dbf7fe700 1 -- 172.21.15.66:0/3681533138 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0009790 con 0x7f1dd00728b0 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.561+0000 7f1dbf7fe700 1 -- 172.21.15.66:0/3681533138 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc000ce80 con 0x7f1dd00728b0 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.562+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc40095f0 con 0x7f1dd00728b0 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.562+0000 7f1dbf7fe700 1 -- 172.21.15.66:0/3681533138 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0018680 con 0x7f1dd00728b0 2023-07-21T22:24:06.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.562+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd01a4a60 con 0x7f1dd00728b0 2023-07-21T22:24:06.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.565+0000 7f1dbf7fe700 1 -- 172.21.15.66:0/3681533138 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1dc0010430 con 0x7f1dd00728b0 2023-07-21T22:24:06.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 Processor -- start 2023-07-21T22:24:06.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 -- start start 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908c1b0 0x5637f908c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 0x5637f9087350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f9087da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8f8ccb0 con 0x5637f908c1b0 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8fdfa40 con 0x5637f9087930 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.566+0000 7f1de8762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8fd8ed0 con 0x5637f908cac0 2023-07-21T22:24:06.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 0x5637f9087350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dced9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908c1b0 0x5637f908c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dcdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f9087da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 0x5637f9087350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49836/0 (socket says 172.21.15.66:49836) 2023-07-21T22:24:06.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dce59c700 1 -- 172.21.15.66:0/1235238359 learned_addr learned my addr 172.21.15.66:0/1235238359 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dbf7fe700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1db80a12c0 0x7f1db80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/1235238359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 msgr2=0x5637f9087350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 0x5637f9087350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dbf7fe700 1 -- 172.21.15.66:0/3681533138 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1dc0014070 con 0x7f1dd00728b0 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/1235238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908c1b0 msgr2=0x5637f908c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908c1b0 0x5637f908c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/1235238359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f8e06de0 con 0x5637f9087930 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.567+0000 7f1dce59c700 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 0x5637f9087350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:06.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f9087da0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400b2e0 tx=0x7f1dc4005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adb926cdf66360ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1dced9d700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1db80a12c0 0x7f1db80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1dbcff9700 1 -- 172.21.15.66:0/1235238359 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401d070 con 0x5637f9087930 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1dbcff9700 1 -- 172.21.15.66:0/1235238359 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc401e040 con 0x5637f9087930 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1dbcff9700 1 -- 172.21.15.66:0/1235238359 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4004830 con 0x5637f9087930 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1dced9d700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1db80a12c0 0x7f1db80a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009fd0 tx=0x7f1dc8004ad0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.568+0000 7f1de8762b80 1 -- 172.21.15.66:0/1235238359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 msgr2=0x5637f9087da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f9087da0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400b2e0 tx=0x7f1dc4005ea0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 -- 172.21.15.66:0/1235238359 shutdown_connections 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f9087da0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908c1b0 0x5637f908c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 --2- 172.21.15.66:0/1235238359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908cac0 0x5637f9087350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 -- 172.21.15.66:0/1235238359 >> 172.21.15.66:0/1235238359 conn(0x5637f907cef0 msgr2=0x5637f9082830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 -- 172.21.15.66:0/1235238359 shutdown_connections 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 -- 172.21.15.66:0/1235238359 wait complete. 2023-07-21T22:24:06.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.569+0000 7f1de8762b80 1 -- start start 2023-07-21T22:24:06.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 Processor -- start 2023-07-21T22:24:06.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f9087930 0x5637f9093bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f9090e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908cac0 0x5637f9091470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8f78ac0 con 0x5637f908cac0 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8f787c0 con 0x5637f908c1b0 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8e5c920 con 0x5637f9087930 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1de8762b80 10 client.0 _ll_register_callbacks cb 0x5637f8f76760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1dcdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f9090e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f9087930 0x5637f9093bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1dcdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f9090e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:53744/0 (socket says 172.21.15.66:53744) 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1dced9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908cac0 0x5637f9091470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.570+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3712244347 learned_addr learned my addr 172.21.15.66:0/3712244347 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3712244347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f9087930 msgr2=0x5637f9093bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f9087930 0x5637f9093bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3712244347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908cac0 msgr2=0x5637f9091470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908cac0 0x5637f9091470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3712244347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd01a4a60 con 0x5637f908c1b0 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f9087930 0x5637f9093bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:06.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f9090e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009ce0 tx=0x7f1dc400f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaaaf625d1631397 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3712244347 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401d040 con 0x5637f908c1b0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637f8f324f0 con 0x5637f908c1b0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.571+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3712244347 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1dc400fae0 con 0x5637f908c1b0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3712244347 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc400fca0 con 0x5637f908c1b0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 10 client.26162 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f1dd010f390 con 0x5637f908c1b0 2023-07-21T22:24:06.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3712244347 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637f8f324f0 con 0x5637f908c1b0 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3712244347 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1dc4022400 con 0x5637f908c1b0 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 20 client.26162 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f90a1120 0x5637f90a3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 4 client.26162 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:06.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.572+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637f8fd1eb0 con 0x5637f90a1120 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.573+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f90a1120 0x5637f90a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.573+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f90a1120 0x5637f90a3530 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b049220061af764 server_cookie=66a1d3b2fa1d7c43 in_seq=0 out_seq=0 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.573+0000 7f1db5ffb700 10 client.26162 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.574+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3712244347 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5637f8fd1eb0 con 0x5637f90a1120 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.574+0000 7f1db5ffb700 10 client.26162 handle_command_reply: tid=0 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.574+0000 7f1de8762b80 1 client.26162 shutdown 2023-07-21T22:24:06.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.574+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 msgr2=0x5637f9090e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.574+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f9090e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009ce0 tx=0x7f1dc400f460 comp rx=0 tx=0).stop 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 shutdown_connections 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f90a1120 0x5637f90a3530 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f908cac0 0x5637f9091470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f9087930 0x5637f9093bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1db5ffb700 0 client.26162 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3712244347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f9090e40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 >> 172.21.15.66:0/3712244347 conn(0x5637f907cef0 msgr2=0x5637f9082420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 shutdown_connections 2023-07-21T22:24:06.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 1 -- 172.21.15.66:0/3712244347 wait complete. 2023-07-21T22:24:06.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.575+0000 7f1de8762b80 20 client.26162 trim_cache size 0 max 16384 2023-07-21T22:24:06.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.592+0000 7f1de8762b80 1 Processor -- start 2023-07-21T22:24:06.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.592+0000 7f1de8762b80 1 -- start start 2023-07-21T22:24:06.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 0x5637f9098520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f9087930 0x5637f909ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f90822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1de8762b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8fd1eb0 con 0x5637f9087930 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1de8762b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f8f324f0 con 0x5637f908c1b0 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1de8762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f9082db0 con 0x5637f907cef0 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 0x5637f9098520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 0x5637f9098520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:49854/0 (socket says 172.21.15.66:49854) 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 -- 172.21.15.66:0/3199090265 learned_addr learned my addr 172.21.15.66:0/3199090265 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f9087930 0x5637f909ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dced9d700 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f90822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 -- 172.21.15.66:0/3199090265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 msgr2=0x5637f90822d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f90822d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 -- 172.21.15.66:0/3199090265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f9087930 msgr2=0x5637f909ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f9087930 0x5637f909ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.593+0000 7f1dce59c700 1 -- 172.21.15.66:0/3199090265 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637f8defc60 con 0x5637f907cef0 2023-07-21T22:24:06.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.594+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 0x5637f9098520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000c5b0 tx=0x7f1dc00cf4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d49d88461261c26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.594+0000 7f1dbd7fa700 1 -- 172.21.15.66:0/3199090265 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0099af0 con 0x5637f907cef0 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.594+0000 7f1dbd7fa700 1 -- 172.21.15.66:0/3199090265 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc0099910 con 0x5637f907cef0 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.594+0000 7f1dbd7fa700 1 -- 172.21.15.66:0/3199090265 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00983e0 con 0x5637f907cef0 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.594+0000 7f1de8762b80 1 -- 172.21.15.66:0/3199090265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 msgr2=0x5637f9098520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.594+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 0x5637f9098520 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000c5b0 tx=0x7f1dc00cf4e0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 -- 172.21.15.66:0/3199090265 shutdown_connections 2023-07-21T22:24:06.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f908c1b0 0x5637f90822d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f907cef0 0x5637f9098520 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3199090265 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f9087930 0x5637f909ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 -- 172.21.15.66:0/3199090265 >> 172.21.15.66:0/3199090265 conn(0x5637f9088060 msgr2=0x5637f90937f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 -- 172.21.15.66:0/3199090265 shutdown_connections 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 -- 172.21.15.66:0/3199090265 wait complete. 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 -- start start 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.595+0000 7f1de8762b80 1 Processor -- start 2023-07-21T22:24:06.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 0x5637f90e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f90e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908c1b0 0x5637f90ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f90ee860 con 0x5637f907cef0 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f90ee9d0 con 0x5637f9087930 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637f90eeb90 con 0x5637f908c1b0 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1de8762b80 10 client.0 _ll_register_callbacks cb 0x7f1dc800e280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:06.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 0x5637f90e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dce59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 0x5637f90e9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:60024/0 (socket says 172.21.15.66:60024) 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dce59c700 1 -- 172.21.15.66:0/3985245902 learned_addr learned my addr 172.21.15.66:0/3985245902 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dcdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f90e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dced9d700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908c1b0 0x5637f90ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3985245902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908c1b0 msgr2=0x5637f90ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908c1b0 0x5637f90ee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3985245902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 msgr2=0x5637f90e9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:06.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 0x5637f90e9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dcdd9b700 1 -- 172.21.15.66:0/3985245902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc40095f0 con 0x5637f9087930 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.596+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 0x5637f90e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1dcdd9b700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f90e97b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4005510 tx=0x7f1dc400efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc43a84dcab202f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1db5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3985245902 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc401d050 con 0x5637f9087930 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dc401e040 con 0x5637f9087930 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3985245902 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1dc4000e40 con 0x5637f9087930 2023-07-21T22:24:06.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3985245902 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc4022e50 con 0x5637f9087930 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1de8762b80 10 client.26168 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5637f8e2d250 con 0x5637f9087930 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.597+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3985245902 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc401e040 con 0x5637f9087930 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3985245902 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1dc4022490 con 0x5637f9087930 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1de8762b80 10 client.26168 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1de8762b80 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:06.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1de8762b80 20 client.26168 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:06.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f908cac0 0x5637f90f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:06.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1de8762b80 4 client.26168 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:24:06.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637f8f94050 con 0x5637f908cac0 2023-07-21T22:24:06.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.598+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f908cac0 0x5637f90f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:06.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1db5ffb700 10 client.26168 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1dce59c700 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f908cac0 0x5637f90f80f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732c2114dfb10d74 server_cookie=edb58b8f1ce84a4d in_seq=0 out_seq=0 2023-07-21T22:24:06.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1db5ffb700 1 -- 172.21.15.66:0/3985245902 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5637f8f94050 con 0x5637f908cac0 2023-07-21T22:24:06.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1db5ffb700 10 client.26168 handle_command_reply: tid=0 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 client.26168 shutdown 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 msgr2=0x5637f90e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f90e97b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4005510 tx=0x7f1dc400efb0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 shutdown_connections 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5637f908cac0 0x5637f90f80f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5637f908c1b0 0x5637f90ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5637f907cef0 0x5637f90e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1db5ffb700 0 client.26168 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 --2- 172.21.15.66:0/3985245902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5637f9087930 0x5637f90e97b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.599+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 >> 172.21.15.66:0/3985245902 conn(0x5637f9088060 msgr2=0x5637f90822b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 shutdown_connections 2023-07-21T22:24:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1de8762b80 1 -- 172.21.15.66:0/3985245902 wait complete. 2023-07-21T22:24:06.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1de8762b80 20 client.26168 trim_cache size 0 max 16384 2023-07-21T22:24:06.653 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: "f02df1f3-3f71-4097-ad40-a9c505a197ae": { 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "f02df1f3-3f71-4097-ad40-a9c505a197ae", 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:24:06.654 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:24:06.655 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:24:06.655 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:24:06.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1db80a12c0 msgr2=0x7f1db80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1db80a12c0 0x7f1db80a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009fd0 tx=0x7f1dc8004ad0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 msgr2=0x7f1dd0197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 0x7f1dd0197e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1dc000e9a0 tx=0x7f1dc000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 shutdown_connections 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dd00728b0 0x7f1dd0197e10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1db80a12c0 0x7f1db80a3770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1dd010c310 0x7f1dd019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 --2- 172.21.15.66:0/3681533138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1dd00731c0 0x7f1dd0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.600+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 >> 172.21.15.66:0/3681533138 conn(0x7f1dd00fb5f0 msgr2=0x7f1dd010e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.601+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 shutdown_connections 2023-07-21T22:24:06.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:06.601+0000 7f1dd4dff700 1 -- 172.21.15.66:0/3681533138 wait complete. 2023-07-21T22:24:06.664 INFO:tasks.cephfs.filesystem:scrub status for tag:f02df1f3-3f71-4097-ad40-a9c505a197ae - {'path': '/', 'tag': 'f02df1f3-3f71-4097-ad40-a9c505a197ae', 'options': 'recursive,force'} 2023-07-21T22:24:07.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:06 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:24:07.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:06 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:06 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:07.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:06 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:07.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:06 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:06 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:06 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:24:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:06 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:06 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:06 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:07.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:06 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:06 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:06 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:24:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:06 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:06 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:06 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:07.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:06 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:06 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:07.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39190 2.45 MB/sec warmup 94 sec latency 12.294 ms 2023-07-21T22:24:08.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:07 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:08.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:07 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:08.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:07 smithi149 ceph-mon[129268]: pgmap v82: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 123 KiB/s wr, 34 op/s 2023-07-21T22:24:08.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:07 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:08.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:07 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:07 smithi104 ceph-mon[129021]: pgmap v82: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 123 KiB/s wr, 34 op/s 2023-07-21T22:24:08.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:07 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:08.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:07 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:08.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:07 smithi066 ceph-mon[126379]: pgmap v82: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 123 KiB/s wr, 34 op/s 2023-07-21T22:24:08.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39722 2.44 MB/sec warmup 95 sec latency 392.158 ms 2023-07-21T22:24:09.516 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39722 2.41 MB/sec warmup 96 sec latency 1392.282 ms 2023-07-21T22:24:10.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:09 smithi149 ceph-mon[129268]: pgmap v83: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 123 KiB/s wr, 34 op/s 2023-07-21T22:24:10.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:09 smithi104 ceph-mon[129021]: pgmap v83: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 123 KiB/s wr, 34 op/s 2023-07-21T22:24:10.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:09 smithi066 ceph-mon[126379]: pgmap v83: 129 pgs: 129 active+clean; 39 MiB data, 480 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 123 KiB/s wr, 34 op/s 2023-07-21T22:24:10.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39722 2.39 MB/sec warmup 97 sec latency 2392.338 ms 2023-07-21T22:24:11.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39722 2.36 MB/sec warmup 98 sec latency 3392.440 ms 2023-07-21T22:24:12.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:11 smithi149 ceph-mon[129268]: pgmap v84: 129 pgs: 129 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 327 KiB/s wr, 69 op/s 2023-07-21T22:24:12.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:11 smithi104 ceph-mon[129021]: pgmap v84: 129 pgs: 129 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 327 KiB/s wr, 69 op/s 2023-07-21T22:24:12.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:11 smithi066 ceph-mon[126379]: pgmap v84: 129 pgs: 129 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 327 KiB/s wr, 69 op/s 2023-07-21T22:24:12.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39778 2.34 MB/sec warmup 99 sec latency 3621.649 ms 2023-07-21T22:24:13.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39846 2.32 MB/sec warmup 100 sec latency 937.159 ms 2023-07-21T22:24:14.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:13 smithi104 ceph-mon[129021]: pgmap v85: 129 pgs: 129 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 322 KiB/s wr, 65 op/s 2023-07-21T22:24:14.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:13 smithi066 ceph-mon[126379]: pgmap v85: 129 pgs: 129 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 322 KiB/s wr, 65 op/s 2023-07-21T22:24:14.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:13 smithi149 ceph-mon[129268]: pgmap v85: 129 pgs: 129 active+clean; 41 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 322 KiB/s wr, 65 op/s 2023-07-21T22:24:14.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39846 2.29 MB/sec warmup 101 sec latency 1937.237 ms 2023-07-21T22:24:15.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39846 2.27 MB/sec warmup 102 sec latency 2937.352 ms 2023-07-21T22:24:16.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:15 smithi104 ceph-mon[129021]: pgmap v86: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 863 KiB/s wr, 88 op/s 2023-07-21T22:24:16.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:15 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:24:16.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:15 smithi066 ceph-mon[126379]: pgmap v86: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 863 KiB/s wr, 88 op/s 2023-07-21T22:24:16.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:15 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:24:16.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:15 smithi149 ceph-mon[129268]: pgmap v86: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 863 KiB/s wr, 88 op/s 2023-07-21T22:24:16.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:15 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:24:16.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39846 2.25 MB/sec warmup 103 sec latency 3937.493 ms 2023-07-21T22:24:17.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:16 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:24:17.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:16 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:24:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:16 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:24:17.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:16 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:24:17.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:16 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:24:17.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:16 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:24:17.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 39846 2.23 MB/sec warmup 104 sec latency 4937.563 ms 2023-07-21T22:24:18.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:17 smithi104 ceph-mon[129021]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 746 KiB/s wr, 58 op/s 2023-07-21T22:24:18.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:17 smithi066 ceph-mon[126379]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 746 KiB/s wr, 58 op/s 2023-07-21T22:24:18.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:17 smithi149 ceph-mon[129268]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 746 KiB/s wr, 58 op/s 2023-07-21T22:24:18.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 42574 2.37 MB/sec warmup 105 sec latency 4944.493 ms 2023-07-21T22:24:19.517 INFO:tasks.workunit.client.0.smithi066.stdout: 1 43388 2.37 MB/sec warmup 106 sec latency 129.578 ms 2023-07-21T22:24:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:19 smithi104 ceph-mon[129021]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 746 KiB/s wr, 58 op/s 2023-07-21T22:24:20.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:19 smithi066 ceph-mon[126379]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 746 KiB/s wr, 58 op/s 2023-07-21T22:24:20.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:19 smithi149 ceph-mon[129268]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 746 KiB/s wr, 58 op/s 2023-07-21T22:24:20.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 43388 2.34 MB/sec warmup 107 sec latency 1129.728 ms 2023-07-21T22:24:21.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 43388 2.32 MB/sec warmup 108 sec latency 2129.817 ms 2023-07-21T22:24:22.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:21 smithi104 ceph-mon[129021]: pgmap v89: 129 pgs: 129 active+clean; 45 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 959 KiB/s wr, 94 op/s 2023-07-21T22:24:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:21 smithi066 ceph-mon[126379]: pgmap v89: 129 pgs: 129 active+clean; 45 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 959 KiB/s wr, 94 op/s 2023-07-21T22:24:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:21 smithi149 ceph-mon[129268]: pgmap v89: 129 pgs: 129 active+clean; 45 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 959 KiB/s wr, 94 op/s 2023-07-21T22:24:22.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 43388 2.30 MB/sec warmup 109 sec latency 3129.901 ms 2023-07-21T22:24:23.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 45961 2.41 MB/sec warmup 110 sec latency 3146.338 ms 2023-07-21T22:24:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:23 smithi104 ceph-mon[129021]: pgmap v90: 129 pgs: 129 active+clean; 45 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 755 KiB/s wr, 59 op/s 2023-07-21T22:24:24.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:23 smithi066 ceph-mon[126379]: pgmap v90: 129 pgs: 129 active+clean; 45 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 755 KiB/s wr, 59 op/s 2023-07-21T22:24:24.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:23 smithi149 ceph-mon[129268]: pgmap v90: 129 pgs: 129 active+clean; 45 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 755 KiB/s wr, 59 op/s 2023-07-21T22:24:24.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46814 2.43 MB/sec warmup 111 sec latency 243.169 ms 2023-07-21T22:24:25.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46814 2.41 MB/sec warmup 112 sec latency 1243.311 ms 2023-07-21T22:24:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:25 smithi104 ceph-mon[129021]: pgmap v91: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 967 KiB/s wr, 94 op/s 2023-07-21T22:24:26.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:25 smithi066 ceph-mon[126379]: pgmap v91: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 967 KiB/s wr, 94 op/s 2023-07-21T22:24:26.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:25 smithi149 ceph-mon[129268]: pgmap v91: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 967 KiB/s wr, 94 op/s 2023-07-21T22:24:26.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46814 2.39 MB/sec warmup 113 sec latency 2243.435 ms 2023-07-21T22:24:27.518 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46814 2.37 MB/sec warmup 114 sec latency 3243.633 ms 2023-07-21T22:24:28.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:27 smithi104 ceph-mon[129021]: pgmap v92: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 426 KiB/s wr, 70 op/s 2023-07-21T22:24:28.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:27 smithi066 ceph-mon[126379]: pgmap v92: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 426 KiB/s wr, 70 op/s 2023-07-21T22:24:28.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:27 smithi149 ceph-mon[129268]: pgmap v92: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 426 KiB/s wr, 70 op/s 2023-07-21T22:24:28.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46870 2.35 MB/sec warmup 115 sec latency 3256.171 ms 2023-07-21T22:24:29.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46870 2.33 MB/sec warmup 116 sec latency 1874.636 ms 2023-07-21T22:24:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:29 smithi104 ceph-mon[129021]: pgmap v93: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 426 KiB/s wr, 70 op/s 2023-07-21T22:24:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:29 smithi066 ceph-mon[126379]: pgmap v93: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 426 KiB/s wr, 70 op/s 2023-07-21T22:24:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:29 smithi149 ceph-mon[129268]: pgmap v93: 129 pgs: 129 active+clean; 47 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 426 KiB/s wr, 70 op/s 2023-07-21T22:24:30.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46870 2.31 MB/sec warmup 117 sec latency 2874.746 ms 2023-07-21T22:24:31.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46870 2.29 MB/sec warmup 118 sec latency 3874.891 ms 2023-07-21T22:24:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:31 smithi104 ceph-mon[129021]: pgmap v94: 129 pgs: 129 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 538 KiB/s wr, 79 op/s 2023-07-21T22:24:32.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:31 smithi066 ceph-mon[126379]: pgmap v94: 129 pgs: 129 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 538 KiB/s wr, 79 op/s 2023-07-21T22:24:32.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:31 smithi149 ceph-mon[129268]: pgmap v94: 129 pgs: 129 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 538 KiB/s wr, 79 op/s 2023-07-21T22:24:32.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46870 2.27 MB/sec warmup 119 sec latency 4874.945 ms 2023-07-21T22:24:34.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:33 smithi104 ceph-mon[129021]: pgmap v95: 129 pgs: 129 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 325 KiB/s wr, 42 op/s 2023-07-21T22:24:34.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:33 smithi066 ceph-mon[126379]: pgmap v95: 129 pgs: 129 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 325 KiB/s wr, 42 op/s 2023-07-21T22:24:34.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:33 smithi149 ceph-mon[129268]: pgmap v95: 129 pgs: 129 active+clean; 47 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 325 KiB/s wr, 42 op/s 2023-07-21T22:24:34.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46938 0.00 MB/sec execute 1 sec latency 4879.471 ms 2023-07-21T22:24:35.519 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46938 0.00 MB/sec execute 2 sec latency 2934.925 ms 2023-07-21T22:24:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:35 smithi104 ceph-mon[129021]: pgmap v96: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 354 KiB/s wr, 49 op/s 2023-07-21T22:24:36.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:35 smithi066 ceph-mon[126379]: pgmap v96: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 354 KiB/s wr, 49 op/s 2023-07-21T22:24:36.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:35 smithi149 ceph-mon[129268]: pgmap v96: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 354 KiB/s wr, 49 op/s 2023-07-21T22:24:36.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46938 0.00 MB/sec execute 3 sec latency 3935.075 ms 2023-07-21T22:24:36.665 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:24:36.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 -- 172.21.15.66:0/2059805874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 msgr2=0x7f18ac105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 --2- 172.21.15.66:0/2059805874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac105ee0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f189c009990 tx=0x7f189c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 -- 172.21.15.66:0/2059805874 shutdown_connections 2023-07-21T22:24:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 --2- 172.21.15.66:0/2059805874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18ac1004b0 0x7f18ac10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 --2- 172.21.15.66:0/2059805874 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac105ee0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 --2- 172.21.15.66:0/2059805874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac0ffb00 0x7f18ac0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 -- 172.21.15.66:0/2059805874 >> 172.21.15.66:0/2059805874 conn(0x7f18ac0fb600 msgr2=0x7f18ac0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.842+0000 7f18b27ff700 1 -- 172.21.15.66:0/2059805874 shutdown_connections 2023-07-21T22:24:36.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.843+0000 7f18b27ff700 1 -- 172.21.15.66:0/2059805874 wait complete. 2023-07-21T22:24:36.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.843+0000 7f18b27ff700 1 Processor -- start 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 -- start start 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18ac0ffb00 0x7f18ac197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 0x7f18ac198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac19cdd0 con 0x7f18ac105b10 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac19cf40 con 0x7f18ac1004b0 2023-07-21T22:24:36.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b27ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac19d100 con 0x7f18ac0ffb00 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b0d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 0x7f18ac198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b0d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35428/0 (socket says 172.21.15.66:35428) 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 0x7f18ac198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50622/0 (socket says 172.21.15.66:50622) 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18ac0ffb00 0x7f18ac197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.844+0000 7f18b0d9c700 1 -- 172.21.15.66:0/1656426024 learned_addr learned my addr 172.21.15.66:0/1656426024 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:36.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b0d9c700 1 -- 172.21.15.66:0/1656426024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18ac0ffb00 msgr2=0x7f18ac197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18ac0ffb00 0x7f18ac197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b0d9c700 1 -- 172.21.15.66:0/1656426024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 msgr2=0x7f18ac198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 0x7f18ac198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b0d9c700 1 -- 172.21.15.66:0/1656426024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894009520 con 0x7f18ac105b10 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18a3fff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 0x7f18ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac19c790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f18a4007ee0 tx=0x7f18a400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18a9ffb700 1 -- 172.21.15.66:0/1656426024 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400dd20 con 0x7f18ac105b10 2023-07-21T22:24:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c0095f0 con 0x7f18ac105b10 2023-07-21T22:24:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18a9ffb700 1 -- 172.21.15.66:0/1656426024 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18a400f040 con 0x7f18ac105b10 2023-07-21T22:24:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.845+0000 7f18a9ffb700 1 -- 172.21.15.66:0/1656426024 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a40048a0 con 0x7f18ac105b10 2023-07-21T22:24:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.846+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac19d5d0 con 0x7f18ac105b10 2023-07-21T22:24:36.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.849+0000 7f18a9ffb700 1 -- 172.21.15.66:0/1656426024 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f18a4003a00 con 0x7f18ac105b10 2023-07-21T22:24:36.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.850+0000 7f18c6052b80 1 Processor -- start 2023-07-21T22:24:36.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.850+0000 7f18c6052b80 1 -- start start 2023-07-21T22:24:36.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.850+0000 7f18a9ffb700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18980a12c0 0x7f18980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd974580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18a9ffb700 1 -- 172.21.15.66:0/1656426024 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f18a40cc840 con 0x7f18ac105b10 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd96f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd96fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18c6052b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd874cb0 con 0x5599dd9741b0 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18c6052b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd8c7a40 con 0x5599dd96f930 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18c6052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd8c0ed0 con 0x5599dd974ac0 2023-07-21T22:24:36.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18a3fff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18980a12c0 0x7f18980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd96f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd96f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47514/0 (socket says 172.21.15.66:47514) 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18a3fff700 1 -- 172.21.15.66:0/3376935321 learned_addr learned my addr 172.21.15.66:0/3376935321 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd96fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.851+0000 7f18abfff700 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd974580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18b0d9c700 1 -- 172.21.15.66:0/3376935321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 msgr2=0x5599dd96f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd96f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18b0d9c700 1 -- 172.21.15.66:0/3376935321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 msgr2=0x5599dd974580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd974580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18b0d9c700 1 -- 172.21.15.66:0/3376935321 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599dd6eede0 con 0x5599dd96f930 2023-07-21T22:24:36.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18abfff700 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd974580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:36.901 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18a3fff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18980a12c0 0x7f18980a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f189400f3c0 tx=0x7f18940135e0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd96fda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18a40c9be0 tx=0x7f18a4099380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae2d8311a35991b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18a27fc700 1 -- 172.21.15.66:0/3376935321 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4099560 con 0x5599dd96f930 2023-07-21T22:24:36.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18a27fc700 1 -- 172.21.15.66:0/3376935321 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18a409d590 con 0x5599dd96f930 2023-07-21T22:24:36.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.852+0000 7f18a27fc700 1 -- 172.21.15.66:0/3376935321 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4095c70 con 0x5599dd96f930 2023-07-21T22:24:36.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 -- 172.21.15.66:0/3376935321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 msgr2=0x5599dd96fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd96fda0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f18a40c9be0 tx=0x7f18a4099380 comp rx=0 tx=0).stop 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 -- 172.21.15.66:0/3376935321 shutdown_connections 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd96fda0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd974580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3376935321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd96f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 -- 172.21.15.66:0/3376935321 >> 172.21.15.66:0/3376935321 conn(0x5599dd964ef0 msgr2=0x5599dd96a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 -- 172.21.15.66:0/3376935321 shutdown_connections 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.853+0000 7f18c6052b80 1 -- 172.21.15.66:0/3376935321 wait complete. 2023-07-21T22:24:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 -- start start 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 Processor -- start 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd97bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd978e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd979590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd860ac0 con 0x5599dd9741b0 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd8607c0 con 0x5599dd96f930 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.854+0000 7f18c6052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd744920 con 0x5599dd974ac0 2023-07-21T22:24:36.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18c6052b80 10 client.0 _ll_register_callbacks cb 0x5599dd85e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd978e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd97bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd978e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35442/0 (socket says 172.21.15.66:35442) 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 -- 172.21.15.66:0/1551258490 learned_addr learned my addr 172.21.15.66:0/1551258490 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd979590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 -- 172.21.15.66:0/1551258490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 msgr2=0x5599dd979590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd979590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 -- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 msgr2=0x5599dd97bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd97bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 -- 172.21.15.66:0/1551258490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ac10d240 con 0x5599dd9741b0 2023-07-21T22:24:36.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f18a3fff700 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd978e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f189400ff80 tx=0x7f18940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387dae30a47b3e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.855+0000 7f188b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:24:36.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f188b7fe700 1 -- 172.21.15.66:0/1551258490 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940109d0 con 0x5599dd9741b0 2023-07-21T22:24:36.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599dd81a4f0 con 0x5599dd9741b0 2023-07-21T22:24:36.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f188b7fe700 1 -- 172.21.15.66:0/1551258490 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1894010b30 con 0x5599dd9741b0 2023-07-21T22:24:36.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f188b7fe700 1 -- 172.21.15.66:0/1551258490 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1894010cf0 con 0x5599dd9741b0 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f18c6052b80 10 client.16131 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f18ac19d5d0 con 0x5599dd9741b0 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f188b7fe700 1 -- 172.21.15.66:0/1551258490 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599dd81a4f0 con 0x5599dd9741b0 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.856+0000 7f188b7fe700 1 -- 172.21.15.66:0/1551258490 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1894029410 con 0x5599dd9741b0 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18c6052b80 10 client.16131 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18c6052b80 10 client.16131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:36.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18c6052b80 20 client.16131 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:36.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18c6052b80 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9894d0 0x5599dd98b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18c6052b80 4 client.16131 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:36.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599dd8b9eb0 con 0x5599dd9894d0 2023-07-21T22:24:36.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18abfff700 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9894d0 0x5599dd98b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f18abfff700 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9894d0 0x5599dd98b8e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=238eead26979fe79 server_cookie=25094f59f84ca74e in_seq=0 out_seq=0 2023-07-21T22:24:36.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.857+0000 7f188b7fe700 10 client.16131 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:36.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f188b7fe700 1 -- 172.21.15.66:0/1551258490 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5599dd8b9eb0 con 0x5599dd9894d0 2023-07-21T22:24:36.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f188b7fe700 10 client.16131 handle_command_reply: tid=0 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 client.16131 shutdown 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 msgr2=0x5599dd978e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd978e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f189400ff80 tx=0x7f18940047c0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 shutdown_connections 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd97bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9894d0 0x5599dd98b8e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd979590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 --2- 172.21.15.66:0/1551258490 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd978e50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f188b7fe700 0 client.16131 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:36.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.858+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 >> 172.21.15.66:0/1551258490 conn(0x5599dd964ef0 msgr2=0x5599dd96a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.859+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 shutdown_connections 2023-07-21T22:24:36.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.859+0000 7f18c6052b80 1 -- 172.21.15.66:0/1551258490 wait complete. 2023-07-21T22:24:36.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.859+0000 7f18c6052b80 20 client.16131 trim_cache size 0 max 16384 2023-07-21T22:24:36.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.876+0000 7f18c6052b80 1 Processor -- start 2023-07-21T22:24:36.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.876+0000 7f18c6052b80 1 -- start start 2023-07-21T22:24:36.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.876+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd96f930 0x5599dd980570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.876+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd96f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd974ac0 0x5599dd97b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18c6052b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd9706d0 con 0x5599dd9741b0 2023-07-21T22:24:36.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18c6052b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd967010 con 0x5599dd974ac0 2023-07-21T22:24:36.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18c6052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd962050 con 0x5599dd96f930 2023-07-21T22:24:36.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd96f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd96f930 0x5599dd980570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd96f930 0x5599dd980570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47528/0 (socket says 172.21.15.66:47528) 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18abfff700 1 -- 172.21.15.66:0/3983568162 learned_addr learned my addr 172.21.15.66:0/3983568162 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd974ac0 0x5599dd97b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 -- 172.21.15.66:0/3983568162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd96f930 msgr2=0x5599dd980570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd96f930 0x5599dd980570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 -- 172.21.15.66:0/3983568162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd974ac0 msgr2=0x5599dd97b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd974ac0 0x5599dd97b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 -- 172.21.15.66:0/3983568162 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599dd6d7c60 con 0x5599dd9741b0 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd974ac0 0x5599dd97b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.877+0000 7f18a3fff700 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd96f050 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1894029740 tx=0x7f18940134e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4644597949bfe3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18a2ffd700 1 -- 172.21.15.66:0/3983568162 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940092d0 con 0x5599dd9741b0 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18a2ffd700 1 -- 172.21.15.66:0/3983568162 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1894010e90 con 0x5599dd9741b0 2023-07-21T22:24:36.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18a2ffd700 1 -- 172.21.15.66:0/3983568162 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894029e10 con 0x5599dd9741b0 2023-07-21T22:24:36.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 -- 172.21.15.66:0/3983568162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 msgr2=0x5599dd96f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd96f050 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1894029740 tx=0x7f18940134e0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 -- 172.21.15.66:0/3983568162 shutdown_connections 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd974ac0 0x5599dd97b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd96f930 0x5599dd980570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 --2- 172.21.15.66:0/3983568162 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd96f050 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 -- 172.21.15.66:0/3983568162 >> 172.21.15.66:0/3983568162 conn(0x5599dd964ef0 msgr2=0x5599dd96e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 -- 172.21.15.66:0/3983568162 shutdown_connections 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.878+0000 7f18c6052b80 1 -- 172.21.15.66:0/3983568162 wait complete. 2023-07-21T22:24:36.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.879+0000 7f18c6052b80 1 -- start start 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.879+0000 7f18c6052b80 1 Processor -- start 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd977a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd977fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd9784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd980ba0 con 0x5599dd9741b0 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd9dd390 con 0x5599dd96f930 2023-07-21T22:24:36.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599dd9dd520 con 0x5599dd974ac0 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18c6052b80 10 client.0 _ll_register_callbacks cb 0x7f1894005ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd977a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd977a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50656/0 (socket says 172.21.15.66:50656) 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18abfff700 1 -- 172.21.15.66:0/2624714173 learned_addr learned my addr 172.21.15.66:0/2624714173 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd9784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18a3fff700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd977fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18b0d9c700 1 -- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 msgr2=0x5599dd977a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd977a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18b0d9c700 1 -- 172.21.15.66:0/2624714173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 msgr2=0x5599dd977fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd977fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18b0d9c700 1 -- 172.21.15.66:0/2624714173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189402e040 con 0x5599dd974ac0 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.880+0000 7f18abfff700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd977a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f18b0d9c700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd9784f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f18a40a1b70 tx=0x7f18a40b9180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e602e2f3f85b46 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:36.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f188b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:36.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f188b7fe700 1 -- 172.21.15.66:0/2624714173 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4099560 con 0x5599dd974ac0 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f18a409d590 con 0x5599dd974ac0 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f188b7fe700 1 -- 172.21.15.66:0/2624714173 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f18a4009410 con 0x5599dd974ac0 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f188b7fe700 1 -- 172.21.15.66:0/2624714173 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a40cc430 con 0x5599dd974ac0 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f18c6052b80 10 client.26164 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5599dd715250 con 0x5599dd974ac0 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.881+0000 7f188b7fe700 1 -- 172.21.15.66:0/2624714173 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18a40c7080 con 0x5599dd974ac0 2023-07-21T22:24:36.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f188b7fe700 1 -- 172.21.15.66:0/2624714173 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f18a409d170 con 0x5599dd974ac0 2023-07-21T22:24:36.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18c6052b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:36.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18c6052b80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:36.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18c6052b80 20 client.26164 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:36.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18c6052b80 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9de930 0x5599dd9e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:36.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18c6052b80 4 client.26164 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:24:36.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5599dd87c050 con 0x5599dd9de930 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18abfff700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9de930 0x5599dd9e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f188b7fe700 10 client.26164 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.882+0000 7f18abfff700 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9de930 0x5599dd9e0d40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf4c66e764d9e6c server_cookie=1c3e88f92a2ba33f in_seq=0 out_seq=0 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f188b7fe700 1 -- 172.21.15.66:0/2624714173 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5599dd87c050 con 0x5599dd9de930 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f188b7fe700 10 client.26164 handle_command_reply: tid=0 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 client.26164 shutdown 2023-07-21T22:24:36.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 msgr2=0x5599dd9784f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd9784f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f18a40a1b70 tx=0x7f18a40b9180 comp rx=0 tx=0).stop 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 shutdown_connections 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599dd96f930 0x5599dd977a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5599dd9de930 0x5599dd9e0d40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5599dd974ac0 0x5599dd9784f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 --2- 172.21.15.66:0/2624714173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5599dd9741b0 0x5599dd977fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 >> 172.21.15.66:0/2624714173 conn(0x5599dd964ef0 msgr2=0x5599dd96f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f188b7fe700 0 client.26164 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:36.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 shutdown_connections 2023-07-21T22:24:36.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 1 -- 172.21.15.66:0/2624714173 wait complete. 2023-07-21T22:24:36.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.883+0000 7f18c6052b80 20 client.26164 trim_cache size 0 max 16384 2023-07-21T22:24:36.939 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:24:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:24:36.940 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:24:36.940 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18980a12c0 msgr2=0x7f18980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18980a12c0 0x7f18980a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f189400f3c0 tx=0x7f18940135e0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 msgr2=0x7f18ac19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac19c790 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f18a4007ee0 tx=0x7f18a400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 shutdown_connections 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18980a12c0 0x7f18980a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f18ac105b10 0x7f18ac19c790 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f18ac0ffb00 0x7f18ac197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 --2- 172.21.15.66:0/1656426024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18ac1004b0 0x7f18ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:36.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 >> 172.21.15.66:0/1656426024 conn(0x7f18ac0fb600 msgr2=0x7f18ac10ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:36.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 shutdown_connections 2023-07-21T22:24:36.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:36.884+0000 7f18b27ff700 1 -- 172.21.15.66:0/1656426024 wait complete. 2023-07-21T22:24:36.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:24:36.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:24:37.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3932493639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 msgr2=0x7f4f140fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3932493639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f140fff40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4f000098f0 tx=0x7f4f00009c00 comp rx=0 tx=0).stop 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3932493639 shutdown_connections 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3932493639 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 0x7f4f1410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3932493639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 0x7f4f14105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3932493639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f140fff40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3932493639 >> 172.21.15.66:0/3932493639 conn(0x7f4f140fb590 msgr2=0x7f4f140fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3932493639 shutdown_connections 2023-07-21T22:24:37.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.111+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3932493639 wait complete. 2023-07-21T22:24:37.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.112+0000 7f4f1b1ff700 1 Processor -- start 2023-07-21T22:24:37.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.112+0000 7f4f1b1ff700 1 -- start start 2023-07-21T22:24:37.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1b1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f14072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1b1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 0x7f4f14072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1b1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 0x7f4f141a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1b1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f14073580 con 0x7f4f14100480 2023-07-21T22:24:37.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1b1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141a4310 con 0x7f4f140ffad0 2023-07-21T22:24:37.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1b1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f141a4450 con 0x7f4f14105ae0 2023-07-21T22:24:37.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f18f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f14072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f13fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 0x7f4f14072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f18f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f14072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50684/0 (socket says 172.21.15.66:50684) 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f18f9b700 1 -- 172.21.15.66:0/3993670438 learned_addr learned my addr 172.21.15.66:0/3993670438 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1979c700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 0x7f4f141a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1979c700 1 -- 172.21.15.66:0/3993670438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 msgr2=0x7f4f14072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1979c700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f14072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1979c700 1 -- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 msgr2=0x7f4f14072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.113+0000 7f4f1979c700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 0x7f4f14072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f1979c700 1 -- 172.21.15.66:0/3993670438 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009c40 con 0x7f4f14105ae0 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f13fff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 0x7f4f14072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f1979c700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 0x7f4f141a3dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00efb0 tx=0x7f4f0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c003a00 con 0x7f4f14105ae0 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c00f040 con 0x7f4f14105ae0 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f00009520 con 0x7f4f14105ae0 2023-07-21T22:24:37.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004540 con 0x7f4f14105ae0 2023-07-21T22:24:37.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.114+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f141a4a50 con 0x7f4f14105ae0 2023-07-21T22:24:37.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.115+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f14067300 con 0x7f4f14105ae0 2023-07-21T22:24:37.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.117+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4f0c003bf0 con 0x7f4f14105ae0 2023-07-21T22:24:37.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.119+0000 7f4f11ffb700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4f040a1380 0x7f4f040a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.119+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4f0c0ccc40 con 0x7f4f14105ae0 2023-07-21T22:24:37.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.119+0000 7f4f18f9b700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4f040a1380 0x7f4f040a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.119+0000 7f4f18f9b700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4f040a1380 0x7f4f040a3830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005950 tx=0x7f4f08009310 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.122+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4f0c095d30 con 0x7f4f14105ae0 2023-07-21T22:24:37.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.374+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f14073d40 con 0x7f4f14105ae0 2023-07-21T22:24:37.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.375+0000 7f4f11ffb700 1 -- 172.21.15.66:0/3993670438 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f4f0c099350 con 0x7f4f14105ae0 2023-07-21T22:24:37.420 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:24:37.420 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:23:35.850636+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:24:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4f040a1380 msgr2=0x7f4f040a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4f040a1380 0x7f4f040a3830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005950 tx=0x7f4f08009310 comp rx=0 tx=0).stop 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 msgr2=0x7f4f141a3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 0x7f4f141a3dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00efb0 tx=0x7f4f0c00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 shutdown_connections 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f140ffad0 0x7f4f14072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4f040a1380 0x7f4f040a3830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f14105ae0 0x7f4f141a3dd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 --2- 172.21.15.66:0/3993670438 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4f14100480 0x7f4f14072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 >> 172.21.15.66:0/3993670438 conn(0x7f4f140fb590 msgr2=0x7f4f140fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.377+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 shutdown_connections 2023-07-21T22:24:37.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.378+0000 7f4f1b1ff700 1 -- 172.21.15.66:0/3993670438 wait complete. 2023-07-21T22:24:37.424 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 28 2023-07-21T22:24:37.441 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:24:37.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 46938 0.00 MB/sec execute 4 sec latency 4935.140 ms 2023-07-21T22:24:37.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.615+0000 7f1eaffff700 1 -- 172.21.15.66:0/3081256728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb0105a10 msgr2=0x7f1eb0105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.615+0000 7f1eaffff700 1 --2- 172.21.15.66:0/3081256728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb0105a10 0x7f1eb0105de0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009990 tx=0x7f1ea0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.615+0000 7f1eaffff700 1 -- 172.21.15.66:0/3081256728 shutdown_connections 2023-07-21T22:24:37.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.615+0000 7f1eaffff700 1 --2- 172.21.15.66:0/3081256728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb0105a10 0x7f1eb0105de0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.616+0000 7f1eaffff700 1 --2- 172.21.15.66:0/3081256728 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb01003b0 0x7f1eb010c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.616+0000 7f1eaffff700 1 --2- 172.21.15.66:0/3081256728 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb00ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.616+0000 7f1eaffff700 1 -- 172.21.15.66:0/3081256728 >> 172.21.15.66:0/3081256728 conn(0x7f1eb00fb500 msgr2=0x7f1eb00fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.616+0000 7f1eaffff700 1 -- 172.21.15.66:0/3081256728 shutdown_connections 2023-07-21T22:24:37.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.616+0000 7f1eaffff700 1 -- 172.21.15.66:0/3081256728 wait complete. 2023-07-21T22:24:37.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.617+0000 7f1eaffff700 1 Processor -- start 2023-07-21T22:24:37.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.617+0000 7f1eaffff700 1 -- start start 2023-07-21T22:24:37.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.617+0000 7f1eaffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.617+0000 7f1eaffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb01003b0 0x7f1eb0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb0105a10 0x7f1eb01a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaffff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb0073580 con 0x7f1eb0105a10 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb01a4080 con 0x7f1eb01003b0 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaffff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb01a41c0 con 0x7f1eb00ffa00 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47584/0 (socket says 172.21.15.66:47584) 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 -- 172.21.15.66:0/413222950 learned_addr learned my addr 172.21.15.66:0/413222950 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb01003b0 0x7f1eb0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb0105a10 0x7f1eb01a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 -- 172.21.15.66:0/413222950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb01003b0 msgr2=0x7f1eb0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb01003b0 0x7f1eb0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 -- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb0105a10 msgr2=0x7f1eb01a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb0105a10 0x7f1eb01a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 -- 172.21.15.66:0/413222950 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00095f0 con 0x7f1eb00ffa00 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb01003b0 0x7f1eb0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb0105a10 0x7f1eb01a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.618+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb0072810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0000c00 tx=0x7f1ea0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.619+0000 7f1eb49fd700 1 -- 172.21.15.66:0/413222950 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea001d070 con 0x7f1eb00ffa00 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.619+0000 7f1eb49fd700 1 -- 172.21.15.66:0/413222950 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea0022470 con 0x7f1eb00ffa00 2023-07-21T22:24:37.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.619+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb01a43c0 con 0x7f1eb00ffa00 2023-07-21T22:24:37.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.619+0000 7f1eb49fd700 1 -- 172.21.15.66:0/413222950 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0017650 con 0x7f1eb00ffa00 2023-07-21T22:24:37.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.619+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb01a4840 con 0x7f1eb00ffa00 2023-07-21T22:24:37.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.623+0000 7f1eb49fd700 1 -- 172.21.15.66:0/413222950 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1ea00225e0 con 0x7f1eb00ffa00 2023-07-21T22:24:37.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 Processor -- start 2023-07-21T22:24:37.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 -- start start 2023-07-21T22:24:37.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd25fa860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2605700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2608020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd2500cb0 con 0x55cdd25fa490 2023-07-21T22:24:37.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd2553a40 con 0x55cdd2605c40 2023-07-21T22:24:37.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.624+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd254ced0 con 0x55cdd25fada0 2023-07-21T22:24:37.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2608020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2605700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2608020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50702/0 (socket says 172.21.15.66:50702) 2023-07-21T22:24:37.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 -- 172.21.15.66:0/1023203908 learned_addr learned my addr 172.21.15.66:0/1023203908 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd25fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 -- 172.21.15.66:0/1023203908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 msgr2=0x55cdd2605700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2605700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 -- 172.21.15.66:0/1023203908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 msgr2=0x55cdd25fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd25fa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 -- 172.21.15.66:0/1023203908 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdd237ade0 con 0x55cdd2605c40 2023-07-21T22:24:37.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eb49fd700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1e980a13e0 0x7f1e980a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eb49fd700 1 -- 172.21.15.66:0/413222950 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ea00ce4a0 con 0x7f1eb00ffa00 2023-07-21T22:24:37.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2608020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40115f0 tx=0x7f1ea4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3801a23c067e5617 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.625+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1e980a13e0 0x7f1e980a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.626+0000 7f1e95ffb700 1 -- 172.21.15.66:0/1023203908 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004230 con 0x55cdd2605c40 2023-07-21T22:24:37.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.626+0000 7f1e95ffb700 1 -- 172.21.15.66:0/1023203908 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea4004390 con 0x55cdd2605c40 2023-07-21T22:24:37.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.626+0000 7f1e95ffb700 1 -- 172.21.15.66:0/1023203908 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4019890 con 0x55cdd2605c40 2023-07-21T22:24:37.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.626+0000 7f1ec9346b80 1 -- 172.21.15.66:0/1023203908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 msgr2=0x55cdd2608020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.626+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1e980a13e0 0x7f1e980a3890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009fd0 tx=0x7f1ea8009040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.626+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2608020 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40115f0 tx=0x7f1ea4011900 comp rx=0 tx=0).stop 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 -- 172.21.15.66:0/1023203908 shutdown_connections 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2608020 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd25fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/1023203908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2605700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 -- 172.21.15.66:0/1023203908 >> 172.21.15.66:0/1023203908 conn(0x55cdd25f0ef0 msgr2=0x55cdd25f6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 -- 172.21.15.66:0/1023203908 shutdown_connections 2023-07-21T22:24:37.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 -- 172.21.15.66:0/1023203908 wait complete. 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.627+0000 7f1ec9346b80 1 -- start start 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.628+0000 7f1ec9346b80 1 Processor -- start 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.628+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd26055e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd26028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2602f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd24ecac0 con 0x55cdd25fa490 2023-07-21T22:24:37.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd24ec7c0 con 0x55cdd2605c40 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd23d0920 con 0x55cdd25fada0 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1ec9346b80 10 client.0 _ll_register_callbacks cb 0x55cdd24ea760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd26028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd26055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd26028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47608/0 (socket says 172.21.15.66:47608) 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eaf7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2602f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 -- 172.21.15.66:0/2237517350 learned_addr learned my addr 172.21.15.66:0/2237517350 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 -- 172.21.15.66:0/2237517350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 msgr2=0x55cdd2602f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2602f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 -- 172.21.15.66:0/2237517350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 msgr2=0x55cdd26055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd26055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eae7fc700 1 -- 172.21.15.66:0/2237517350 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea4009d50 con 0x55cdd25fada0 2023-07-21T22:24:37.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.629+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd26055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:37.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd26028d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40115c0 tx=0x7f1ea4005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24e96dd19eac9c38 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1e8effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:37.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1e8effd700 1 -- 172.21.15.66:0/2237517350 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4005720 con 0x55cdd25fada0 2023-07-21T22:24:37.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdd24a64f0 con 0x55cdd25fada0 2023-07-21T22:24:37.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1e8effd700 1 -- 172.21.15.66:0/2237517350 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ea4004950 con 0x55cdd25fada0 2023-07-21T22:24:37.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1e8effd700 1 -- 172.21.15.66:0/2237517350 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea4004570 con 0x55cdd25fada0 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1ec9346b80 10 client.26182 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cdd2363c60 con 0x55cdd25fada0 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.630+0000 7f1e8effd700 1 -- 172.21.15.66:0/2237517350 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cdd24a64f0 con 0x55cdd25fada0 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1e8effd700 1 -- 172.21.15.66:0/2237517350 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1ea4021040 con 0x55cdd25fada0 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1ec9346b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1ec9346b80 10 client.26182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:24:37.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1ec9346b80 20 client.26182 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd2615550 0x55cdd2617960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1ec9346b80 4 client.26182 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdd2545eb0 con 0x55cdd2615550 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd2615550 0x55cdd2617960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd2615550 0x55cdd2617960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f15dead75455f283 server_cookie=ac80e95154aae432 in_seq=0 out_seq=0 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.631+0000 7f1e8effd700 10 client.26182 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.632+0000 7f1e8effd700 1 -- 172.21.15.66:0/2237517350 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cdd2545eb0 con 0x55cdd2615550 2023-07-21T22:24:37.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.632+0000 7f1e8effd700 10 client.26182 handle_command_reply: tid=0 2023-07-21T22:24:37.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.632+0000 7f1ec9346b80 1 client.26182 shutdown 2023-07-21T22:24:37.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 msgr2=0x55cdd26028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd26028d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40115c0 tx=0x7f1ea4005270 comp rx=0 tx=0).stop 2023-07-21T22:24:37.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 shutdown_connections 2023-07-21T22:24:37.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd2615550 0x55cdd2617960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd2605c40 0x55cdd2602f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fa490 0x55cdd26055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/2237517350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd26028d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 >> 172.21.15.66:0/2237517350 conn(0x55cdd25f0ef0 msgr2=0x55cdd25f63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1e8effd700 0 client.26182 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:37.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 shutdown_connections 2023-07-21T22:24:37.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 1 -- 172.21.15.66:0/2237517350 wait complete. 2023-07-21T22:24:37.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.633+0000 7f1ec9346b80 20 client.26182 trim_cache size 0 max 16384 2023-07-21T22:24:37.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 Processor -- start 2023-07-21T22:24:37.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 -- start start 2023-07-21T22:24:37.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2610ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd2605c40 0x55cdd25fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd2608540 con 0x55cdd2605c40 2023-07-21T22:24:37.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd25f76b0 con 0x55cdd25fa490 2023-07-21T22:24:37.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.648+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd25fb1f0 con 0x55cdd25fada0 2023-07-21T22:24:37.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eae7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2610ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50722/0 (socket says 172.21.15.66:50722) 2023-07-21T22:24:37.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 -- 172.21.15.66:0/3929137228 learned_addr learned my addr 172.21.15.66:0/3929137228 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd2605c40 0x55cdd25fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 -- 172.21.15.66:0/3929137228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 msgr2=0x55cdd2610ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2610ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 -- 172.21.15.66:0/3929137228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd2605c40 msgr2=0x55cdd25fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd2605c40 0x55cdd25fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 -- 172.21.15.66:0/3929137228 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdd23a1250 con 0x55cdd25fa490 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.649+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260c740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ea009b300 tx=0x7f1ea001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6726f768c4a91a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1e967fc700 1 -- 172.21.15.66:0/3929137228 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea009b140 con 0x55cdd25fa490 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1e967fc700 1 -- 172.21.15.66:0/3929137228 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea009aeb0 con 0x55cdd25fa490 2023-07-21T22:24:37.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1e967fc700 1 -- 172.21.15.66:0/3929137228 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea00ce110 con 0x55cdd25fa490 2023-07-21T22:24:37.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 -- 172.21.15.66:0/3929137228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 msgr2=0x55cdd260c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260c740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1ea009b300 tx=0x7f1ea001a040 comp rx=0 tx=0).stop 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 -- 172.21.15.66:0/3929137228 shutdown_connections 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260c740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd2605c40 0x55cdd25fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/3929137228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd25fada0 0x55cdd2610ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 -- 172.21.15.66:0/3929137228 >> 172.21.15.66:0/3929137228 conn(0x55cdd25f0ef0 msgr2=0x55cdd26119d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 -- 172.21.15.66:0/3929137228 shutdown_connections 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.650+0000 7f1ec9346b80 1 -- 172.21.15.66:0/3929137228 wait complete. 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.651+0000 7f1ec9346b80 1 -- start start 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.651+0000 7f1ec9346b80 1 Processor -- start 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fada0 0x55cdd260ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd2605c40 0x55cdd26623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd2662900 con 0x55cdd25fada0 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd2662a40 con 0x55cdd25fa490 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd2662c00 con 0x55cdd2605c40 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1ec9346b80 10 client.0 _ll_register_callbacks cb 0x7f1ea8009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50724/0 (socket says 172.21.15.66:50724) 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 -- 172.21.15.66:0/143016309 learned_addr learned my addr 172.21.15.66:0/143016309 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fada0 0x55cdd260ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd2605c40 0x55cdd26623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 -- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd2605c40 msgr2=0x55cdd26623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd2605c40 0x55cdd26623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 -- 172.21.15.66:0/143016309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fada0 msgr2=0x55cdd260ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fada0 0x55cdd260ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 -- 172.21.15.66:0/143016309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb01a4840 con 0x55cdd25fa490 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fada0 0x55cdd260ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaf7fe700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd2605c40 0x55cdd26623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:37.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1eaeffd700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260a910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0097870 tx=0x7f1ea00a3730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1a57934e6dea044 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.652+0000 7f1e8effd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1e8effd700 1 -- 172.21.15.66:0/143016309 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea009b140 con 0x55cdd25fa490 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ea4004390 con 0x55cdd25fa490 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1e8effd700 1 -- 172.21.15.66:0/143016309 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ea009b370 con 0x55cdd25fa490 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1e8effd700 1 -- 172.21.15.66:0/143016309 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea009acd0 con 0x55cdd25fa490 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1ec9346b80 10 client.26210 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55cdd23792b0 con 0x55cdd25fa490 2023-07-21T22:24:37.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1e8effd700 1 -- 172.21.15.66:0/143016309 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ea4004390 con 0x55cdd25fa490 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1e8effd700 1 -- 172.21.15.66:0/143016309 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1ea009b5d0 con 0x55cdd25fa490 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1ec9346b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1ec9346b80 10 client.26210 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.653+0000 7f1ec9346b80 20 client.26210 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.654+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd266a060 0x55cdd266c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.654+0000 7f1ec9346b80 4 client.26210 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.654+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cdd2508050 con 0x55cdd266a060 2023-07-21T22:24:37.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.654+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd266a060 0x55cdd266c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.654+0000 7f1eae7fc700 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd266a060 0x55cdd266c470 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bee7177cefd49224 server_cookie=7481f3b6ad489601 in_seq=0 out_seq=0 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.654+0000 7f1e8effd700 10 client.26210 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1e8effd700 1 -- 172.21.15.66:0/143016309 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cdd2508050 con 0x55cdd266a060 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1e8effd700 10 client.26210 handle_command_reply: tid=0 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 client.26210 shutdown 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 msgr2=0x55cdd260a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260a910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0097870 tx=0x7f1ea00a3730 comp rx=0 tx=0).stop 2023-07-21T22:24:37.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 shutdown_connections 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55cdd266a060 0x55cdd266c470 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cdd25fa490 0x55cdd260a910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cdd2605c40 0x55cdd26623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 --2- 172.21.15.66:0/143016309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cdd25fada0 0x55cdd260ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1e8effd700 0 client.26210 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.655+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 >> 172.21.15.66:0/143016309 conn(0x55cdd25f0ef0 msgr2=0x55cdd2610ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 shutdown_connections 2023-07-21T22:24:37.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1ec9346b80 1 -- 172.21.15.66:0/143016309 wait complete. 2023-07-21T22:24:37.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1ec9346b80 20 client.26210 trim_cache size 0 max 16384 2023-07-21T22:24:37.708 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:24:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1e980a13e0 msgr2=0x7f1e980a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1eaffff700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1e980a13e0 0x7f1e980a3890 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009fd0 tx=0x7f1ea8009040 comp rx=0 tx=0).stop 2023-07-21T22:24:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 msgr2=0x7f1eb0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.656+0000 7f1eaffff700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb0072810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0000c00 tx=0x7f1ea0005740 comp rx=0 tx=0).stop 2023-07-21T22:24:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 shutdown_connections 2023-07-21T22:24:37.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1e980a13e0 0x7f1e980a3890 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1eb0105a10 0x7f1eb01a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1eb00ffa00 0x7f1eb0072810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 --2- 172.21.15.66:0/413222950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1eb01003b0 0x7f1eb0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 >> 172.21.15.66:0/413222950 conn(0x7f1eb00fb500 msgr2=0x7f1eb0106810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 shutdown_connections 2023-07-21T22:24:37.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.657+0000 7f1eaffff700 1 -- 172.21.15.66:0/413222950 wait complete. 2023-07-21T22:24:37.720 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:24:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.898+0000 7f916ffff700 1 -- 172.21.15.66:0/1993805104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 msgr2=0x7f9168105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.898+0000 7f916ffff700 1 --2- 172.21.15.66:0/1993805104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 0x7f9168105eb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9164009990 tx=0x7f9164009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 -- 172.21.15.66:0/1993805104 shutdown_connections 2023-07-21T22:24:37.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 --2- 172.21.15.66:0/1993805104 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9168100480 0x7f916810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 --2- 172.21.15.66:0/1993805104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 0x7f9168105eb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 --2- 172.21.15.66:0/1993805104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91680ffad0 0x7f91680fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 -- 172.21.15.66:0/1993805104 >> 172.21.15.66:0/1993805104 conn(0x7f91680fb5f0 msgr2=0x7f91680fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 -- 172.21.15.66:0/1993805104 shutdown_connections 2023-07-21T22:24:37.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.899+0000 7f916ffff700 1 -- 172.21.15.66:0/1993805104 wait complete. 2023-07-21T22:24:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.900+0000 7f916ffff700 1 Processor -- start 2023-07-21T22:24:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.900+0000 7f916ffff700 1 -- start start 2023-07-21T22:24:37.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91680ffad0 0x7f9168072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 0x7f9168072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 0x7f91681a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916ffff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168073580 con 0x7f91680ffad0 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916ffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91681a42e0 con 0x7f9168100480 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916ffff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91681a4420 con 0x7f9168105ae0 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 0x7f9168072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91680ffad0 0x7f9168072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 0x7f9168072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50734/0 (socket says 172.21.15.66:50734) 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 -- 172.21.15.66:0/3581877963 learned_addr learned my addr 172.21.15.66:0/3581877963 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916e59c700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 0x7f91681a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 -- 172.21.15.66:0/3581877963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 msgr2=0x7f91681a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 0x7f91681a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 -- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91680ffad0 msgr2=0x7f9168072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91680ffad0 0x7f9168072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916d59a700 1 -- 172.21.15.66:0/3581877963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640095f0 con 0x7f9168100480 2023-07-21T22:24:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.901+0000 7f916dd9b700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91680ffad0 0x7f9168072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.902+0000 7f916d59a700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 0x7f9168072d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f915800e9a0 tx=0x7f915800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.902+0000 7f915effd700 1 -- 172.21.15.66:0/3581877963 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158009790 con 0x7f9168100480 2023-07-21T22:24:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.902+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91681a4680 con 0x7f9168100480 2023-07-21T22:24:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.902+0000 7f915effd700 1 -- 172.21.15.66:0/3581877963 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9158010430 con 0x7f9168100480 2023-07-21T22:24:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.902+0000 7f915effd700 1 -- 172.21.15.66:0/3581877963 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158018680 con 0x7f9168100480 2023-07-21T22:24:37.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.902+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91681a4b60 con 0x7f9168100480 2023-07-21T22:24:37.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.907+0000 7f915effd700 1 -- 172.21.15.66:0/3581877963 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f91580105a0 con 0x7f9168100480 2023-07-21T22:24:37.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.908+0000 7f9183914b80 1 Processor -- start 2023-07-21T22:24:37.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.908+0000 7f915effd700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91540a12c0 0x7f91540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.908+0000 7f9183914b80 1 -- start start 2023-07-21T22:24:37.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.908+0000 7f915effd700 1 -- 172.21.15.66:0/3581877963 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9158014070 con 0x7f9168100480 2023-07-21T22:24:37.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.908+0000 7f916e59c700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91540a12c0 0x7f91540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f9183914b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f9183914b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f9183914b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6a7930 0x561b2c6a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f9183914b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c5accb0 con 0x561b2c6acac0 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f9183914b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c5ffa40 con 0x561b2c6ac1b0 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f9183914b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c5f8ed0 con 0x561b2c6a7930 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f916dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f916dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50742/0 (socket says 172.21.15.66:50742) 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f916d59a700 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f916dd9b700 1 -- 172.21.15.66:0/415944771 learned_addr learned my addr 172.21.15.66:0/415944771 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f916e59c700 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6a7930 0x561b2c6a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.909+0000 7f916e59c700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91540a12c0 0x7f91540a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9168074920 tx=0x7f916000b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f916dd9b700 1 -- 172.21.15.66:0/415944771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6a7930 msgr2=0x561b2c6a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f916dd9b700 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6a7930 0x561b2c6a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f916dd9b700 1 -- 172.21.15.66:0/415944771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 msgr2=0x561b2c6a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f916dd9b700 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f916dd9b700 1 -- 172.21.15.66:0/415944771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b2c426de0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f916dd9b700 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6ac580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9164005ea0 tx=0x7f9164005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5f67db7bbc2556 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f9153fff700 1 -- 172.21.15.66:0/415944771 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916401d070 con 0x561b2c6ac1b0 2023-07-21T22:24:37.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f9153fff700 1 -- 172.21.15.66:0/415944771 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916401e040 con 0x561b2c6ac1b0 2023-07-21T22:24:37.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.910+0000 7f9153fff700 1 -- 172.21.15.66:0/415944771 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640055f0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 -- 172.21.15.66:0/415944771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 msgr2=0x561b2c6ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6ac580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9164005ea0 tx=0x7f9164005ed0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 -- 172.21.15.66:0/415944771 shutdown_connections 2023-07-21T22:24:37.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6ac580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6a7930 0x561b2c6a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 --2- 172.21.15.66:0/415944771 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 -- 172.21.15.66:0/415944771 >> 172.21.15.66:0/415944771 conn(0x561b2c69cef0 msgr2=0x561b2c6a2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 -- 172.21.15.66:0/415944771 shutdown_connections 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.911+0000 7f9183914b80 1 -- 172.21.15.66:0/415944771 wait complete. 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.912+0000 7f9183914b80 1 -- start start 2023-07-21T22:24:37.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.912+0000 7f9183914b80 1 Processor -- start 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6acac0 0x561b2c6b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c598ac0 con 0x561b2c6a7930 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c5987c0 con 0x561b2c6acac0 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c47c920 con 0x561b2c6ac1b0 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f9183914b80 10 client.0 _ll_register_callbacks cb 0x561b2c596760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:37.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47672/0 (socket says 172.21.15.66:47672) 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 -- 172.21.15.66:0/2471197682 learned_addr learned my addr 172.21.15.66:0/2471197682 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916e59c700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6acac0 0x561b2c6b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916dd9b700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 -- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6acac0 msgr2=0x561b2c6b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6acac0 0x561b2c6b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 -- 172.21.15.66:0/2471197682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 msgr2=0x561b2c6b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.913+0000 7f916d59a700 1 -- 172.21.15.66:0/2471197682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916810d210 con 0x561b2c6ac1b0 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f916d59a700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b13d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9158099cb0 tx=0x7f91580cd190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=918a26f3c3e0c11 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f91517fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f91517fa700 1 -- 172.21.15.66:0/2471197682 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580b1070 con 0x561b2c6ac1b0 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f91517fa700 1 -- 172.21.15.66:0/2471197682 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f915809dfe0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b2c5524f0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f91517fa700 1 -- 172.21.15.66:0/2471197682 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91580a2400 con 0x561b2c6ac1b0 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f9183914b80 10 client.26200 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561b2c40fc60 con 0x561b2c6ac1b0 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f91517fa700 1 -- 172.21.15.66:0/2471197682 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91580c8080 con 0x561b2c6ac1b0 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.914+0000 7f91517fa700 1 -- 172.21.15.66:0/2471197682 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f91580a1f70 con 0x561b2c6ac1b0 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f9183914b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f9183914b80 10 client.26200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f9183914b80 20 client.26200 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:37.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f9183914b80 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c6c1410 0x561b2c6c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f9183914b80 4 client.26200 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:37.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b2c5f1eb0 con 0x561b2c6c1410 2023-07-21T22:24:37.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f916dd9b700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c6c1410 0x561b2c6c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f91517fa700 10 client.26200 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:37.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.915+0000 7f916dd9b700 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c6c1410 0x561b2c6c3820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4270cc5a6d2e6ac3 server_cookie=69648213e59f8a22 in_seq=0 out_seq=0 2023-07-21T22:24:37.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.916+0000 7f91517fa700 1 -- 172.21.15.66:0/2471197682 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561b2c5f1eb0 con 0x561b2c6c1410 2023-07-21T22:24:37.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.916+0000 7f91517fa700 10 client.26200 handle_command_reply: tid=0 2023-07-21T22:24:37.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.916+0000 7f9183914b80 1 client.26200 shutdown 2023-07-21T22:24:37.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.916+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 msgr2=0x561b2c6b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.916+0000 7f9183914b80 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b13d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9158099cb0 tx=0x7f91580cd190 comp rx=0 tx=0).stop 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 shutdown_connections 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c6c1410 0x561b2c6c3820 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6acac0 0x561b2c6b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f91517fa700 0 client.26200 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 --2- 172.21.15.66:0/2471197682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b13d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 >> 172.21.15.66:0/2471197682 conn(0x561b2c69cef0 msgr2=0x561b2c6a2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 shutdown_connections 2023-07-21T22:24:37.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 1 -- 172.21.15.66:0/2471197682 wait complete. 2023-07-21T22:24:37.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.917+0000 7f9183914b80 20 client.26200 trim_cache size 0 max 16384 2023-07-21T22:24:37.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 Processor -- start 2023-07-21T22:24:37.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 -- start start 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 0x561b2c6a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c69c9e0 con 0x561b2c6acac0 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c69f010 con 0x561b2c6a7930 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f9183914b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c6bebc0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f916dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 0x561b2c6a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f916d59a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f916dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 0x561b2c6a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50750/0 (socket says 172.21.15.66:50750) 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f916d59a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47680/0 (socket says 172.21.15.66:47680) 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f916dd9b700 1 -- 172.21.15.66:0/914810877 learned_addr learned my addr 172.21.15.66:0/914810877 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.933+0000 7f916e59c700 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916d59a700 1 -- 172.21.15.66:0/914810877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 msgr2=0x561b2c6a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916d59a700 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 0x561b2c6a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916d59a700 1 -- 172.21.15.66:0/914810877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 msgr2=0x561b2c6b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916d59a700 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916d59a700 1 -- 172.21.15.66:0/914810877 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b2c6a7fd0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916dd9b700 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 0x561b2c6a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916e59c700 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:37.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f916d59a700 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9158098300 tx=0x7f9158010a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad116a06f04cf9bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f915cff9700 1 -- 172.21.15.66:0/914810877 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580b1050 con 0x561b2c6ac1b0 2023-07-21T22:24:37.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f915cff9700 1 -- 172.21.15.66:0/914810877 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b2c5524f0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.934+0000 7f915cff9700 1 -- 172.21.15.66:0/914810877 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580b22c0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 -- 172.21.15.66:0/914810877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 msgr2=0x561b2c6b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9158098300 tx=0x7f9158010a90 comp rx=0 tx=0).stop 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 -- 172.21.15.66:0/914810877 shutdown_connections 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6a7930 0x561b2c6a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6acac0 0x561b2c6b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 --2- 172.21.15.66:0/914810877 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 -- 172.21.15.66:0/914810877 >> 172.21.15.66:0/914810877 conn(0x561b2c69cef0 msgr2=0x561b2c6a69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 -- 172.21.15.66:0/914810877 shutdown_connections 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.935+0000 7f9183914b80 1 -- 172.21.15.66:0/914810877 wait complete. 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.936+0000 7f9183914b80 1 -- start start 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.936+0000 7f9183914b80 1 Processor -- start 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6acac0 0x561b2c6b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c6b6dd0 con 0x561b2c6a7930 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c6b6f40 con 0x561b2c6ac1b0 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b2c6b7100 con 0x561b2c6acac0 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f9183914b80 10 client.0 _ll_register_callbacks cb 0x7f91580cd0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f916dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f916d59a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f916d59a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50754/0 (socket says 172.21.15.66:50754) 2023-07-21T22:24:37.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f916d59a700 1 -- 172.21.15.66:0/2520863758 learned_addr learned my addr 172.21.15.66:0/2520863758 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.937+0000 7f916e59c700 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6acac0 0x561b2c6b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f916d59a700 1 -- 172.21.15.66:0/2520863758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6acac0 msgr2=0x561b2c6b6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f916d59a700 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6acac0 0x561b2c6b6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f916d59a700 1 -- 172.21.15.66:0/2520863758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 msgr2=0x561b2c6b8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f916d59a700 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f916d59a700 1 -- 172.21.15.66:0/2520863758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b2c426de0 con 0x561b2c6ac1b0 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f916d59a700 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91580964b0 tx=0x7f9158099730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00f4a649780a684 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:37.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f91517fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f91517fa700 1 -- 172.21.15.66:0/2520863758 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580b1070 con 0x561b2c6ac1b0 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f91517fa700 1 -- 172.21.15.66:0/2520863758 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f91580b4e40 con 0x561b2c6ac1b0 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f916401e040 con 0x561b2c6ac1b0 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f91517fa700 1 -- 172.21.15.66:0/2520863758 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91580a2400 con 0x561b2c6ac1b0 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f9183914b80 10 client.26228 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.938+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561b2c40fc60 con 0x561b2c6ac1b0 2023-07-21T22:24:37.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f91517fa700 1 -- 172.21.15.66:0/2520863758 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f916401e040 con 0x561b2c6ac1b0 2023-07-21T22:24:37.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f91517fa700 1 -- 172.21.15.66:0/2520863758 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f91580ca330 con 0x561b2c6ac1b0 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f9183914b80 10 client.26228 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f9183914b80 10 client.26228 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f9183914b80 20 client.26228 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f9183914b80 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c716210 0x561b2c7185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f9183914b80 4 client.26228 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b2c5b4050 con 0x561b2c716210 2023-07-21T22:24:37.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.939+0000 7f916dd9b700 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c716210 0x561b2c7185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f91517fa700 10 client.26228 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f916dd9b700 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c716210 0x561b2c7185f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1b2df6be60ca347 server_cookie=db5180a09a6502ff in_seq=0 out_seq=0 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f91517fa700 1 -- 172.21.15.66:0/2520863758 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b2c5b4050 con 0x561b2c716210 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f91517fa700 10 client.26228 handle_command_reply: tid=0 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 client.26228 shutdown 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 msgr2=0x561b2c6b8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8750 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f91580964b0 tx=0x7f9158099730 comp rx=0 tx=0).stop 2023-07-21T22:24:37.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 shutdown_connections 2023-07-21T22:24:37.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561b2c6acac0 0x561b2c6b6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561b2c6a7930 0x561b2c6b8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561b2c716210 0x561b2c7185f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 --2- 172.21.15.66:0/2520863758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b2c6ac1b0 0x561b2c6b8750 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 >> 172.21.15.66:0/2520863758 conn(0x561b2c69cef0 msgr2=0x561b2c6b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f91517fa700 0 client.26228 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:24:37.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 shutdown_connections 2023-07-21T22:24:37.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 1 -- 172.21.15.66:0/2520863758 wait complete. 2023-07-21T22:24:37.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.940+0000 7f9183914b80 20 client.26228 trim_cache size 0 max 16384 2023-07-21T22:24:37.993 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:24:37.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91540a12c0 msgr2=0x7f91540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91540a12c0 0x7f91540a3770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9168074920 tx=0x7f916000b040 comp rx=0 tx=0).stop 2023-07-21T22:24:37.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 msgr2=0x7f9168072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:37.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 0x7f9168072d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f915800e9a0 tx=0x7f915800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 shutdown_connections 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91540a12c0 0x7f91540a3770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9168105ae0 0x7f91681a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91680ffad0 0x7f9168072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 --2- 172.21.15.66:0/3581877963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168100480 0x7f9168072d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 >> 172.21.15.66:0/3581877963 conn(0x7f91680fb5f0 msgr2=0x7f91680fbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 shutdown_connections 2023-07-21T22:24:37.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:37.941+0000 7f916ffff700 1 -- 172.21.15.66:0/3581877963 wait complete. 2023-07-21T22:24:38.004 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:24:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 -- 172.21.15.66:0/1189617446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe97010a0a0 msgr2=0x7fe97010c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 --2- 172.21.15.66:0/1189617446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe97010a0a0 0x7fe97010c4c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe968009f90 tx=0x7fe96800b330 comp rx=0 tx=0).stop 2023-07-21T22:24:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 -- 172.21.15.66:0/1189617446 shutdown_connections 2023-07-21T22:24:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 --2- 172.21.15.66:0/1189617446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe97010a0a0 0x7fe97010c4c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 --2- 172.21.15.66:0/1189617446 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970100b80 0x7fe970100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 --2- 172.21.15.66:0/1189617446 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe970101490 0x7fe970109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.184+0000 7fe976bff700 1 -- 172.21.15.66:0/1189617446 >> 172.21.15.66:0/1189617446 conn(0x7fe9700fa4b0 msgr2=0x7fe9700fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:38.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.185+0000 7fe976bff700 1 -- 172.21.15.66:0/1189617446 shutdown_connections 2023-07-21T22:24:38.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.185+0000 7fe976bff700 1 -- 172.21.15.66:0/1189617446 wait complete. 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.186+0000 7fe976bff700 1 Processor -- start 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.186+0000 7fe976bff700 1 -- start start 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.186+0000 7fe976bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 0x7fe970197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.186+0000 7fe976bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970101490 0x7fe970198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.186+0000 7fe976bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97010a0a0 0x7fe97019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe976bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97019cfa0 con 0x7fe97010a0a0 2023-07-21T22:24:38.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe976bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97019d110 con 0x7fe970100b80 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe976bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97019d2d0 con 0x7fe970101490 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 0x7fe970197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 0x7fe970197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50774/0 (socket says 172.21.15.66:50774) 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 -- 172.21.15.66:0/2806431651 learned_addr learned my addr 172.21.15.66:0/2806431651 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe96ffff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970101490 0x7fe970198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97519c700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97010a0a0 0x7fe97019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 -- 172.21.15.66:0/2806431651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970101490 msgr2=0x7fe970198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970101490 0x7fe970198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 -- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97010a0a0 msgr2=0x7fe97019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97010a0a0 0x7fe97019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 -- 172.21.15.66:0/2806431651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9640095f0 con 0x7fe970100b80 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97519c700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97010a0a0 0x7fe97019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe96ffff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970101490 0x7fe970198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.187+0000 7fe97499b700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 0x7fe970197fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe96400ec10 tx=0x7fe96400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.188+0000 7fe96dffb700 1 -- 172.21.15.66:0/2806431651 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96400cd40 con 0x7fe970100b80 2023-07-21T22:24:38.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.188+0000 7fe96dffb700 1 -- 172.21.15.66:0/2806431651 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9640044e0 con 0x7fe970100b80 2023-07-21T22:24:38.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.188+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c009520 con 0x7fe970100b80 2023-07-21T22:24:38.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.188+0000 7fe96dffb700 1 -- 172.21.15.66:0/2806431651 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964010690 con 0x7fe970100b80 2023-07-21T22:24:38.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.188+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe968009c40 con 0x7fe970100b80 2023-07-21T22:24:38.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.191+0000 7fe96dffb700 1 -- 172.21.15.66:0/2806431651 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe964003710 con 0x7fe970100b80 2023-07-21T22:24:38.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 Processor -- start 2023-07-21T22:24:38.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 -- start start 2023-07-21T22:24:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d83580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d7e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d7eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79c83cb0 con 0x55ef79d83ac0 2023-07-21T22:24:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79cd6a40 con 0x55ef79d7e930 2023-07-21T22:24:38.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe98a572b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79ccfed0 con 0x55ef79d831b0 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe96ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d7eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe97499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d7e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.193+0000 7fe97519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d83580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d83580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47726/0 (socket says 172.21.15.66:47726) 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 -- 172.21.15.66:0/898890683 learned_addr learned my addr 172.21.15.66:0/898890683 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe96dffb700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9600a12c0 0x7fe9600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 -- 172.21.15.66:0/898890683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 msgr2=0x55ef79d7eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d7eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 -- 172.21.15.66:0/898890683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 msgr2=0x55ef79d7e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d7e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 -- 172.21.15.66:0/898890683 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef79afdde0 con 0x55ef79d831b0 2023-07-21T22:24:38.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe96dffb700 1 -- 172.21.15.66:0/2806431651 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe964014070 con 0x7fe970100b80 2023-07-21T22:24:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.194+0000 7fe97519c700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9600a12c0 0x7fe9600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.195+0000 7fe97519c700 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d83580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9680093d0 tx=0x7fe968009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e003014a4d1799 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.195+0000 7fe95affd700 1 -- 172.21.15.66:0/898890683 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800e070 con 0x55ef79d831b0 2023-07-21T22:24:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.195+0000 7fe95affd700 1 -- 172.21.15.66:0/898890683 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe968004bb0 con 0x55ef79d831b0 2023-07-21T22:24:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.195+0000 7fe95affd700 1 -- 172.21.15.66:0/898890683 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe968005540 con 0x55ef79d831b0 2023-07-21T22:24:38.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 -- 172.21.15.66:0/898890683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 msgr2=0x55ef79d83580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe97519c700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9600a12c0 0x7fe9600a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe968026000 tx=0x7fe968029000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d83580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe9680093d0 tx=0x7fe968009400 comp rx=0 tx=0).stop 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 -- 172.21.15.66:0/898890683 shutdown_connections 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d7eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d83580 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 --2- 172.21.15.66:0/898890683 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d7e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 -- 172.21.15.66:0/898890683 >> 172.21.15.66:0/898890683 conn(0x55ef79d73ef0 msgr2=0x55ef79d79830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 -- 172.21.15.66:0/898890683 shutdown_connections 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.196+0000 7fe98a572b80 1 -- 172.21.15.66:0/898890683 wait complete. 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 -- start start 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 Processor -- start 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 0x55ef79d8ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d831b0 0x55ef79d87e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d88520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79c6fac0 con 0x55ef79d7e930 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79c6f7c0 con 0x55ef79d831b0 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.197+0000 7fe98a572b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79b53920 con 0x55ef79d83ac0 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe98a572b80 10 client.0 _ll_register_callbacks cb 0x55ef79c6d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 0x55ef79d8ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d88520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 0x55ef79d8ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35612/0 (socket says 172.21.15.66:35612) 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97499b700 1 -- 172.21.15.66:0/790922247 learned_addr learned my addr 172.21.15.66:0/790922247 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:38.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe96ffff700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d831b0 0x55ef79d87e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97519c700 1 -- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d831b0 msgr2=0x55ef79d87e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97519c700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d831b0 0x55ef79d87e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97519c700 1 -- 172.21.15.66:0/790922247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 msgr2=0x55ef79d8ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97519c700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 0x55ef79d8ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97519c700 1 -- 172.21.15.66:0/790922247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97010d180 con 0x55ef79d83ac0 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.198+0000 7fe97499b700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 0x55ef79d8ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:38.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe97519c700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d88520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9680093d0 tx=0x7fe9680051c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93cc72dbd7e22abe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe94ffff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe94ffff700 1 -- 172.21.15.66:0/790922247 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96800e040 con 0x55ef79d83ac0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe94ffff700 1 -- 172.21.15.66:0/790922247 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe9680057b0 con 0x55ef79d83ac0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef79c294f0 con 0x55ef79d83ac0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe94ffff700 1 -- 172.21.15.66:0/790922247 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe968005970 con 0x55ef79d83ac0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe98a572b80 10 client.26218 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.199+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ef79ae6c60 con 0x55ef79d83ac0 2023-07-21T22:24:38.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe94ffff700 1 -- 172.21.15.66:0/790922247 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef79c294f0 con 0x55ef79d83ac0 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe94ffff700 1 -- 172.21.15.66:0/790922247 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe968030670 con 0x55ef79d83ac0 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe98a572b80 10 client.26218 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe98a572b80 10 client.26218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe98a572b80 20 client.26218 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe98a572b80 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79d98100 0x55ef79d9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe98a572b80 4 client.26218 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:38.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef79cc8eb0 con 0x55ef79d98100 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.200+0000 7fe97499b700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79d98100 0x55ef79d9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.201+0000 7fe97499b700 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79d98100 0x55ef79d9a510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a4ba55d8f8e932 server_cookie=f5178229e06aebae in_seq=0 out_seq=0 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.201+0000 7fe94ffff700 10 client.26218 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.201+0000 7fe94ffff700 1 -- 172.21.15.66:0/790922247 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef79cc8eb0 con 0x55ef79d98100 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.201+0000 7fe94ffff700 10 client.26218 handle_command_reply: tid=0 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.201+0000 7fe98a572b80 1 client.26218 shutdown 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 msgr2=0x55ef79d88520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d88520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe9680093d0 tx=0x7fe9680051c0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 shutdown_connections 2023-07-21T22:24:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79d98100 0x55ef79d9a510 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d88520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe94ffff700 0 client.26218 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d7e930 0x55ef79d8ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 --2- 172.21.15.66:0/790922247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d831b0 0x55ef79d87e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 >> 172.21.15.66:0/790922247 conn(0x55ef79d73ef0 msgr2=0x55ef79d79420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 shutdown_connections 2023-07-21T22:24:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 1 -- 172.21.15.66:0/790922247 wait complete. 2023-07-21T22:24:38.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.202+0000 7fe98a572b80 20 client.26218 trim_cache size 0 max 16384 2023-07-21T22:24:38.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 Processor -- start 2023-07-21T22:24:38.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 -- start start 2023-07-21T22:24:38.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.264 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d831b0 0x55ef79d8a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d7ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79d79dd0 con 0x55ef79d831b0 2023-07-21T22:24:38.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79d739b0 con 0x55ef79d7e930 2023-07-21T22:24:38.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe98a572b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79d76010 con 0x55ef79d83ac0 2023-07-21T22:24:38.265 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe97499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe96ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d831b0 0x55ef79d8a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.218+0000 7fe97519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d7ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50796/0 (socket says 172.21.15.66:50796) 2023-07-21T22:24:38.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 -- 172.21.15.66:0/972349386 learned_addr learned my addr 172.21.15.66:0/972349386 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:38.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 -- 172.21.15.66:0/972349386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 msgr2=0x55ef79d7ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d7ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 -- 172.21.15.66:0/972349386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d831b0 msgr2=0x55ef79d8a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d831b0 0x55ef79d8a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 -- 172.21.15.66:0/972349386 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef79b24250 con 0x55ef79d7e930 2023-07-21T22:24:38.269 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe96ffff700 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d831b0 0x55ef79d8a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe97499b700 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe964003ac0 tx=0x7fe9640d0f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de3cbf6f5388d92e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe95b7fe700 1 -- 172.21.15.66:0/972349386 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe964009a10 con 0x55ef79d7e930 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe95b7fe700 1 -- 172.21.15.66:0/972349386 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9640cc050 con 0x55ef79d7e930 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.219+0000 7fe95b7fe700 1 -- 172.21.15.66:0/972349386 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9640d0a30 con 0x55ef79d7e930 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 -- 172.21.15.66:0/972349386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 msgr2=0x55ef79d8f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe964003ac0 tx=0x7fe9640d0f00 comp rx=0 tx=0).stop 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 -- 172.21.15.66:0/972349386 shutdown_connections 2023-07-21T22:24:38.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f570 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d83ac0 0x55ef79d7ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 --2- 172.21.15.66:0/972349386 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d831b0 0x55ef79d8a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 -- 172.21.15.66:0/972349386 >> 172.21.15.66:0/972349386 conn(0x55ef79d73ef0 msgr2=0x55ef79d792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 -- 172.21.15.66:0/972349386 shutdown_connections 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.220+0000 7fe98a572b80 1 -- 172.21.15.66:0/972349386 wait complete. 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.221+0000 7fe98a572b80 1 -- start start 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.221+0000 7fe98a572b80 1 Processor -- start 2023-07-21T22:24:38.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d864b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d869f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79d86f30 con 0x55ef79d83ac0 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79d870a0 con 0x55ef79d7e930 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef79d87260 con 0x55ef79d831b0 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe98a572b80 10 client.0 _ll_register_callbacks cb 0x7fe9640d0dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:38.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe96ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d864b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe97499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe96ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d864b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47752/0 (socket says 172.21.15.66:47752) 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe96ffff700 1 -- 172.21.15.66:0/1729659522 learned_addr learned my addr 172.21.15.66:0/1729659522 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe97519c700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d869f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.222+0000 7fe97499b700 1 -- 172.21.15.66:0/1729659522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 msgr2=0x55ef79d864b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe97499b700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d864b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe97499b700 1 -- 172.21.15.66:0/1729659522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 msgr2=0x55ef79d869f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:38.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe97499b700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d869f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe97499b700 1 -- 172.21.15.66:0/1729659522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0104e0 con 0x55ef79d7e930 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe97519c700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe96ffff700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe97499b700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c660 tx=0x7fe964003b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45e2f89758d26eee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe94ffff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe94ffff700 1 -- 172.21.15.66:0/1729659522 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96409a140 con 0x55ef79d7e930 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe94ffff700 1 -- 172.21.15.66:0/1729659522 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe964016330 con 0x55ef79d7e930 2023-07-21T22:24:38.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe968004bb0 con 0x55ef79d7e930 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe94ffff700 1 -- 172.21.15.66:0/1729659522 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe964099eb0 con 0x55ef79d7e930 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe98a572b80 10 client.26246 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.223+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fe968009c40 con 0x55ef79d7e930 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe94ffff700 1 -- 172.21.15.66:0/1729659522 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe968004bb0 con 0x55ef79d7e930 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe94ffff700 1 -- 172.21.15.66:0/1729659522 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe9640cd260 con 0x55ef79d7e930 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe98a572b80 10 client.26246 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:38.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe98a572b80 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:38.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe98a572b80 20 client.26246 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:38.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe98a572b80 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79ded230 0x55ef79def640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:38.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe98a572b80 4 client.26246 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:24:38.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ef79c8b050 con 0x55ef79ded230 2023-07-21T22:24:38.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.224+0000 7fe96ffff700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79ded230 0x55ef79def640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:38.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe94ffff700 10 client.26246 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:38.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe96ffff700 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79ded230 0x55ef79def640 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801f46be0a3ffbe3 server_cookie=4fab40272955d897 in_seq=0 out_seq=0 2023-07-21T22:24:38.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe94ffff700 1 -- 172.21.15.66:0/1729659522 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ef79c8b050 con 0x55ef79ded230 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe94ffff700 10 client.26246 handle_command_reply: tid=0 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 client.26246 shutdown 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 msgr2=0x55ef79d8f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe96400c660 tx=0x7fe964003b10 comp rx=0 tx=0).stop 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 shutdown_connections 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ef79d7e930 0x55ef79d8f580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ef79ded230 0x55ef79def640 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ef79d83ac0 0x55ef79d869f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 --2- 172.21.15.66:0/1729659522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ef79d831b0 0x55ef79d864b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe94ffff700 0 client.26246 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 >> 172.21.15.66:0/1729659522 conn(0x55ef79d73ef0 msgr2=0x55ef79d7ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 shutdown_connections 2023-07-21T22:24:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 1 -- 172.21.15.66:0/1729659522 wait complete. 2023-07-21T22:24:38.279 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.225+0000 7fe98a572b80 20 client.26246 trim_cache size 0 max 16384 2023-07-21T22:24:38.280 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:24:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9600a12c0 msgr2=0x7fe9600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9600a12c0 0x7fe9600a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe968026000 tx=0x7fe968029000 comp rx=0 tx=0).stop 2023-07-21T22:24:38.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 msgr2=0x7fe970197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:38.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 0x7fe970197fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe96400ec10 tx=0x7fe96400ef20 comp rx=0 tx=0).stop 2023-07-21T22:24:38.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 shutdown_connections 2023-07-21T22:24:38.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe970100b80 0x7fe970197fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9600a12c0 0x7fe9600a3770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe97010a0a0 0x7fe97019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 --2- 172.21.15.66:0/2806431651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe970101490 0x7fe970198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:38.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 >> 172.21.15.66:0/2806431651 conn(0x7fe9700fa4b0 msgr2=0x7fe97010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:38.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 shutdown_connections 2023-07-21T22:24:38.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:38.226+0000 7fe976bff700 1 -- 172.21.15.66:0/2806431651 wait complete. 2023-07-21T22:24:38.289 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:37 smithi066 ceph-mon[126379]: pgmap v97: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-07-21T22:24:38.289 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:37 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3993670438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:24:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:37 smithi104 ceph-mon[129021]: pgmap v97: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-07-21T22:24:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:37 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3993670438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:24:38.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:37 smithi149 ceph-mon[129268]: pgmap v97: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-07-21T22:24:38.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:37 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3993670438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:24:38.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 49320 2.55 MB/sec execute 5 sec latency 4942.218 ms 2023-07-21T22:24:39.293 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:24:39.293 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:24:39.357 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:38 smithi066 ceph-mon[126379]: pgmap v98: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-07-21T22:24:39.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:38 smithi104 ceph-mon[129021]: pgmap v98: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-07-21T22:24:39.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:38 smithi149 ceph-mon[129268]: pgmap v98: 129 pgs: 129 active+clean; 48 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 141 KiB/s wr, 15 op/s 2023-07-21T22:24:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 -- 172.21.15.66:0/3744722596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 msgr2=0x7f7130105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 --2- 172.21.15.66:0/3744722596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f7130105ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7120009990 tx=0x7f7120009ca0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 -- 172.21.15.66:0/3744722596 shutdown_connections 2023-07-21T22:24:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 --2- 172.21.15.66:0/3744722596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7130100490 0x7f713010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 --2- 172.21.15.66:0/3744722596 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f7130105ec0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 --2- 172.21.15.66:0/3744722596 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71300ffae0 0x7f71300fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 -- 172.21.15.66:0/3744722596 >> 172.21.15.66:0/3744722596 conn(0x7f71300fb5e0 msgr2=0x7f71300fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 -- 172.21.15.66:0/3744722596 shutdown_connections 2023-07-21T22:24:39.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.473+0000 7f71369ff700 1 -- 172.21.15.66:0/3744722596 wait complete. 2023-07-21T22:24:39.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.474+0000 7f71369ff700 1 Processor -- start 2023-07-21T22:24:39.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.474+0000 7f71369ff700 1 -- start start 2023-07-21T22:24:39.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f71369ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71300ffae0 0x7f7130197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f71369ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7130100490 0x7f7130198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f71369ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f713019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f71369ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713019cd80 con 0x7f7130105af0 2023-07-21T22:24:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f71369ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713019cef0 con 0x7f7130100490 2023-07-21T22:24:39.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f71369ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713019d0b0 con 0x7f71300ffae0 2023-07-21T22:24:39.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 50479 3.19 MB/sec execute 6 sec latency 30.978 ms 2023-07-21T22:24:39.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f713019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f713019c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35646/0 (socket says 172.21.15.66:35646) 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 -- 172.21.15.66:0/2908418891 learned_addr learned my addr 172.21.15.66:0/2908418891 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f712f7fe700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7130100490 0x7f7130198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f712ffff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71300ffae0 0x7f7130197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 -- 172.21.15.66:0/2908418891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71300ffae0 msgr2=0x7f7130197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71300ffae0 0x7f7130197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 -- 172.21.15.66:0/2908418891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7130100490 msgr2=0x7f7130198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7130100490 0x7f7130198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.475+0000 7f7134f9c700 1 -- 172.21.15.66:0/2908418891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200095f0 con 0x7f7130105af0 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.476+0000 7f7134f9c700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f713019c740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f712800ef30 tx=0x7f712800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.476+0000 7f712d7fa700 1 -- 172.21.15.66:0/2908418891 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7128018070 con 0x7f7130105af0 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.476+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713019d330 con 0x7f7130105af0 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.476+0000 7f712d7fa700 1 -- 172.21.15.66:0/2908418891 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712800f040 con 0x7f7130105af0 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.476+0000 7f712d7fa700 1 -- 172.21.15.66:0/2908418891 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800cd00 con 0x7f7130105af0 2023-07-21T22:24:39.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.476+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71301a4bd0 con 0x7f7130105af0 2023-07-21T22:24:39.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.481+0000 7f712d7fa700 1 -- 172.21.15.66:0/2908418891 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7128003a00 con 0x7f7130105af0 2023-07-21T22:24:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.482+0000 7f714a2c5b80 1 Processor -- start 2023-07-21T22:24:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.482+0000 7f714a2c5b80 1 -- start start 2023-07-21T22:24:39.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712d7fa700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71180a12c0 0x7f71180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712d7fa700 1 -- 172.21.15.66:0/2908418891 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f71280ccb40 con 0x7f7130105af0 2023-07-21T22:24:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712f7fe700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71180a12c0 0x7f71180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd11453d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd104ad30 con 0x557dd114ab40 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd109dac0 con 0x557dd114a230 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1096f50 con 0x557dd11459b0 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50840/0 (socket says 172.21.15.66:50840) 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712ffff700 1 -- 172.21.15.66:0/154342094 learned_addr learned my addr 172.21.15.66:0/154342094 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.483+0000 7f712f7fe700 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd11453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f7134f9c700 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712ffff700 1 -- 172.21.15.66:0/154342094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 msgr2=0x557dd1145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712ffff700 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712ffff700 1 -- 172.21.15.66:0/154342094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 msgr2=0x557dd11453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712ffff700 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd11453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712ffff700 1 -- 172.21.15.66:0/154342094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dd0ec4e60 con 0x557dd114a230 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712f7fe700 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd11453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:39.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712f7fe700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71180a12c0 0x7f71180a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71301993b0 tx=0x7f7124013900 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f712ffff700 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114a600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7120009990 tx=0x7f7120005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb87fc52cb7c026 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.484+0000 7f71167fc700 1 -- 172.21.15.66:0/154342094 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712001d030 con 0x557dd114a230 2023-07-21T22:24:39.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f71167fc700 1 -- 172.21.15.66:0/154342094 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712001e040 con 0x557dd114a230 2023-07-21T22:24:39.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f71167fc700 1 -- 172.21.15.66:0/154342094 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200048b0 con 0x557dd114a230 2023-07-21T22:24:39.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 -- 172.21.15.66:0/154342094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 msgr2=0x557dd114a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114a600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7120009990 tx=0x7f7120005f00 comp rx=0 tx=0).stop 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 -- 172.21.15.66:0/154342094 shutdown_connections 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114a600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/154342094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd11453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 -- 172.21.15.66:0/154342094 >> 172.21.15.66:0/154342094 conn(0x557dd113af70 msgr2=0x557dd11408b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 -- 172.21.15.66:0/154342094 shutdown_connections 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.485+0000 7f714a2c5b80 1 -- 172.21.15.66:0/154342094 wait complete. 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.486+0000 7f714a2c5b80 1 -- start start 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.486+0000 7f714a2c5b80 1 Processor -- start 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd114ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1036b40 con 0x557dd114ab40 2023-07-21T22:24:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1036840 con 0x557dd114a230 2023-07-21T22:24:39.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd0f1a9a0 con 0x557dd11459b0 2023-07-21T22:24:39.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f714a2c5b80 10 client.0 _ll_register_callbacks cb 0x557dd10347e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:39.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd114ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f7134f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50854/0 (socket says 172.21.15.66:50854) 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd114ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47786/0 (socket says 172.21.15.66:47786) 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712f7fe700 1 -- 172.21.15.66:0/2381715890 learned_addr learned my addr 172.21.15.66:0/2381715890 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 -- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 msgr2=0x557dd114f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 -- 172.21.15.66:0/2381715890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 msgr2=0x557dd114f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712ffff700 1 -- 172.21.15.66:0/2381715890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713010d220 con 0x557dd11459b0 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f712f7fe700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.487+0000 7f7134f9c700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f712ffff700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd114ef20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f712000b6f0 tx=0x7f712000f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad6ac0abc1a5a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f710f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f710f7fe700 1 -- 172.21.15.66:0/2381715890 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120032070 con 0x557dd11459b0 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557dd0ff0570 con 0x557dd11459b0 2023-07-21T22:24:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f710f7fe700 1 -- 172.21.15.66:0/2381715890 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f712000fab0 con 0x557dd11459b0 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f710f7fe700 1 -- 172.21.15.66:0/2381715890 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f712001d040 con 0x557dd11459b0 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f714a2c5b80 10 client.26236 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.488+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557dd0eadce0 con 0x557dd11459b0 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f710f7fe700 1 -- 172.21.15.66:0/2381715890 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557dd0ff0570 con 0x557dd11459b0 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f710f7fe700 1 -- 172.21.15.66:0/2381715890 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7120035030 con 0x557dd11459b0 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f714a2c5b80 10 client.26236 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f714a2c5b80 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f714a2c5b80 20 client.26236 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd115f0d0 0x557dd11614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f714a2c5b80 4 client.26236 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dd108ff30 con 0x557dd115f0d0 2023-07-21T22:24:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.489+0000 7f712f7fe700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd115f0d0 0x557dd11614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.490+0000 7f712f7fe700 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd115f0d0 0x557dd11614b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc908421b59ef1f server_cookie=10c6b14a8628b1e5 in_seq=0 out_seq=0 2023-07-21T22:24:39.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.490+0000 7f710f7fe700 10 client.26236 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.490+0000 7f710f7fe700 1 -- 172.21.15.66:0/2381715890 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557dd108ff30 con 0x557dd115f0d0 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.490+0000 7f710f7fe700 10 client.26236 handle_command_reply: tid=0 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.490+0000 7f714a2c5b80 1 client.26236 shutdown 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 msgr2=0x557dd114ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd114ef20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f712000b6f0 tx=0x7f712000f670 comp rx=0 tx=0).stop 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 shutdown_connections 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd115f0d0 0x557dd11614b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f710f7fe700 0 client.26236 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd114ef20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/2381715890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 >> 172.21.15.66:0/2381715890 conn(0x557dd113af70 msgr2=0x557dd11404a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 shutdown_connections 2023-07-21T22:24:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 1 -- 172.21.15.66:0/2381715890 wait complete. 2023-07-21T22:24:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.491+0000 7f714a2c5b80 20 client.26236 trim_cache size 0 max 16384 2023-07-21T22:24:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.509+0000 7f714a2c5b80 1 Processor -- start 2023-07-21T22:24:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.509+0000 7f714a2c5b80 1 -- start start 2023-07-21T22:24:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1146260 con 0x557dd114ab40 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd114cf20 con 0x557dd114a230 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1161ae0 con 0x557dd11459b0 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47790/0 (socket says 172.21.15.66:47790) 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 -- 172.21.15.66:0/276642659 learned_addr learned my addr 172.21.15.66:0/276642659 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f7134f9c700 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712f7fe700 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 -- 172.21.15.66:0/276642659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 msgr2=0x557dd114f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 -- 172.21.15.66:0/276642659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 msgr2=0x557dd114de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.510+0000 7f712ffff700 1 -- 172.21.15.66:0/276642659 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dd0eeb2d0 con 0x557dd11459b0 2023-07-21T22:24:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f712ffff700 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7120009990 tx=0x7f7120009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42600baad5bce977 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f7116ffd700 1 -- 172.21.15.66:0/276642659 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120032040 con 0x557dd11459b0 2023-07-21T22:24:39.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f7116ffd700 1 -- 172.21.15.66:0/276642659 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71200163a0 con 0x557dd11459b0 2023-07-21T22:24:39.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f7116ffd700 1 -- 172.21.15.66:0/276642659 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200053f0 con 0x557dd11459b0 2023-07-21T22:24:39.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 -- 172.21.15.66:0/276642659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 msgr2=0x557dd1145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145310 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7120009990 tx=0x7f7120009310 comp rx=0 tx=0).stop 2023-07-21T22:24:39.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 -- 172.21.15.66:0/276642659 shutdown_connections 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd114de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd11459b0 0x557dd1145310 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/276642659 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd114a230 0x557dd114f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 -- 172.21.15.66:0/276642659 >> 172.21.15.66:0/276642659 conn(0x557dd113af70 msgr2=0x557dd114d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.511+0000 7f714a2c5b80 1 -- 172.21.15.66:0/276642659 shutdown_connections 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.512+0000 7f714a2c5b80 1 -- 172.21.15.66:0/276642659 wait complete. 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.512+0000 7f714a2c5b80 1 -- start start 2023-07-21T22:24:39.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.512+0000 7f714a2c5b80 1 Processor -- start 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 0x557dd1156570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd114a230 0x557dd11549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd1154f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd1155580 con 0x557dd114ab40 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd11556f0 con 0x557dd11459b0 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dd11b2f00 con 0x557dd114a230 2023-07-21T22:24:39.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f714a2c5b80 10 client.0 _ll_register_callbacks cb 0x7f7128004cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f712ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 0x557dd1156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f712ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 0x557dd1156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50868/0 (socket says 172.21.15.66:50868) 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f712ffff700 1 -- 172.21.15.66:0/3990270300 learned_addr learned my addr 172.21.15.66:0/3990270300 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f7134f9c700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd1154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.513+0000 7f712f7fe700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd114a230 0x557dd11549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712ffff700 1 -- 172.21.15.66:0/3990270300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd114a230 msgr2=0x557dd11549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712ffff700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd114a230 0x557dd11549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712ffff700 1 -- 172.21.15.66:0/3990270300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 msgr2=0x557dd1154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712ffff700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd1154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712ffff700 1 -- 172.21.15.66:0/3990270300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dd0eadce0 con 0x557dd11459b0 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712f7fe700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd114a230 0x557dd11549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f712ffff700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 0x557dd1156570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7120005580 tx=0x7f7120023e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578d9442c1cc84c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f710f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f710f7fe700 1 -- 172.21.15.66:0/3990270300 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712001d030 con 0x557dd11459b0 2023-07-21T22:24:39.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f710f7fe700 1 -- 172.21.15.66:0/3990270300 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7120000e40 con 0x557dd11459b0 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557dd10520d0 con 0x557dd11459b0 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f710f7fe700 1 -- 172.21.15.66:0/3990270300 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7120032070 con 0x557dd11459b0 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f714a2c5b80 10 client.26264 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.514+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x557dd0ec3330 con 0x557dd11459b0 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f710f7fe700 1 -- 172.21.15.66:0/3990270300 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557dd10520d0 con 0x557dd11459b0 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f710f7fe700 1 -- 172.21.15.66:0/3990270300 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7120035070 con 0x557dd11459b0 2023-07-21T22:24:39.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f714a2c5b80 10 client.26264 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f714a2c5b80 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f714a2c5b80 20 client.26264 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd11b4120 0x557dd11b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f714a2c5b80 4 client.26264 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:24:39.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557dd0ee5ff0 con 0x557dd11b4120 2023-07-21T22:24:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.515+0000 7f712f7fe700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd11b4120 0x557dd11b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.516+0000 7f712f7fe700 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd11b4120 0x557dd11b6530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dd91084d43387c6 server_cookie=f1edb75b938e9d3e in_seq=0 out_seq=0 2023-07-21T22:24:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.516+0000 7f710f7fe700 10 client.26264 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f710f7fe700 1 -- 172.21.15.66:0/3990270300 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557dd0ee5ff0 con 0x557dd11b4120 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f710f7fe700 10 client.26264 handle_command_reply: tid=0 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 client.26264 shutdown 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 msgr2=0x557dd1156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 0x557dd1156570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7120005580 tx=0x7f7120023e40 comp rx=0 tx=0).stop 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 shutdown_connections 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557dd11459b0 0x557dd1156570 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557dd11b4120 0x557dd11b6530 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557dd114ab40 0x557dd1154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f710f7fe700 0 client.26264 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 --2- 172.21.15.66:0/3990270300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557dd114a230 0x557dd11549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 >> 172.21.15.66:0/3990270300 conn(0x557dd113af70 msgr2=0x557dd114dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 shutdown_connections 2023-07-21T22:24:39.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 1 -- 172.21.15.66:0/3990270300 wait complete. 2023-07-21T22:24:39.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.517+0000 7f714a2c5b80 20 client.26264 trim_cache size 0 max 16384 2023-07-21T22:24:39.570 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:24:39.570 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:24:39.570 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "0c932dc8-d0dd-4b57-ba7e-8e2644b77eef", 2023-07-21T22:24:39.570 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:24:39.570 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:24:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71180a12c0 msgr2=0x7f71180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71180a12c0 0x7f71180a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71301993b0 tx=0x7f7124013900 comp rx=0 tx=0).stop 2023-07-21T22:24:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 msgr2=0x7f713019c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f713019c740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f712800ef30 tx=0x7f712800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 shutdown_connections 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f71180a12c0 0x7f71180a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7130105af0 0x7f713019c740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71300ffae0 0x7f7130197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 --2- 172.21.15.66:0/2908418891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7130100490 0x7f7130198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 >> 172.21.15.66:0/2908418891 conn(0x7f71300fb5e0 msgr2=0x7f71300fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 shutdown_connections 2023-07-21T22:24:39.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.518+0000 7f71369ff700 1 -- 172.21.15.66:0/2908418891 wait complete. 2023-07-21T22:24:39.586 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:24:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 -- 172.21.15.66:0/1819009784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 msgr2=0x7f5f4c10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 --2- 172.21.15.66:0/1819009784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 0x7f5f4c10c920 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b0a0 tx=0x7f5f4400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 -- 172.21.15.66:0/1819009784 shutdown_connections 2023-07-21T22:24:39.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 --2- 172.21.15.66:0/1819009784 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 0x7f5f4c10c920 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 --2- 172.21.15.66:0/1819009784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c105af0 0x7f5f4c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 --2- 172.21.15.66:0/1819009784 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 -- 172.21.15.66:0/1819009784 >> 172.21.15.66:0/1819009784 conn(0x7f5f4c0fb5e0 msgr2=0x7f5f4c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 -- 172.21.15.66:0/1819009784 shutdown_connections 2023-07-21T22:24:39.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.765+0000 7f5f511ff700 1 -- 172.21.15.66:0/1819009784 wait complete. 2023-07-21T22:24:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.766+0000 7f5f511ff700 1 Processor -- start 2023-07-21T22:24:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.766+0000 7f5f511ff700 1 -- start start 2023-07-21T22:24:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.766+0000 7f5f511ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.766+0000 7f5f511ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 0x7f5f4c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.766+0000 7f5f511ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c105af0 0x7f5f4c073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f511ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a40f0 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f511ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a4260 con 0x7f5f4c100490 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f511ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1a4420 con 0x7f5f4c105af0 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35710/0 (socket says 172.21.15.66:35710) 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/723193662 learned_addr learned my addr 172.21.15.66:0/723193662 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 0x7f5f4c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c105af0 0x7f5f4c073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/723193662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c105af0 msgr2=0x7f5f4c073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c105af0 0x7f5f4c073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/723193662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 msgr2=0x7f5f4c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 0x7f5f4c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/723193662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44009d00 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.767+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c072810 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e9a0 tx=0x7f5f3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.768+0000 7f5f33fff700 1 -- 172.21.15.66:0/723193662 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c009790 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.768+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c1a4700 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.768+0000 7f5f33fff700 1 -- 172.21.15.66:0/723193662 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c010430 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.768+0000 7f5f33fff700 1 -- 172.21.15.66:0/723193662 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c018680 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.768+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c1a4c60 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.772+0000 7f5f33fff700 1 -- 172.21.15.66:0/723193662 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5f3c0105a0 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 Processor -- start 2023-07-21T22:24:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 -- start start 2023-07-21T22:24:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f33fff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f340a1340 0x7f5f340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804c9c0 0x55c64804cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 0x55c64803f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f33fff700 1 -- 172.21.15.66:0/723193662 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5f3c014070 con 0x7f5f4c0ffae0 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c64803fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647f44cb0 con 0x55c64804c9c0 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647f97a40 con 0x55c64804d2d0 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.773+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647f90ed0 con 0x55c64803f930 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f340a1340 0x7f5f340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 0x55c64803f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 0x55c64803f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50890/0 (socket says 172.21.15.66:50890) 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 -- 172.21.15.66:0/3140835482 learned_addr learned my addr 172.21.15.66:0/3140835482 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c64803fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804c9c0 0x55c64804cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 -- 172.21.15.66:0/3140835482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 msgr2=0x55c64803fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c64803fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 -- 172.21.15.66:0/3140835482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804c9c0 msgr2=0x55c64804cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804c9c0 0x55c64804cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4a59c700 1 -- 172.21.15.66:0/3140835482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c647dbede0 con 0x55c64804d2d0 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.774+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f340a1340 0x7f5f340a37f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b780 tx=0x7f5f440094b0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.775+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 0x55c64803f350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009d20 tx=0x7f5f40011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f45d44da611258 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.775+0000 7f5f317fa700 1 -- 172.21.15.66:0/3140835482 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40019630 con 0x55c64804d2d0 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.775+0000 7f5f317fa700 1 -- 172.21.15.66:0/3140835482 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f40019c50 con 0x55c64804d2d0 2023-07-21T22:24:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.775+0000 7f5f317fa700 1 -- 172.21.15.66:0/3140835482 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000ec90 con 0x55c64804d2d0 2023-07-21T22:24:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.775+0000 7f5f64b40b80 1 -- 172.21.15.66:0/3140835482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 msgr2=0x55c64803f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.775+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 0x55c64803f350 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009d20 tx=0x7f5f40011720 comp rx=0 tx=0).stop 2023-07-21T22:24:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 -- 172.21.15.66:0/3140835482 shutdown_connections 2023-07-21T22:24:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c64803fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804c9c0 0x55c64804cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/3140835482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804d2d0 0x55c64803f350 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 -- 172.21.15.66:0/3140835482 >> 172.21.15.66:0/3140835482 conn(0x55c648034ef0 msgr2=0x55c64803a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 -- 172.21.15.66:0/3140835482 shutdown_connections 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 -- 172.21.15.66:0/3140835482 wait complete. 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.776+0000 7f5f64b40b80 1 -- start start 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 Processor -- start 2023-07-21T22:24:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c6480543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648051620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648051ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647f30ac0 con 0x55c64804d2d0 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647f307c0 con 0x55c64803f930 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c647e14920 con 0x55c64804c9c0 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f64b40b80 10 client.0 _ll_register_callbacks cb 0x55c647f2e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c6480543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c6480543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50904/0 (socket says 172.21.15.66:50904) 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.777+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/2729319912 learned_addr learned my addr 172.21.15.66:0/2729319912 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648051620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648051ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/2729319912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 msgr2=0x55c648051620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648051620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/2729319912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 msgr2=0x55c648051ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648051ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/2729319912 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f40009980 con 0x55c64803f930 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648051ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c6480543a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e970 tx=0x7f5f3c0a2240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715e3cd6fe6157ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f2eff5700 1 -- 172.21.15.66:0/2729319912 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0a1db0 con 0x55c64803f930 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f2eff5700 1 -- 172.21.15.66:0/2729319912 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5f3c0a1f10 con 0x55c64803f930 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c647eea4f0 con 0x55c64803f930 2023-07-21T22:24:39.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.778+0000 7f5f2eff5700 1 -- 172.21.15.66:0/2729319912 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f3c0cabc0 con 0x55c64803f930 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 10 client.26276 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c647da7c60 con 0x55c64803f930 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f2eff5700 1 -- 172.21.15.66:0/2729319912 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f3c0ca080 con 0x55c64803f930 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f2eff5700 1 -- 172.21.15.66:0/2729319912 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5f3c0adb80 con 0x55c64803f930 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 10 client.26276 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 10 client.26276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:39.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 20 client.26276 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:39.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 4 client.26276 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:24:39.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c647f89eb0 con 0x55c648061c90 2023-07-21T22:24:39.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.779+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.780+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480640a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa8b1797994b6d12 server_cookie=76da0f4d70ded70f in_seq=0 out_seq=0 2023-07-21T22:24:39.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.780+0000 7f5f2eff5700 10 client.26276 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f2eff5700 1 -- 172.21.15.66:0/2729319912 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c647f89eb0 con 0x55c648061c90 2023-07-21T22:24:39.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f2eff5700 10 client.26276 handle_command_reply: tid=0 2023-07-21T22:24:39.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 client.26276 shutdown 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 msgr2=0x55c6480543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c6480543a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e970 tx=0x7f5f3c0a2240 comp rx=0 tx=0).stop 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 shutdown_connections 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c6480543a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480640a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648051ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/2729319912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648051620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.781+0000 7f5f2eff5700 0 client.26276 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.782+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 >> 172.21.15.66:0/2729319912 conn(0x55c648034ef0 msgr2=0x55c64803a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.782+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 shutdown_connections 2023-07-21T22:24:39.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.782+0000 7f5f64b40b80 1 -- 172.21.15.66:0/2729319912 wait complete. 2023-07-21T22:24:39.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.782+0000 7f5f64b40b80 20 client.26276 trim_cache size 0 max 16384 2023-07-21T22:24:39.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 Processor -- start 2023-07-21T22:24:39.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 -- start start 2023-07-21T22:24:39.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c648058d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648053f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c64803d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c648037010 con 0x55c64804d2d0 2023-07-21T22:24:39.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c648032050 con 0x55c64803f930 2023-07-21T22:24:39.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6480525b0 con 0x55c64804c9c0 2023-07-21T22:24:39.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c648058d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f4b59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c64803d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c648058d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50912/0 (socket says 172.21.15.66:50912) 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/1936182479 learned_addr learned my addr 172.21.15.66:0/1936182479 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f4b59e700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c64803d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35754/0 (socket says 172.21.15.66:35754) 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.803+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648053f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/1936182479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 msgr2=0x55c648053f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648053f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/1936182479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 msgr2=0x55c64803d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c64803d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/1936182479 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c647de5250 con 0x55c64803f930 2023-07-21T22:24:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c64803d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:24:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c648058d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0a1d10 tx=0x7f5f3c099a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615c7069723fe576 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f31ffb700 1 -- 172.21.15.66:0/1936182479 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c095df0 con 0x55c64803f930 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f31ffb700 1 -- 172.21.15.66:0/1936182479 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c647eea4f0 con 0x55c64803f930 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.804+0000 7f5f31ffb700 1 -- 172.21.15.66:0/1936182479 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0cae20 con 0x55c64803f930 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 -- 172.21.15.66:0/1936182479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 msgr2=0x55c648058d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c648058d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0a1d10 tx=0x7f5f3c099a60 comp rx=0 tx=0).stop 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 -- 172.21.15.66:0/1936182479 shutdown_connections 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64803f930 0x55c648058d80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c64803d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/1936182479 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64804c9c0 0x55c648053f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 -- 172.21.15.66:0/1936182479 >> 172.21.15.66:0/1936182479 conn(0x55c648034ef0 msgr2=0x55c64805df80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 -- 172.21.15.66:0/1936182479 shutdown_connections 2023-07-21T22:24:39.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.805+0000 7f5f64b40b80 1 -- 172.21.15.66:0/1936182479 wait complete. 2023-07-21T22:24:39.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.806+0000 7f5f64b40b80 1 -- start start 2023-07-21T22:24:39.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.806+0000 7f5f64b40b80 1 Processor -- start 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.806+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c648051400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.806+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804c9c0 0x55c64804fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.806+0000 7f5f64b40b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648050330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.806+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c648051a20 con 0x55c64804d2d0 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c648050870 con 0x55c64804c9c0 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f64b40b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c648050a30 con 0x55c64803f930 2023-07-21T22:24:39.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f64b40b80 10 client.0 _ll_register_callbacks cb 0x7f5f3c099df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804c9c0 0x55c64804fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c648051400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c648051400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47854/0 (socket says 172.21.15.66:47854) 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/204325654 learned_addr learned my addr 172.21.15.66:0/204325654 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804c9c0 0x55c64804fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50924/0 (socket says 172.21.15.66:50924) 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648050330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804c9c0 msgr2=0x55c64804fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804c9c0 0x55c64804fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/204325654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 msgr2=0x55c648050330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648050330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 -- 172.21.15.66:0/204325654 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c10d220 con 0x55c64803f930 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4b59e700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648050330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f4ad9d700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c648051400 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0962f0 tx=0x7f5f3c0b8330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c9ce5248e9dd30 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.807+0000 7f5f2eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:24:39.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f2eff5700 1 -- 172.21.15.66:0/204325654 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c099bc0 con 0x55c64803f930 2023-07-21T22:24:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f40019c50 con 0x55c64803f930 2023-07-21T22:24:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f2eff5700 1 -- 172.21.15.66:0/204325654 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5f3c0cabc0 con 0x55c64803f930 2023-07-21T22:24:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f2eff5700 1 -- 172.21.15.66:0/204325654 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f3c0cad80 con 0x55c64803f930 2023-07-21T22:24:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f64b40b80 10 client.26260 fetch_fsmap learned FSMap version 28 2023-07-21T22:24:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55c647da7c60 con 0x55c64803f930 2023-07-21T22:24:39.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f2eff5700 1 -- 172.21.15.66:0/204325654 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f40019c50 con 0x55c64803f930 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f2eff5700 1 -- 172.21.15.66:0/204325654 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5f3c0adaf0 con 0x55c64803f930 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f64b40b80 10 client.26260 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f64b40b80 10 client.26260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.808+0000 7f5f64b40b80 20 client.26260 populate_metadata read hostname 'smithi066' 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.809+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.809+0000 7f5f64b40b80 4 client.26260 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.809+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c647f4c050 con 0x55c648061c90 2023-07-21T22:24:39.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.809+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f2eff5700 10 client.26260 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f4a59c700 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480b8950 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96f0d21add5a15a server_cookie=cd1ed28690a71145 in_seq=0 out_seq=0 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f2eff5700 1 -- 172.21.15.66:0/204325654 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c647f4c050 con 0x55c648061c90 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f2eff5700 10 client.26260 handle_command_reply: tid=0 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 client.26260 shutdown 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 msgr2=0x55c648051400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c648051400 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0962f0 tx=0x7f5f3c0b8330 comp rx=0 tx=0).stop 2023-07-21T22:24:39.862 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 shutdown_connections 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c648061c90 0x55c6480b8950 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c64804d2d0 0x55c648050330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c64803f930 0x55c648051400 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 --2- 172.21.15.66:0/204325654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c64804c9c0 0x55c64804fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 >> 172.21.15.66:0/204325654 conn(0x55c648034ef0 msgr2=0x55c64803d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f2eff5700 0 client.26260 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:24:39.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 shutdown_connections 2023-07-21T22:24:39.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 1 -- 172.21.15.66:0/204325654 wait complete. 2023-07-21T22:24:39.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.810+0000 7f5f64b40b80 20 client.26260 trim_cache size 0 max 16384 2023-07-21T22:24:39.864 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:24:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:24:39.864 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "0c932dc8-d0dd-4b57-ba7e-8e2644b77eef": { 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "0c932dc8-d0dd-4b57-ba7e-8e2644b77eef", 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:24:39.865 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:24:39.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f340a1340 msgr2=0x7f5f340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f340a1340 0x7f5f340a37f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400b780 tx=0x7f5f440094b0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 msgr2=0x7f5f4c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:24:39.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c072810 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e9a0 tx=0x7f5f3c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 shutdown_connections 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f340a1340 0x7f5f340a37f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c105af0 0x7f5f4c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f4c0ffae0 0x7f5f4c072810 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 --2- 172.21.15.66:0/723193662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f4c100490 0x7f5f4c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 >> 172.21.15.66:0/723193662 conn(0x7f5f4c0fb5e0 msgr2=0x7f5f4c1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 shutdown_connections 2023-07-21T22:24:39.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:24:39.811+0000 7f5f511ff700 1 -- 172.21.15.66:0/723193662 wait complete. 2023-07-21T22:24:39.876 INFO:tasks.cephfs.filesystem:scrub status for tag:0c932dc8-d0dd-4b57-ba7e-8e2644b77eef - {'path': '/', 'tag': '0c932dc8-d0dd-4b57-ba7e-8e2644b77eef', 'options': 'recursive,force'} 2023-07-21T22:24:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:39 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:24:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:39 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:39 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:39 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:39 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:39 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:24:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:39 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:39 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:39 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:40.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:39 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:39 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:24:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:39 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:39 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:39 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:39 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:24:40.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 52583 4.37 MB/sec execute 7 sec latency 91.359 ms 2023-07-21T22:24:41.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:41.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: pgmap v99: 129 pgs: 129 active+clean; 50 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 395 KiB/s wr, 54 op/s 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:41.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:24:41.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:41.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:41.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:24:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: pgmap v99: 129 pgs: 129 active+clean; 50 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 395 KiB/s wr, 54 op/s 2023-07-21T22:24:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:24:41.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:24:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:24:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: pgmap v99: 129 pgs: 129 active+clean; 50 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 395 KiB/s wr, 54 op/s 2023-07-21T22:24:41.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:24:41.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:24:41.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:24:41.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53856 4.73 MB/sec execute 8 sec latency 14.714 ms 2023-07-21T22:24:42.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53903 4.22 MB/sec execute 9 sec latency 982.179 ms 2023-07-21T22:24:43.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53959 3.81 MB/sec execute 10 sec latency 990.669 ms 2023-07-21T22:24:44.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:43 smithi066 ceph-mon[126379]: pgmap v100: 129 pgs: 129 active+clean; 50 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 283 KiB/s wr, 45 op/s 2023-07-21T22:24:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:43 smithi149 ceph-mon[129268]: pgmap v100: 129 pgs: 129 active+clean; 50 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 283 KiB/s wr, 45 op/s 2023-07-21T22:24:44.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:43 smithi104 ceph-mon[129021]: pgmap v100: 129 pgs: 129 active+clean; 50 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 283 KiB/s wr, 45 op/s 2023-07-21T22:24:44.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53959 3.46 MB/sec execute 11 sec latency 1940.774 ms 2023-07-21T22:24:45.520 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53959 3.17 MB/sec execute 12 sec latency 2940.918 ms 2023-07-21T22:24:46.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:45 smithi066 ceph-mon[126379]: pgmap v101: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 454 KiB/s wr, 78 op/s 2023-07-21T22:24:46.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:45 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:24:46.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:45 smithi149 ceph-mon[129268]: pgmap v101: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 454 KiB/s wr, 78 op/s 2023-07-21T22:24:46.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:45 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:24:46.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:45 smithi104 ceph-mon[129021]: pgmap v101: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 454 KiB/s wr, 78 op/s 2023-07-21T22:24:46.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:45 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:24:46.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53959 2.93 MB/sec execute 13 sec latency 3941.066 ms 2023-07-21T22:24:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:46 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:24:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:46 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:24:47.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:46 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:24:47.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:46 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:24:47.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:46 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:24:47.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:46 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:24:47.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 53959 2.72 MB/sec execute 14 sec latency 4941.187 ms 2023-07-21T22:24:48.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:47 smithi066 ceph-mon[126379]: pgmap v102: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 425 KiB/s wr, 71 op/s 2023-07-21T22:24:48.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:47 smithi149 ceph-mon[129268]: pgmap v102: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 425 KiB/s wr, 71 op/s 2023-07-21T22:24:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:47 smithi104 ceph-mon[129021]: pgmap v102: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 425 KiB/s wr, 71 op/s 2023-07-21T22:24:48.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 54027 2.55 MB/sec execute 15 sec latency 4950.393 ms 2023-07-21T22:24:49.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 54027 2.39 MB/sec execute 16 sec latency 1929.198 ms 2023-07-21T22:24:50.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:49 smithi066 ceph-mon[126379]: pgmap v103: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 425 KiB/s wr, 71 op/s 2023-07-21T22:24:50.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:49 smithi149 ceph-mon[129268]: pgmap v103: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 425 KiB/s wr, 71 op/s 2023-07-21T22:24:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:49 smithi104 ceph-mon[129021]: pgmap v103: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 425 KiB/s wr, 71 op/s 2023-07-21T22:24:50.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 54027 2.25 MB/sec execute 17 sec latency 2929.285 ms 2023-07-21T22:24:51.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 54027 2.12 MB/sec execute 18 sec latency 3929.361 ms 2023-07-21T22:24:52.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:51 smithi066 ceph-mon[126379]: pgmap v104: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 476 KiB/s wr, 79 op/s 2023-07-21T22:24:52.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:51 smithi149 ceph-mon[129268]: pgmap v104: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 476 KiB/s wr, 79 op/s 2023-07-21T22:24:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:51 smithi104 ceph-mon[129021]: pgmap v104: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 476 KiB/s wr, 79 op/s 2023-07-21T22:24:52.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 54027 2.01 MB/sec execute 19 sec latency 4929.442 ms 2023-07-21T22:24:53.521 INFO:tasks.workunit.client.0.smithi066.stdout: 1 56137 2.48 MB/sec execute 20 sec latency 4933.307 ms 2023-07-21T22:24:54.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:53 smithi066 ceph-mon[126379]: pgmap v105: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 222 KiB/s wr, 40 op/s 2023-07-21T22:24:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:53 smithi149 ceph-mon[129268]: pgmap v105: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 222 KiB/s wr, 40 op/s 2023-07-21T22:24:54.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:53 smithi104 ceph-mon[129021]: pgmap v105: 129 pgs: 129 active+clean; 52 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 222 KiB/s wr, 40 op/s 2023-07-21T22:24:54.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57380 2.71 MB/sec execute 21 sec latency 92.669 ms 2023-07-21T22:24:55.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57447 2.59 MB/sec execute 22 sec latency 970.199 ms 2023-07-21T22:24:56.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:55 smithi066 ceph-mon[126379]: pgmap v106: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 112 op/s 2023-07-21T22:24:56.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:55 smithi149 ceph-mon[129268]: pgmap v106: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 112 op/s 2023-07-21T22:24:56.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:55 smithi104 ceph-mon[129021]: pgmap v106: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 112 op/s 2023-07-21T22:24:56.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57447 2.48 MB/sec execute 23 sec latency 1970.365 ms 2023-07-21T22:24:57.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57447 2.38 MB/sec execute 24 sec latency 2970.436 ms 2023-07-21T22:24:58.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:57 smithi066 ceph-mon[126379]: pgmap v107: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2023-07-21T22:24:58.244 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:57 smithi149 ceph-mon[129268]: pgmap v107: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2023-07-21T22:24:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:57 smithi104 ceph-mon[129021]: pgmap v107: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2023-07-21T22:24:58.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57503 2.29 MB/sec execute 25 sec latency 2973.865 ms 2023-07-21T22:24:59.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:58 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:24:59.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:58 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:24:59.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:58 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:24:59.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57503 2.20 MB/sec execute 26 sec latency 1932.981 ms 2023-07-21T22:25:00.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:24:59 smithi066 ceph-mon[126379]: pgmap v108: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2023-07-21T22:25:00.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:24:59 smithi149 ceph-mon[129268]: pgmap v108: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2023-07-21T22:25:00.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:24:59 smithi104 ceph-mon[129021]: pgmap v108: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2023-07-21T22:25:00.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57503 2.12 MB/sec execute 27 sec latency 2933.104 ms 2023-07-21T22:25:01.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57571 2.05 MB/sec execute 28 sec latency 3088.119 ms 2023-07-21T22:25:02.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:01 smithi066 ceph-mon[126379]: pgmap v109: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.3 MiB/s wr, 87 op/s 2023-07-21T22:25:02.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:01 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:01 smithi149 ceph-mon[129268]: pgmap v109: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.3 MiB/s wr, 87 op/s 2023-07-21T22:25:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:01 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:01 smithi104 ceph-mon[129021]: pgmap v109: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 1.3 MiB/s wr, 87 op/s 2023-07-21T22:25:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:01 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:02.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 57571 1.98 MB/sec execute 29 sec latency 1773.627 ms 2023-07-21T22:25:03.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 58359 2.07 MB/sec execute 30 sec latency 1776.854 ms 2023-07-21T22:25:04.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:03 smithi066 ceph-mon[126379]: pgmap v110: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2023-07-21T22:25:04.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:03 smithi149 ceph-mon[129268]: pgmap v110: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2023-07-21T22:25:04.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:03 smithi104 ceph-mon[129021]: pgmap v110: 129 pgs: 129 active+clean; 52 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 1.3 MiB/s wr, 79 op/s 2023-07-21T22:25:04.522 INFO:tasks.workunit.client.0.smithi066.stdout: 1 60695 2.45 MB/sec execute 31 sec latency 9.605 ms 2023-07-21T22:25:05.523 INFO:tasks.workunit.client.0.smithi066.stdout: 1 60995 2.38 MB/sec execute 32 sec latency 637.963 ms 2023-07-21T22:25:06.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:05 smithi066 ceph-mon[126379]: pgmap v111: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 145 op/s 2023-07-21T22:25:06.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:05 smithi149 ceph-mon[129268]: pgmap v111: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 145 op/s 2023-07-21T22:25:06.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:05 smithi104 ceph-mon[129021]: pgmap v111: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 145 op/s 2023-07-21T22:25:06.523 INFO:tasks.workunit.client.0.smithi066.stdout: 1 60995 2.31 MB/sec execute 33 sec latency 1638.094 ms 2023-07-21T22:25:07.523 INFO:tasks.workunit.client.0.smithi066.stdout: 1 60995 2.24 MB/sec execute 34 sec latency 2638.226 ms 2023-07-21T22:25:08.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:07 smithi066 ceph-mon[126379]: pgmap v112: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 347 KiB/s wr, 72 op/s 2023-07-21T22:25:08.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:07 smithi149 ceph-mon[129268]: pgmap v112: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 347 KiB/s wr, 72 op/s 2023-07-21T22:25:08.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:07 smithi104 ceph-mon[129021]: pgmap v112: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 347 KiB/s wr, 72 op/s 2023-07-21T22:25:08.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 61119 2.18 MB/sec execute 35 sec latency 2643.663 ms 2023-07-21T22:25:09.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 61119 2.12 MB/sec execute 36 sec latency 1874.211 ms 2023-07-21T22:25:09.877 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:25:10.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 -- 172.21.15.66:0/1881131958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 msgr2=0x7fcc4c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 --2- 172.21.15.66:0/1881131958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c0fff40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c009990 tx=0x7fcc3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 -- 172.21.15.66:0/1881131958 shutdown_connections 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 --2- 172.21.15.66:0/1881131958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc4c105ae0 0x7fcc4c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 --2- 172.21.15.66:0/1881131958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc4c100480 0x7fcc4c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 --2- 172.21.15.66:0/1881131958 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c0fff40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 -- 172.21.15.66:0/1881131958 >> 172.21.15.66:0/1881131958 conn(0x7fcc4c0fb5f0 msgr2=0x7fcc4c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 -- 172.21.15.66:0/1881131958 shutdown_connections 2023-07-21T22:25:10.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.054+0000 7fcc533ff700 1 -- 172.21.15.66:0/1881131958 wait complete. 2023-07-21T22:25:10.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.055+0000 7fcc533ff700 1 Processor -- start 2023-07-21T22:25:10.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.055+0000 7fcc533ff700 1 -- start start 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc533ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc533ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc4c100480 0x7fcc4c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc533ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc4c105ae0 0x7fcc4c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc533ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c19cde0 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc533ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c19cf50 con 0x7fcc4c100480 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc533ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c19d110 con 0x7fcc4c105ae0 2023-07-21T22:25:10.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49474/0 (socket says 172.21.15.66:49474) 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 -- 172.21.15.66:0/3612779996 learned_addr learned my addr 172.21.15.66:0/3612779996 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5199c700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc4c105ae0 0x7fcc4c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc4c100480 0x7fcc4c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 -- 172.21.15.66:0/3612779996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc4c105ae0 msgr2=0x7fcc4c19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc4c105ae0 0x7fcc4c19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 -- 172.21.15.66:0/3612779996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc4c100480 msgr2=0x7fcc4c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc4c100480 0x7fcc4c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.056+0000 7fcc5119b700 1 -- 172.21.15.66:0/3612779996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c0095f0 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.057+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c197e00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e9a0 tx=0x7fcc4800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.057+0000 7fcc427fc700 1 -- 172.21.15.66:0/3612779996 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48009790 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.057+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c19d3f0 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.057+0000 7fcc427fc700 1 -- 172.21.15.66:0/3612779996 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc48010430 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.057+0000 7fcc427fc700 1 -- 172.21.15.66:0/3612779996 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48018680 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.057+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c1a4c30 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.060+0000 7fcc427fc700 1 -- 172.21.15.66:0/3612779996 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcc480105a0 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.061+0000 7fcc66d42b80 1 Processor -- start 2023-07-21T22:25:10.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 -- start start 2023-07-21T22:25:10.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc427fc700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcc380a12c0 0x7fcc380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b981b0 0x564632b98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b98ac0 0x564632b93350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b93930 0x564632b93da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632a98cb0 con 0x564632b98ac0 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632aeba40 con 0x564632b981b0 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632ae4ed0 con 0x564632b93930 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc427fc700 1 -- 172.21.15.66:0/3612779996 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcc48014070 con 0x7fcc4c0ffad0 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc5199c700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcc380a12c0 0x7fcc380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc5099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b98ac0 0x564632b93350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.062+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b981b0 0x564632b98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b98ac0 0x564632b93350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49478/0 (socket says 172.21.15.66:49478) 2023-07-21T22:25:10.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5099a700 1 -- 172.21.15.66:0/574431244 learned_addr learned my addr 172.21.15.66:0/574431244 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5199c700 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b93930 0x564632b93da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5119b700 1 -- 172.21.15.66:0/574431244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b93930 msgr2=0x564632b93da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5119b700 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b93930 0x564632b93da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5119b700 1 -- 172.21.15.66:0/574431244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b98ac0 msgr2=0x564632b93350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5119b700 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b98ac0 0x564632b93350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5119b700 1 -- 172.21.15.66:0/574431244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564632912de0 con 0x564632b981b0 2023-07-21T22:25:10.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5119b700 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b981b0 0x564632b98580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc480ca5a0 tx=0x7fcc48099a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2a0fa07628f891 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc5199c700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcc380a12c0 0x7fcc380a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009ec0 tx=0x7fcc44009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.063+0000 7fcc377fe700 1 -- 172.21.15.66:0/574431244 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48084ea0 con 0x564632b981b0 2023-07-21T22:25:10.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc377fe700 1 -- 172.21.15.66:0/574431244 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc48099f20 con 0x564632b981b0 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc377fe700 1 -- 172.21.15.66:0/574431244 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc480109a0 con 0x564632b981b0 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 -- 172.21.15.66:0/574431244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b981b0 msgr2=0x564632b98580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b981b0 0x564632b98580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc480ca5a0 tx=0x7fcc48099a60 comp rx=0 tx=0).stop 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 -- 172.21.15.66:0/574431244 shutdown_connections 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b981b0 0x564632b98580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b93930 0x564632b93da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/574431244 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b98ac0 0x564632b93350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.064+0000 7fcc66d42b80 1 -- 172.21.15.66:0/574431244 >> 172.21.15.66:0/574431244 conn(0x564632b88ef0 msgr2=0x564632b8e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.065+0000 7fcc66d42b80 1 -- 172.21.15.66:0/574431244 shutdown_connections 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.065+0000 7fcc66d42b80 1 -- 172.21.15.66:0/574431244 wait complete. 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.065+0000 7fcc66d42b80 1 -- start start 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.065+0000 7fcc66d42b80 1 Processor -- start 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632a84ac0 con 0x564632b981b0 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632a847c0 con 0x564632b93930 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632968920 con 0x564632b98ac0 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc66d42b80 10 client.0 _ll_register_callbacks cb 0x564632a82760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc5199c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc5099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc5199c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52290/0 (socket says 172.21.15.66:52290) 2023-07-21T22:25:10.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc5199c700 1 -- 172.21.15.66:0/3105064713 learned_addr learned my addr 172.21.15.66:0/3105064713 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.066+0000 7fcc5099a700 1 -- 172.21.15.66:0/3105064713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 msgr2=0x564632b9d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc5099a700 1 -- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 msgr2=0x564632b9fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc5099a700 1 -- 172.21.15.66:0/3105064713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c1a4c30 con 0x564632b981b0 2023-07-21T22:25:10.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9cea0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x564632b945b0 tx=0x7fcc3c009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8204f9fea15fcffa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3105064713 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c01d030 con 0x564632b981b0 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564632a3e4f0 con 0x564632b981b0 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3105064713 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcc3c0056b0 con 0x564632b981b0 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.067+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3105064713 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc3c004700 con 0x564632b981b0 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 10 client.16242 fetch_fsmap learned FSMap version 30 2023-07-21T22:25:10.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5646328fbc60 con 0x564632b981b0 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3105064713 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564632a3e4f0 con 0x564632b981b0 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3105064713 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcc3c00f800 con 0x564632b981b0 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 20 client.16242 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632bad400 0x564632baf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 4 client.16242 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:10.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564632addeb0 con 0x564632bad400 2023-07-21T22:25:10.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.068+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632bad400 0x564632baf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.069+0000 7fcc34ff9700 10 client.16242 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:10.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.069+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632bad400 0x564632baf810 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3638c41a38c2fa14 server_cookie=c14c27315814bded in_seq=0 out_seq=0 2023-07-21T22:25:10.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3105064713 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564632addeb0 con 0x564632bad400 2023-07-21T22:25:10.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc34ff9700 10 client.16242 handle_command_reply: tid=0 2023-07-21T22:25:10.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 client.16242 shutdown 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 msgr2=0x564632b9cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9cea0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x564632b945b0 tx=0x7fcc3c009530 comp rx=0 tx=0).stop 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 shutdown_connections 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632bad400 0x564632baf810 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3105064713 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9cea0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc34ff9700 0 client.16242 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:10.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 >> 172.21.15.66:0/3105064713 conn(0x564632b88ef0 msgr2=0x564632b8e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 shutdown_connections 2023-07-21T22:25:10.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3105064713 wait complete. 2023-07-21T22:25:10.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.070+0000 7fcc66d42b80 20 client.16242 trim_cache size 0 max 16384 2023-07-21T22:25:10.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 Processor -- start 2023-07-21T22:25:10.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 -- start start 2023-07-21T22:25:10.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632b94220 con 0x564632b981b0 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632baab60 con 0x564632b93930 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632b946d0 con 0x564632b98ac0 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33864/0 (socket says 172.21.15.66:33864) 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc5099a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.088+0000 7fcc5119b700 1 -- 172.21.15.66:0/3368786695 learned_addr learned my addr 172.21.15.66:0/3368786695 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5199c700 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5119b700 1 -- 172.21.15.66:0/3368786695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 msgr2=0x564632b9cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5119b700 1 -- 172.21.15.66:0/3368786695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 msgr2=0x564632b9c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5119b700 1 -- 172.21.15.66:0/3368786695 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564632939250 con 0x564632b93930 2023-07-21T22:25:10.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:10.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9f7c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcc480860b0 tx=0x7fcc48099cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59dcf2b60a12ad62 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc37fff700 1 -- 172.21.15.66:0/3368786695 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48084ea0 con 0x564632b93930 2023-07-21T22:25:10.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc37fff700 1 -- 172.21.15.66:0/3368786695 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc480982c0 con 0x564632b93930 2023-07-21T22:25:10.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.089+0000 7fcc37fff700 1 -- 172.21.15.66:0/3368786695 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4809a180 con 0x564632b93930 2023-07-21T22:25:10.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3368786695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 msgr2=0x564632b9f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9f7c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcc480860b0 tx=0x7fcc48099cf0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3368786695 shutdown_connections 2023-07-21T22:25:10.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b93930 0x564632b9f7c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b98ac0 0x564632b9cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3368786695 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b981b0 0x564632b9c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3368786695 >> 172.21.15.66:0/3368786695 conn(0x564632b88ef0 msgr2=0x564632b8cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3368786695 shutdown_connections 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3368786695 wait complete. 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.090+0000 7fcc66d42b80 1 -- start start 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.091+0000 7fcc66d42b80 1 Processor -- start 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.091+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 0x564632bfa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.091+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b981b0 0x564632bfaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc66d42b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b98ac0 0x564632ba2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632ba2aa0 con 0x564632b93930 2023-07-21T22:25:10.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632ba2be0 con 0x564632b98ac0 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc66d42b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564632ba2da0 con 0x564632b981b0 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc66d42b80 10 client.0 _ll_register_callbacks cb 0x7fcc3c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 0x564632bfa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b981b0 0x564632bfaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 0x564632bfa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49500/0 (socket says 172.21.15.66:49500) 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5119b700 1 -- 172.21.15.66:0/3801384473 learned_addr learned my addr 172.21.15.66:0/3801384473 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5199c700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b98ac0 0x564632ba2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 -- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b98ac0 msgr2=0x564632ba2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b98ac0 0x564632ba2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 -- 172.21.15.66:0/3801384473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 msgr2=0x564632bfa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 0x564632bfa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 -- 172.21.15.66:0/3801384473 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564632912de0 con 0x564632b981b0 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 0x564632bfa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:10.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.092+0000 7fcc5099a700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b981b0 0x564632bfaaa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c010 tx=0x7fcc3c0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5c81efcb3b7d7e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:10.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3801384473 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c00fea0 con 0x564632b981b0 2023-07-21T22:25:10.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc48099f20 con 0x564632b981b0 2023-07-21T22:25:10.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3801384473 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcc3c000e40 con 0x564632b981b0 2023-07-21T22:25:10.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3801384473 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc3c01d040 con 0x564632b981b0 2023-07-21T22:25:10.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc66d42b80 10 client.26278 fetch_fsmap learned FSMap version 30 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5646329112b0 con 0x564632b981b0 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.093+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3801384473 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc48099f20 con 0x564632b981b0 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3801384473 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcc3c02dc80 con 0x564632b981b0 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc66d42b80 10 client.26278 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc66d42b80 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc66d42b80 20 client.26278 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:10.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632c01ee0 0x564632c042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc66d42b80 4 client.26278 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:25:10.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564632aa0050 con 0x564632c01ee0 2023-07-21T22:25:10.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.094+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632c01ee0 0x564632c042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc34ff9700 10 client.26278 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:10.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc5119b700 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632c01ee0 0x564632c042f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4b74c8e89467eb server_cookie=59050c84d0df0f5b in_seq=0 out_seq=0 2023-07-21T22:25:10.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc34ff9700 1 -- 172.21.15.66:0/3801384473 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564632aa0050 con 0x564632c01ee0 2023-07-21T22:25:10.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc34ff9700 10 client.26278 handle_command_reply: tid=0 2023-07-21T22:25:10.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 client.26278 shutdown 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b981b0 msgr2=0x564632bfaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b981b0 0x564632bfaaa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c010 tx=0x7fcc3c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 shutdown_connections 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564632b98ac0 0x564632ba2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564632c01ee0 0x564632c042f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564632b93930 0x564632bfa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 --2- 172.21.15.66:0/3801384473 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564632b981b0 0x564632bfaaa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc34ff9700 0 client.26278 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:10.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 >> 172.21.15.66:0/3801384473 conn(0x564632b88ef0 msgr2=0x564632b929b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 shutdown_connections 2023-07-21T22:25:10.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 1 -- 172.21.15.66:0/3801384473 wait complete. 2023-07-21T22:25:10.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.095+0000 7fcc66d42b80 20 client.26278 trim_cache size 0 max 16384 2023-07-21T22:25:10.149 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:25:10.150 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:25:10.150 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:25:10.150 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:25:10.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcc380a12c0 msgr2=0x7fcc380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcc380a12c0 0x7fcc380a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009ec0 tx=0x7fcc44009920 comp rx=0 tx=0).stop 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 msgr2=0x7fcc4c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c197e00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800e9a0 tx=0x7fcc4800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 shutdown_connections 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcc380a12c0 0x7fcc380a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc4c105ae0 0x7fcc4c19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcc4c0ffad0 0x7fcc4c197e00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 --2- 172.21.15.66:0/3612779996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc4c100480 0x7fcc4c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 >> 172.21.15.66:0/3612779996 conn(0x7fcc4c0fb5f0 msgr2=0x7fcc4c10ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 shutdown_connections 2023-07-21T22:25:10.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.096+0000 7fcc533ff700 1 -- 172.21.15.66:0/3612779996 wait complete. 2023-07-21T22:25:10.159 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:25:10.159 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:25:10.222 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:09 smithi066 ceph-mon[126379]: pgmap v113: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 347 KiB/s wr, 72 op/s 2023-07-21T22:25:10.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:09 smithi149 ceph-mon[129268]: pgmap v113: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 347 KiB/s wr, 72 op/s 2023-07-21T22:25:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 -- 172.21.15.66:0/1641296108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70105ae0 msgr2=0x7fad70105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 --2- 172.21.15.66:0/1641296108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70105ae0 0x7fad70105eb0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009990 tx=0x7fad6c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 -- 172.21.15.66:0/1641296108 shutdown_connections 2023-07-21T22:25:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 --2- 172.21.15.66:0/1641296108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad70100480 0x7fad7010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 --2- 172.21.15.66:0/1641296108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad70105ae0 0x7fad70105eb0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 --2- 172.21.15.66:0/1641296108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad700ffad0 0x7fad700fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 -- 172.21.15.66:0/1641296108 >> 172.21.15.66:0/1641296108 conn(0x7fad700fb590 msgr2=0x7fad700fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 -- 172.21.15.66:0/1641296108 shutdown_connections 2023-07-21T22:25:10.370 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.323+0000 7fad785ff700 1 -- 172.21.15.66:0/1641296108 wait complete. 2023-07-21T22:25:10.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:09 smithi104 ceph-mon[129021]: pgmap v113: 129 pgs: 129 active+clean; 55 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 347 KiB/s wr, 72 op/s 2023-07-21T22:25:10.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.324+0000 7fad785ff700 1 Processor -- start 2023-07-21T22:25:10.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 -- start start 2023-07-21T22:25:10.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 0x7fad70072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad70100480 0x7fad70072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad70105ae0 0x7fad701a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad70073580 con 0x7fad700ffad0 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad701a42e0 con 0x7fad70105ae0 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad785ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad701a4420 con 0x7fad70100480 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad7639b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 0x7fad70072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad7639b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 0x7fad70072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49516/0 (socket says 172.21.15.66:49516) 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad7639b700 1 -- 172.21.15.66:0/2081259810 learned_addr learned my addr 172.21.15.66:0/2081259810 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad76b9c700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad70105ae0 0x7fad701a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.325+0000 7fad75b9a700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad70100480 0x7fad70072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad7639b700 1 -- 172.21.15.66:0/2081259810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad70100480 msgr2=0x7fad70072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad7639b700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad70100480 0x7fad70072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad7639b700 1 -- 172.21.15.66:0/2081259810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad70105ae0 msgr2=0x7fad701a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad7639b700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad70105ae0 0x7fad701a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad7639b700 1 -- 172.21.15.66:0/2081259810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0095f0 con 0x7fad700ffad0 2023-07-21T22:25:10.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad7639b700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 0x7fad70072810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005910 tx=0x7fad6c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c01d070 con 0x7fad700ffad0 2023-07-21T22:25:10.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad6c022470 con 0x7fad700ffad0 2023-07-21T22:25:10.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad701a4620 con 0x7fad700ffad0 2023-07-21T22:25:10.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.326+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c017650 con 0x7fad700ffad0 2023-07-21T22:25:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.327+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad701a4a40 con 0x7fad700ffad0 2023-07-21T22:25:10.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.328+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad70067300 con 0x7fad700ffad0 2023-07-21T22:25:10.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.329+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fad6c0225e0 con 0x7fad700ffad0 2023-07-21T22:25:10.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.331+0000 7fad677fe700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fad5c0a1300 0x7fad5c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.331+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fad6c0ce3d0 con 0x7fad700ffad0 2023-07-21T22:25:10.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.331+0000 7fad75b9a700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fad5c0a1300 0x7fad5c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.332+0000 7fad75b9a700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fad5c0a1300 0x7fad5c0a37b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad70074000 tx=0x7fad60009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.335+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fad6c0974c0 con 0x7fad700ffad0 2023-07-21T22:25:10.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 61119 2.07 MB/sec execute 37 sec latency 2874.333 ms 2023-07-21T22:25:10.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.590+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad701a4cf0 con 0x7fad700ffad0 2023-07-21T22:25:10.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.591+0000 7fad677fe700 1 -- 172.21.15.66:0/2081259810 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fad6c09aae0 con 0x7fad700ffad0 2023-07-21T22:25:10.636 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:25:10.636 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:25:00.878687+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fad5c0a1300 msgr2=0x7fad5c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fad5c0a1300 0x7fad5c0a37b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad70074000 tx=0x7fad60009260 comp rx=0 tx=0).stop 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 msgr2=0x7fad70072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 0x7fad70072810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fad6c005910 tx=0x7fad6c005720 comp rx=0 tx=0).stop 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 shutdown_connections 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad70105ae0 0x7fad701a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fad5c0a1300 0x7fad5c0a37b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fad700ffad0 0x7fad70072810 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 --2- 172.21.15.66:0/2081259810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad70100480 0x7fad70072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.593+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 >> 172.21.15.66:0/2081259810 conn(0x7fad700fb590 msgr2=0x7fad700fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.594+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 shutdown_connections 2023-07-21T22:25:10.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.594+0000 7fad785ff700 1 -- 172.21.15.66:0/2081259810 wait complete. 2023-07-21T22:25:10.641 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 30 2023-07-21T22:25:10.658 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:25:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 -- 172.21.15.66:0/4270472066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 msgr2=0x7fdc601105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 --2- 172.21.15.66:0/4270472066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 0x7fdc601105b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009990 tx=0x7fdc50009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 -- 172.21.15.66:0/4270472066 shutdown_connections 2023-07-21T22:25:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 --2- 172.21.15.66:0/4270472066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc60110af0 0x7fdc60112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 --2- 172.21.15.66:0/4270472066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc600728b0 0x7fdc60072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 --2- 172.21.15.66:0/4270472066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 0x7fdc601105b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 -- 172.21.15.66:0/4270472066 >> 172.21.15.66:0/4270472066 conn(0x7fdc600fb5e0 msgr2=0x7fdc600fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 -- 172.21.15.66:0/4270472066 shutdown_connections 2023-07-21T22:25:10.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.825+0000 7fdc679ff700 1 -- 172.21.15.66:0/4270472066 wait complete. 2023-07-21T22:25:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 Processor -- start 2023-07-21T22:25:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 -- start start 2023-07-21T22:25:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc600728b0 0x7fdc6019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 0x7fdc6019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 0x7fdc601a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601a11b0 con 0x7fdc60110af0 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601a1320 con 0x7fdc600728b0 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.826+0000 7fdc679ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601a14e0 con 0x7fdc600731c0 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 0x7fdc601a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc6579b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc600728b0 0x7fdc6019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 0x7fdc601a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49532/0 (socket says 172.21.15.66:49532) 2023-07-21T22:25:10.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2861302730 learned_addr learned my addr 172.21.15.66:0/2861302730 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 0x7fdc6019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2861302730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 msgr2=0x7fdc6019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 0x7fdc6019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2861302730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc600728b0 msgr2=0x7fdc6019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc600728b0 0x7fdc6019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2861302730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500095f0 con 0x7fdc60110af0 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.827+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 0x7fdc601a0b70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdc580094f0 tx=0x7fdc5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.828+0000 7fdc567fc700 1 -- 172.21.15.66:0/2861302730 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004db0 con 0x7fdc60110af0 2023-07-21T22:25:10.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.828+0000 7fdc567fc700 1 -- 172.21.15.66:0/2861302730 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5800f040 con 0x7fdc60110af0 2023-07-21T22:25:10.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.828+0000 7fdc567fc700 1 -- 172.21.15.66:0/2861302730 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58005980 con 0x7fdc60110af0 2023-07-21T22:25:10.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.828+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc601a17c0 con 0x7fdc60110af0 2023-07-21T22:25:10.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.828+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc601a9000 con 0x7fdc60110af0 2023-07-21T22:25:10.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.831+0000 7fdc567fc700 1 -- 172.21.15.66:0/2861302730 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdc58005020 con 0x7fdc60110af0 2023-07-21T22:25:10.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.832+0000 7fdc7b23cb80 1 Processor -- start 2023-07-21T22:25:10.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.832+0000 7fdc7b23cb80 1 -- start start 2023-07-21T22:25:10.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc567fc700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdc4c0a1340 0x7fdc4c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc567fc700 1 -- 172.21.15.66:0/2861302730 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdc580cc860 con 0x7fdc60110af0 2023-07-21T22:25:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de31c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de317350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de317da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de21ccb0 con 0x5633de31cac0 2023-07-21T22:25:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de26fa40 con 0x5633de31c1b0 2023-07-21T22:25:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de268ed0 con 0x5633de317930 2023-07-21T22:25:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdc4c0a1340 0x7fdc4c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc64f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de317350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc6579b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de31c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc64f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de317350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49548/0 (socket says 172.21.15.66:49548) 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc64f9a700 1 -- 172.21.15.66:0/766683597 learned_addr learned my addr 172.21.15.66:0/766683597 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.833+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de317da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc6579b700 1 -- 172.21.15.66:0/766683597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 msgr2=0x5633de317da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc6579b700 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de317da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc6579b700 1 -- 172.21.15.66:0/766683597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 msgr2=0x5633de317350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc6579b700 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de317350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc6579b700 1 -- 172.21.15.66:0/766683597 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633de096de0 con 0x5633de31c1b0 2023-07-21T22:25:10.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de317da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:10.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdc4c0a1340 0x7fdc4c0a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005850 tx=0x7fdc50005c00 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc6579b700 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de31c580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009cb0 tx=0x7fdc5c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4effc891bb502f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc4b7fe700 1 -- 172.21.15.66:0/766683597 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c011d20 con 0x5633de31c1b0 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc4b7fe700 1 -- 172.21.15.66:0/766683597 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5c011e80 con 0x5633de31c1b0 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.834+0000 7fdc4b7fe700 1 -- 172.21.15.66:0/766683597 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c005280 con 0x5633de31c1b0 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/766683597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 msgr2=0x5633de31c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de31c580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009cb0 tx=0x7fdc5c011440 comp rx=0 tx=0).stop 2023-07-21T22:25:10.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/766683597 shutdown_connections 2023-07-21T22:25:10.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de31c580 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de317da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/766683597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de317350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/766683597 >> 172.21.15.66:0/766683597 conn(0x5633de30cef0 msgr2=0x5633de312830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/766683597 shutdown_connections 2023-07-21T22:25:10.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.835+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/766683597 wait complete. 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.836+0000 7fdc7b23cb80 1 -- start start 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.836+0000 7fdc7b23cb80 1 Processor -- start 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de323c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31c1b0 0x5633de320e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31cac0 0x5633de321480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de208ac0 con 0x5633de31c1b0 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de2087c0 con 0x5633de31cac0 2023-07-21T22:25:10.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de0ec920 con 0x5633de317930 2023-07-21T22:25:10.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc7b23cb80 10 client.0 _ll_register_callbacks cb 0x5633de206760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:10.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de323c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc64f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31c1b0 0x5633de320e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de323c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52368/0 (socket says 172.21.15.66:52368) 2023-07-21T22:25:10.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 -- 172.21.15.66:0/1619970400 learned_addr learned my addr 172.21.15.66:0/1619970400 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc65f9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31cac0 0x5633de321480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 -- 172.21.15.66:0/1619970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31cac0 msgr2=0x5633de321480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31cac0 0x5633de321480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 -- 172.21.15.66:0/1619970400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31c1b0 msgr2=0x5633de320e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31c1b0 0x5633de320e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc6579b700 1 -- 172.21.15.66:0/1619970400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc601a9000 con 0x5633de317930 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.837+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31c1b0 0x5633de320e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc6579b700 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de323c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c004ad0 tx=0x7fdc5c011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5aca7375d7c8416 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:10.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc48ff9700 1 -- 172.21.15.66:0/1619970400 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c004050 con 0x5633de317930 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633de261eb0 con 0x5633de317930 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc48ff9700 1 -- 172.21.15.66:0/1619970400 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdc5c0041b0 con 0x5633de317930 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc48ff9700 1 -- 172.21.15.66:0/1619970400 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5c004370 con 0x5633de317930 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc7b23cb80 10 client.26296 fetch_fsmap learned FSMap version 30 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.838+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fdc5c0098e0 con 0x5633de317930 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc48ff9700 1 -- 172.21.15.66:0/1619970400 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633de261eb0 con 0x5633de317930 2023-07-21T22:25:10.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc48ff9700 1 -- 172.21.15.66:0/1619970400 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc5c0051e0 con 0x5633de317930 2023-07-21T22:25:10.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc7b23cb80 10 client.26296 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:25:10.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc7b23cb80 10 client.26296 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:25:10.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc7b23cb80 20 client.26296 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:10.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de331310 0x5633de333720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc7b23cb80 4 client.26296 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:10.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.839+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633de224050 con 0x5633de331310 2023-07-21T22:25:10.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.840+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de331310 0x5633de333720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.840+0000 7fdc48ff9700 10 client.26296 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.840+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de331310 0x5633de333720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3032cce1edbc67b server_cookie=97b24d2f970c0560 in_seq=0 out_seq=0 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc48ff9700 1 -- 172.21.15.66:0/1619970400 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5633de224050 con 0x5633de331310 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc48ff9700 10 client.26296 handle_command_reply: tid=0 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 client.26296 shutdown 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 msgr2=0x5633de323c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de323c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c004ad0 tx=0x7fdc5c011f90 comp rx=0 tx=0).stop 2023-07-21T22:25:10.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 shutdown_connections 2023-07-21T22:25:10.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de331310 0x5633de333720 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31cac0 0x5633de321480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de317930 0x5633de323c40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1619970400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31c1b0 0x5633de320e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc48ff9700 0 client.26296 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:10.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.841+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 >> 172.21.15.66:0/1619970400 conn(0x5633de30cef0 msgr2=0x5633de312420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.842+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 shutdown_connections 2023-07-21T22:25:10.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.842+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1619970400 wait complete. 2023-07-21T22:25:10.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.842+0000 7fdc7b23cb80 20 client.26296 trim_cache size 0 max 16384 2023-07-21T22:25:10.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.855+0000 7fdc7b23cb80 1 Processor -- start 2023-07-21T22:25:10.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.855+0000 7fdc7b23cb80 1 -- start start 2023-07-21T22:25:10.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 0x5633de323770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de3206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31cac0 0x5633de320c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de321b60 con 0x5633de317930 2023-07-21T22:25:10.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de313350 con 0x5633de31c1b0 2023-07-21T22:25:10.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de32de00 con 0x5633de31cac0 2023-07-21T22:25:10.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 0x5633de323770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 0x5633de323770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49566/0 (socket says 172.21.15.66:49566) 2023-07-21T22:25:10.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 -- 172.21.15.66:0/1084017380 learned_addr learned my addr 172.21.15.66:0/1084017380 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31cac0 0x5633de320c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de3206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.906 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 -- 172.21.15.66:0/1084017380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31cac0 msgr2=0x5633de320c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31cac0 0x5633de320c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 -- 172.21.15.66:0/1084017380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 msgr2=0x5633de3206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de3206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 -- 172.21.15.66:0/1084017380 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633de07fc60 con 0x5633de317930 2023-07-21T22:25:10.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.856+0000 7fdc6579b700 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 0x5633de323770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000c00 tx=0x7fdc5c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a77d0aa566bea5d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc4bfff700 1 -- 172.21.15.66:0/1084017380 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c022400 con 0x5633de317930 2023-07-21T22:25:10.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc4bfff700 1 -- 172.21.15.66:0/1084017380 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc5c022e70 con 0x5633de317930 2023-07-21T22:25:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc4bfff700 1 -- 172.21.15.66:0/1084017380 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c005080 con 0x5633de317930 2023-07-21T22:25:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1084017380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 msgr2=0x5633de323770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 0x5633de323770 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000c00 tx=0x7fdc5c011440 comp rx=0 tx=0).stop 2023-07-21T22:25:10.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1084017380 shutdown_connections 2023-07-21T22:25:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31cac0 0x5633de320c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de317930 0x5633de323770 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/1084017380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de31c1b0 0x5633de3206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.857+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1084017380 >> 172.21.15.66:0/1084017380 conn(0x5633de30cef0 msgr2=0x5633de315a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.858+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1084017380 shutdown_connections 2023-07-21T22:25:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.858+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/1084017380 wait complete. 2023-07-21T22:25:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.858+0000 7fdc7b23cb80 1 -- start start 2023-07-21T22:25:10.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.858+0000 7fdc7b23cb80 1 Processor -- start 2023-07-21T22:25:10.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de317930 0x5633de326590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31c1b0 0x5633de326ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de327010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de3843d0 con 0x5633de31cac0 2023-07-21T22:25:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de384540 con 0x5633de317930 2023-07-21T22:25:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633de384700 con 0x5633de31c1b0 2023-07-21T22:25:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc7b23cb80 10 client.0 _ll_register_callbacks cb 0x7fdc5c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:10.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de327010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de327010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49578/0 (socket says 172.21.15.66:49578) 2023-07-21T22:25:10.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2890160668 learned_addr learned my addr 172.21.15.66:0/2890160668 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:10.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc6579b700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de317930 0x5633de326590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc64f9a700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31c1b0 0x5633de326ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31c1b0 msgr2=0x5633de326ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31c1b0 0x5633de326ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2890160668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de317930 msgr2=0x5633de326590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:10.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de317930 0x5633de326590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc65f9c700 1 -- 172.21.15.66:0/2890160668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c0098e0 con 0x5633de31cac0 2023-07-21T22:25:10.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.859+0000 7fdc6579b700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de317930 0x5633de326590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:10.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc65f9c700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de327010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc580c9c60 tx=0x7fdc58099140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479f14391b56538a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:10.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:25:10.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc48ff9700 1 -- 172.21.15.66:0/2890160668 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5809d350 con 0x5633de31cac0 2023-07-21T22:25:10.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633de1c24f0 con 0x5633de31cac0 2023-07-21T22:25:10.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc48ff9700 1 -- 172.21.15.66:0/2890160668 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdc58095040 con 0x5633de31cac0 2023-07-21T22:25:10.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc48ff9700 1 -- 172.21.15.66:0/2890160668 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc580cc640 con 0x5633de31cac0 2023-07-21T22:25:10.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc7b23cb80 10 client.16272 fetch_fsmap learned FSMap version 30 2023-07-21T22:25:10.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5633de0bd250 con 0x5633de31cac0 2023-07-21T22:25:10.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.860+0000 7fdc48ff9700 1 -- 172.21.15.66:0/2890160668 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633de1c24f0 con 0x5633de31cac0 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc48ff9700 1 -- 172.21.15.66:0/2890160668 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 30) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdc58098a70 con 0x5633de31cac0 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc7b23cb80 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc7b23cb80 10 client.16272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc7b23cb80 20 client.16272 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de385de0 0x5633de3881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc7b23cb80 4 client.16272 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:25:10.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5633de0b7f70 con 0x5633de385de0 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc6579b700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de385de0 0x5633de3881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc48ff9700 10 client.16272 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.861+0000 7fdc6579b700 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de385de0 0x5633de3881f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffb4d3d8f2b12027 server_cookie=24161be43eaa6438 in_seq=0 out_seq=0 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc48ff9700 1 -- 172.21.15.66:0/2890160668 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5633de0b7f70 con 0x5633de385de0 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc48ff9700 10 client.16272 handle_command_reply: tid=0 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 client.16272 shutdown 2023-07-21T22:25:10.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 msgr2=0x5633de327010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de327010 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc580c9c60 tx=0x7fdc58099140 comp rx=0 tx=0).stop 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 shutdown_connections 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5633de385de0 0x5633de3881f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5633de317930 0x5633de326590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5633de31cac0 0x5633de327010 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 --2- 172.21.15.66:0/2890160668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5633de31c1b0 0x5633de326ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc48ff9700 0 client.16272 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:10.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 >> 172.21.15.66:0/2890160668 conn(0x5633de30cef0 msgr2=0x5633de3206c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 shutdown_connections 2023-07-21T22:25:10.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 1 -- 172.21.15.66:0/2890160668 wait complete. 2023-07-21T22:25:10.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.862+0000 7fdc7b23cb80 20 client.16272 trim_cache size 0 max 16384 2023-07-21T22:25:10.927 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:25:10.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdc4c0a1340 msgr2=0x7fdc4c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdc4c0a1340 0x7fdc4c0a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc50005850 tx=0x7fdc50005c00 comp rx=0 tx=0).stop 2023-07-21T22:25:10.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 msgr2=0x7fdc601a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:10.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 0x7fdc601a0b70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fdc580094f0 tx=0x7fdc5800d3b0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 shutdown_connections 2023-07-21T22:25:10.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdc600728b0 0x7fdc6019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdc4c0a1340 0x7fdc4c0a37f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdc60110af0 0x7fdc601a0b70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 --2- 172.21.15.66:0/2861302730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc600731c0 0x7fdc6019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:10.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 >> 172.21.15.66:0/2861302730 conn(0x7fdc600fb5e0 msgr2=0x7fdc600fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:10.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 shutdown_connections 2023-07-21T22:25:10.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:10.863+0000 7fdc679ff700 1 -- 172.21.15.66:0/2861302730 wait complete. 2023-07-21T22:25:10.932 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:25:11.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 -- 172.21.15.66:0/1958822517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00731c0 msgr2=0x7f9fd010bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/1958822517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00731c0 0x7f9fd010bdd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009990 tx=0x7f9fc0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 -- 172.21.15.66:0/1958822517 shutdown_connections 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/1958822517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 0x7f9fd010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/1958822517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00728b0 0x7f9fd0072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/1958822517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00731c0 0x7f9fd010bdd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 -- 172.21.15.66:0/1958822517 >> 172.21.15.66:0/1958822517 conn(0x7f9fd00fb5f0 msgr2=0x7f9fd00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 -- 172.21.15.66:0/1958822517 shutdown_connections 2023-07-21T22:25:11.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.099+0000 7f9fd81ff700 1 -- 172.21.15.66:0/1958822517 wait complete. 2023-07-21T22:25:11.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.100+0000 7f9fd81ff700 1 Processor -- start 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.100+0000 7f9fd81ff700 1 -- start start 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.100+0000 7f9fd81ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00728b0 0x7f9fd0197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.100+0000 7f9fd81ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00731c0 0x7f9fd01982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.100+0000 7f9fd81ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 0x7f9fd019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd81ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019cd80 con 0x7f9fd010c310 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd81ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019cef0 con 0x7f9fd00728b0 2023-07-21T22:25:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd81ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd019d0b0 con 0x7f9fd00731c0 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00731c0 0x7f9fd01982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd679c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 0x7f9fd019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00731c0 0x7f9fd01982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52392/0 (socket says 172.21.15.66:52392) 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd579a700 1 -- 172.21.15.66:0/3274091648 learned_addr learned my addr 172.21.15.66:0/3274091648 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00728b0 0x7f9fd0197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3274091648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00731c0 msgr2=0x7f9fd01982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00731c0 0x7f9fd01982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 msgr2=0x7f9fd019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 0x7f9fd019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3274091648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc009520 con 0x7f9fd00728b0 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.101+0000 7f9fd679c700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 0x7f9fd019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.102+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00728b0 0x7f9fd0197da0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e9f0 tx=0x7f9fcc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.102+0000 7f9fc6ffd700 1 -- 172.21.15.66:0/3274091648 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc00cb70 con 0x7f9fd00728b0 2023-07-21T22:25:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.102+0000 7f9fc6ffd700 1 -- 172.21.15.66:0/3274091648 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fcc00ee80 con 0x7f9fd00728b0 2023-07-21T22:25:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.102+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc8009c40 con 0x7f9fd00728b0 2023-07-21T22:25:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.102+0000 7f9fc6ffd700 1 -- 172.21.15.66:0/3274091648 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc003d30 con 0x7f9fd00728b0 2023-07-21T22:25:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.102+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc00095f0 con 0x7f9fd00728b0 2023-07-21T22:25:11.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.106+0000 7f9fc6ffd700 1 -- 172.21.15.66:0/3274091648 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9fcc00ccd0 con 0x7f9fd00728b0 2023-07-21T22:25:11.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.106+0000 7f9febaefb80 1 Processor -- start 2023-07-21T22:25:11.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 -- start start 2023-07-21T22:25:11.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae72580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fc6ffd700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9fbc0a12c0 0x7f9fbc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae6dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ead72cb0 con 0x5626eae6d930 2023-07-21T22:25:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eadc5a40 con 0x5626eae72ac0 2023-07-21T22:25:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9febaefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eadbeed0 con 0x5626eae721b0 2023-07-21T22:25:11.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fc6ffd700 1 -- 172.21.15.66:0/3274091648 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9fcc014070 con 0x7f9fd00728b0 2023-07-21T22:25:11.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fd679c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae72580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fd5f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fd579a700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9fbc0a12c0 0x7f9fbc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fd679c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae6dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fd5f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae6d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33984/0 (socket says 172.21.15.66:33984) 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.107+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3453680956 learned_addr learned my addr 172.21.15.66:0/3453680956 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3453680956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 msgr2=0x5626eae72580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae72580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3453680956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 msgr2=0x5626eae6dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae6dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd5f9b700 1 -- 172.21.15.66:0/3453680956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626eabecde0 con 0x5626eae72ac0 2023-07-21T22:25:11.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae6d350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0a1f70 tx=0x7f9fcc0999b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc487fd1065fb3bc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fd579a700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9fbc0a12c0 0x7f9fbc0a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005cd0 tx=0x7f9fc0005bc0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fbbfff700 1 -- 172.21.15.66:0/3453680956 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0cd160 con 0x5626eae72ac0 2023-07-21T22:25:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fbbfff700 1 -- 172.21.15.66:0/3453680956 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fcc09a050 con 0x5626eae72ac0 2023-07-21T22:25:11.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.108+0000 7f9fbbfff700 1 -- 172.21.15.66:0/3453680956 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc09a510 con 0x5626eae72ac0 2023-07-21T22:25:11.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 -- 172.21.15.66:0/3453680956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 msgr2=0x5626eae6d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae6d350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0a1f70 tx=0x7f9fcc0999b0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 -- 172.21.15.66:0/3453680956 shutdown_connections 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae6dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae72580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 --2- 172.21.15.66:0/3453680956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae6d350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 -- 172.21.15.66:0/3453680956 >> 172.21.15.66:0/3453680956 conn(0x5626eae62ef0 msgr2=0x5626eae68830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 -- 172.21.15.66:0/3453680956 shutdown_connections 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.109+0000 7f9febaefb80 1 -- 172.21.15.66:0/3453680956 wait complete. 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.110+0000 7f9febaefb80 1 -- start start 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.110+0000 7f9febaefb80 1 Processor -- start 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.110+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae76f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.110+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae779d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9febaefb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ead5eac0 con 0x5626eae6d930 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9febaefb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ead5e7c0 con 0x5626eae72ac0 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9febaefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eac42920 con 0x5626eae721b0 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9febaefb80 10 client.0 _ll_register_callbacks cb 0x5626ead5c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd5f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae76f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae77490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52410/0 (socket says 172.21.15.66:52410) 2023-07-21T22:25:11.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 -- 172.21.15.66:0/2588945794 learned_addr learned my addr 172.21.15.66:0/2588945794 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd679c700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 -- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 msgr2=0x5626eae779d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae779d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 -- 172.21.15.66:0/2588945794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 msgr2=0x5626eae76f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae76f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd579a700 1 -- 172.21.15.66:0/2588945794 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc00095f0 con 0x5626eae721b0 2023-07-21T22:25:11.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae76f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.111+0000 7f9fd679c700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9fd579a700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae77490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098e0 tx=0x7f9fc0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6607bfdf756e8602 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9fb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9fb97fa700 1 -- 172.21.15.66:0/2588945794 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002b070 con 0x5626eae721b0 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626ead184f0 con 0x5626eae721b0 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9fb97fa700 1 -- 172.21.15.66:0/2588945794 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9fc0025440 con 0x5626eae721b0 2023-07-21T22:25:11.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9fb97fa700 1 -- 172.21.15.66:0/2588945794 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc0025600 con 0x5626eae721b0 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9febaefb80 10 client.26314 fetch_fsmap learned FSMap version 31 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5626eabd5c60 con 0x5626eae721b0 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.112+0000 7f9fb97fa700 1 -- 172.21.15.66:0/2588945794 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5626ead184f0 con 0x5626eae721b0 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9fb97fa700 1 -- 172.21.15.66:0/2588945794 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9fc0037070 con 0x5626eae721b0 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9febaefb80 10 client.26314 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9febaefb80 10 client.26314 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:25:11.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9febaefb80 20 client.26314 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9febaefb80 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eae87440 0x5626eae89850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9febaefb80 4 client.26314 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626eadb7eb0 con 0x5626eae87440 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.113+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eae87440 0x5626eae89850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.114+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eae87440 0x5626eae89850 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89b731920d546302 server_cookie=ef87a44fa64f7f4 in_seq=0 out_seq=0 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.114+0000 7f9fb97fa700 10 client.26314 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:11.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9fb97fa700 1 -- 172.21.15.66:0/2588945794 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5626eadb7eb0 con 0x5626eae87440 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9fb97fa700 10 client.26314 handle_command_reply: tid=0 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 client.26314 shutdown 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 msgr2=0x5626eae77490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae77490 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098e0 tx=0x7f9fc0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 shutdown_connections 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eae87440 0x5626eae89850 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae72ac0 0x5626eae779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9fb97fa700 0 client.26314 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae6d930 0x5626eae76f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 --2- 172.21.15.66:0/2588945794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae77490 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 >> 172.21.15.66:0/2588945794 conn(0x5626eae62ef0 msgr2=0x5626eae68420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 shutdown_connections 2023-07-21T22:25:11.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 1 -- 172.21.15.66:0/2588945794 wait complete. 2023-07-21T22:25:11.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.115+0000 7f9febaefb80 20 client.26314 trim_cache size 0 max 16384 2023-07-21T22:25:11.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 Processor -- start 2023-07-21T22:25:11.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 -- start start 2023-07-21T22:25:11.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae7e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae6d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae73fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626ead184f0 con 0x5626eae72ac0 2023-07-21T22:25:11.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eae6e6d0 con 0x5626eae6d930 2023-07-21T22:25:11.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9febaefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eae64900 con 0x5626eae721b0 2023-07-21T22:25:11.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9fd679c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae73fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9fd679c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae73fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49620/0 (socket says 172.21.15.66:49620) 2023-07-21T22:25:11.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9fd679c700 1 -- 172.21.15.66:0/4039273372 learned_addr learned my addr 172.21.15.66:0/4039273372 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9fd579a700 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae6d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.129+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae7e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fd679c700 1 -- 172.21.15.66:0/4039273372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 msgr2=0x5626eae6d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fd679c700 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae6d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fd679c700 1 -- 172.21.15.66:0/4039273372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 msgr2=0x5626eae7e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fd679c700 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae7e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fd679c700 1 -- 172.21.15.66:0/4039273372 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626eac13250 con 0x5626eae72ac0 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fd679c700 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae73fe0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8003c70 tx=0x7f9fc8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af417087258c1ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fc4ff9700 1 -- 172.21.15.66:0/4039273372 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8004200 con 0x5626eae72ac0 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fc4ff9700 1 -- 172.21.15.66:0/4039273372 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fc800f040 con 0x5626eae72ac0 2023-07-21T22:25:11.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.130+0000 7f9fc4ff9700 1 -- 172.21.15.66:0/4039273372 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8007d80 con 0x5626eae72ac0 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 -- 172.21.15.66:0/4039273372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 msgr2=0x5626eae73fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae73fe0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8003c70 tx=0x7f9fc8004b40 comp rx=0 tx=0).stop 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 -- 172.21.15.66:0/4039273372 shutdown_connections 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae7e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae73fe0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 --2- 172.21.15.66:0/4039273372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 -- 172.21.15.66:0/4039273372 >> 172.21.15.66:0/4039273372 conn(0x5626eae62ef0 msgr2=0x5626eae6ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 -- 172.21.15.66:0/4039273372 shutdown_connections 2023-07-21T22:25:11.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 -- 172.21.15.66:0/4039273372 wait complete. 2023-07-21T22:25:11.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.131+0000 7f9febaefb80 1 -- start start 2023-07-21T22:25:11.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 Processor -- start 2023-07-21T22:25:11.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae77870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae767f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae76d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eae77e40 con 0x5626eae72ac0 2023-07-21T22:25:11.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eae77fb0 con 0x5626eae6d930 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626eae77270 con 0x5626eae721b0 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.132+0000 7f9febaefb80 10 client.0 _ll_register_callbacks cb 0x7f9fc000be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae767f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae767f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:52426/0 (socket says 172.21.15.66:52426) 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 -- 172.21.15.66:0/847485202 learned_addr learned my addr 172.21.15.66:0/847485202 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd679c700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae76d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae77870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 -- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 msgr2=0x5626eae77870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae77870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 -- 172.21.15.66:0/847485202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 msgr2=0x5626eae76d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae76d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 -- 172.21.15.66:0/847485202 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc8003a00 con 0x5626eae721b0 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fd579a700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae767f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fc002f900 tx=0x7f9fc002f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7516c3717e9953a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.133+0000 7f9fb97fa700 1 -- 172.21.15.66:0/847485202 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc002b040 con 0x5626eae721b0 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fcc09a050 con 0x5626eae721b0 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9fb97fa700 1 -- 172.21.15.66:0/847485202 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9fc003f860 con 0x5626eae721b0 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9fb97fa700 1 -- 172.21.15.66:0/847485202 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fc0005a00 con 0x5626eae721b0 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 10 client.26320 fetch_fsmap learned FSMap version 31 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5626eabeb2b0 con 0x5626eae721b0 2023-07-21T22:25:11.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9fb97fa700 1 -- 172.21.15.66:0/847485202 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fcc09a050 con 0x5626eae721b0 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9fb97fa700 1 -- 172.21.15.66:0/847485202 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f9fc000f6e0 con 0x5626eae721b0 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 10 client.26320 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 10 client.26320 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 20 client.26320 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eaedc0c0 0x5626eaede4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 4 client.26320 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.134+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626ead7a050 con 0x5626eaedc0c0 2023-07-21T22:25:11.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.135+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eaedc0c0 0x5626eaede4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.135+0000 7f9fd5f9b700 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eaedc0c0 0x5626eaede4d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1481a69245b77489 server_cookie=5edd17520ed95d41 in_seq=0 out_seq=0 2023-07-21T22:25:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.135+0000 7f9fb97fa700 10 client.26320 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.135+0000 7f9fb97fa700 1 -- 172.21.15.66:0/847485202 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626ead7a050 con 0x5626eaedc0c0 2023-07-21T22:25:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.135+0000 7f9fb97fa700 10 client.26320 handle_command_reply: tid=0 2023-07-21T22:25:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.135+0000 7f9febaefb80 1 client.26320 shutdown 2023-07-21T22:25:11.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 msgr2=0x5626eae767f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae767f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9fc002f900 tx=0x7f9fc002f640 comp rx=0 tx=0).stop 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 shutdown_connections 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5626eaedc0c0 0x5626eaede4d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5626eae6d930 0x5626eae77870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5626eae72ac0 0x5626eae76d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9fb97fa700 0 client.26320 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:11.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 --2- 172.21.15.66:0/847485202 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626eae721b0 0x5626eae767f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 >> 172.21.15.66:0/847485202 conn(0x5626eae62ef0 msgr2=0x5626eae72e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 shutdown_connections 2023-07-21T22:25:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 1 -- 172.21.15.66:0/847485202 wait complete. 2023-07-21T22:25:11.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9febaefb80 20 client.26320 trim_cache size 0 max 16384 2023-07-21T22:25:11.190 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:25:11.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9fbc0a12c0 msgr2=0x7f9fbc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9fbc0a12c0 0x7f9fbc0a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0005cd0 tx=0x7f9fc0005bc0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00728b0 msgr2=0x7f9fd0197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.136+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00728b0 0x7f9fd0197da0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e9f0 tx=0x7f9fcc00ed00 comp rx=0 tx=0).stop 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 shutdown_connections 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9fd00728b0 0x7f9fd0197da0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9fbc0a12c0 0x7f9fbc0a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9fd010c310 0x7f9fd019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 --2- 172.21.15.66:0/3274091648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd00731c0 0x7f9fd01982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 >> 172.21.15.66:0/3274091648 conn(0x7f9fd00fb5f0 msgr2=0x7f9fd0103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 shutdown_connections 2023-07-21T22:25:11.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.137+0000 7f9fd81ff700 1 -- 172.21.15.66:0/3274091648 wait complete. 2023-07-21T22:25:11.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:10 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2081259810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:25:11.200 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:25:11.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:10 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2081259810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:25:11.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:10 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2081259810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:25:11.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 -- 172.21.15.66:0/768839222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c100480 msgr2=0x7fac3c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 --2- 172.21.15.66:0/768839222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c100480 0x7fac3c10c910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b0a0 tx=0x7fac3400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 -- 172.21.15.66:0/768839222 shutdown_connections 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 --2- 172.21.15.66:0/768839222 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c100480 0x7fac3c10c910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 --2- 172.21.15.66:0/768839222 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3c105ae0 0x7fac3c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 --2- 172.21.15.66:0/768839222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c0ffad0 0x7fac3c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 -- 172.21.15.66:0/768839222 >> 172.21.15.66:0/768839222 conn(0x7fac3c0fb5f0 msgr2=0x7fac3c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 -- 172.21.15.66:0/768839222 shutdown_connections 2023-07-21T22:25:11.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.365+0000 7fac435ff700 1 -- 172.21.15.66:0/768839222 wait complete. 2023-07-21T22:25:11.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 Processor -- start 2023-07-21T22:25:11.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 -- start start 2023-07-21T22:25:11.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 0x7fac3c197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3c100480 0x7fac3c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c105ae0 0x7fac3c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c19cec0 con 0x7fac3c0ffad0 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c19d030 con 0x7fac3c105ae0 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.366+0000 7fac435ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c19d1f0 con 0x7fac3c100480 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac4139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 0x7fac3c197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac4139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 0x7fac3c197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49644/0 (socket says 172.21.15.66:49644) 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac4139b700 1 -- 172.21.15.66:0/499546561 learned_addr learned my addr 172.21.15.66:0/499546561 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c105ae0 0x7fac3c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac40b9a700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3c100480 0x7fac3c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 -- 172.21.15.66:0/499546561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3c100480 msgr2=0x7fac3c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3c100480 0x7fac3c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 -- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 msgr2=0x7fac3c197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 0x7fac3c197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 -- 172.21.15.66:0/499546561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac34009d00 con 0x7fac3c105ae0 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac4139b700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 0x7fac3c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:11.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.367+0000 7fac41b9c700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c105ae0 0x7fac3c19c880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac34009fd0 tx=0x7fac34007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.368+0000 7fac327fc700 1 -- 172.21.15.66:0/499546561 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3400e040 con 0x7fac3c105ae0 2023-07-21T22:25:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.368+0000 7fac327fc700 1 -- 172.21.15.66:0/499546561 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3401d470 con 0x7fac3c105ae0 2023-07-21T22:25:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.368+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c19d470 con 0x7fac3c105ae0 2023-07-21T22:25:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.368+0000 7fac327fc700 1 -- 172.21.15.66:0/499546561 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340052a0 con 0x7fac3c105ae0 2023-07-21T22:25:11.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.368+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c1a4d10 con 0x7fac3c105ae0 2023-07-21T22:25:11.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.371+0000 7fac327fc700 1 -- 172.21.15.66:0/499546561 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fac3401d5e0 con 0x7fac3c105ae0 2023-07-21T22:25:11.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.372+0000 7fac56ef9b80 1 Processor -- start 2023-07-21T22:25:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.372+0000 7fac56ef9b80 1 -- start start 2023-07-21T22:25:11.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.372+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb81b0 0x558e59cb8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cb3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 0x558e59cb3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59bb8cb0 con 0x558e59cb3930 2023-07-21T22:25:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59c0ba40 con 0x558e59cb81b0 2023-07-21T22:25:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59c04ed0 con 0x558e59cb8ac0 2023-07-21T22:25:11.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac327fc700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac280a12c0 0x7fac280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 0x558e59cb3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 0x558e59cb3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49646/0 (socket says 172.21.15.66:49646) 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 -- 172.21.15.66:0/3535763017 learned_addr learned my addr 172.21.15.66:0/3535763017 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac4139b700 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cb3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb81b0 0x558e59cb8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac327fc700 1 -- 172.21.15.66:0/499546561 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fac340ce4e0 con 0x7fac3c105ae0 2023-07-21T22:25:11.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac41b9c700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac280a12c0 0x7fac280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 -- 172.21.15.66:0/3535763017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 msgr2=0x558e59cb3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cb3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 -- 172.21.15.66:0/3535763017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb81b0 msgr2=0x558e59cb8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb81b0 0x558e59cb8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.373+0000 7fac40b9a700 1 -- 172.21.15.66:0/3535763017 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e59a32de0 con 0x558e59cb3930 2023-07-21T22:25:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.374+0000 7fac40b9a700 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 0x558e59cb3da0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fac2c011880 tx=0x7fac2c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28eea209f23101 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.374+0000 7fac41b9c700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac280a12c0 0x7fac280a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac3409b220 tx=0x7fac3400b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.374+0000 7fac277fe700 1 -- 172.21.15.66:0/3535763017 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c011ea0 con 0x558e59cb3930 2023-07-21T22:25:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.374+0000 7fac277fe700 1 -- 172.21.15.66:0/3535763017 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac2c0044f0 con 0x558e59cb3930 2023-07-21T22:25:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.374+0000 7fac277fe700 1 -- 172.21.15.66:0/3535763017 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c01e3a0 con 0x558e59cb3930 2023-07-21T22:25:11.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3535763017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 msgr2=0x558e59cb3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 0x558e59cb3da0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fac2c011880 tx=0x7fac2c0058e0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3535763017 shutdown_connections 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb81b0 0x558e59cb8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb3930 0x558e59cb3da0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3535763017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cb3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3535763017 >> 172.21.15.66:0/3535763017 conn(0x558e59ca8ef0 msgr2=0x558e59cae830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3535763017 shutdown_connections 2023-07-21T22:25:11.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3535763017 wait complete. 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.375+0000 7fac56ef9b80 1 -- start start 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.376+0000 7fac56ef9b80 1 Processor -- start 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cbce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cbd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59ba4ac0 con 0x558e59cb81b0 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59ba47c0 con 0x558e59cb3930 2023-07-21T22:25:11.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59a88920 con 0x558e59cb8ac0 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac56ef9b80 10 client.0 _ll_register_callbacks cb 0x558e59ba2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cbce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cbd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cbce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49658/0 (socket says 172.21.15.66:49658) 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 -- 172.21.15.66:0/856312927 learned_addr learned my addr 172.21.15.66:0/856312927 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac40b9a700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 -- 172.21.15.66:0/856312927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 msgr2=0x558e59cbd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cbd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 -- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 msgr2=0x558e59cbfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac4139b700 1 -- 172.21.15.66:0/856312927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c019040 con 0x558e59cb81b0 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.377+0000 7fac40b9a700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac4139b700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cbce70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fac38011d10 tx=0x7fac3800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44813793ed037dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:25:11.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac24ff9700 1 -- 172.21.15.66:0/856312927 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38016070 con 0x558e59cb81b0 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac24ff9700 1 -- 172.21.15.66:0/856312927 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fac38004050 con 0x558e59cb81b0 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e59b5e4f0 con 0x558e59cb81b0 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac24ff9700 1 -- 172.21.15.66:0/856312927 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac38004210 con 0x558e59cb81b0 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac56ef9b80 10 client.16302 fetch_fsmap learned FSMap version 31 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558e59a1bc60 con 0x558e59cb81b0 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.378+0000 7fac24ff9700 1 -- 172.21.15.66:0/856312927 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e59b5e4f0 con 0x558e59cb81b0 2023-07-21T22:25:11.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac24ff9700 1 -- 172.21.15.66:0/856312927 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fac3801a410 con 0x558e59cb81b0 2023-07-21T22:25:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac56ef9b80 10 client.16302 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:25:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac56ef9b80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac56ef9b80 20 client.16302 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59ccd4f0 0x558e59ccf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac56ef9b80 4 client.16302 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:11.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e59bfdeb0 con 0x558e59ccd4f0 2023-07-21T22:25:11.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.379+0000 7fac40b9a700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59ccd4f0 0x558e59ccf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.380+0000 7fac40b9a700 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59ccd4f0 0x558e59ccf900 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1986440f16f050 server_cookie=f176d93b1d87c5df in_seq=0 out_seq=0 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.380+0000 7fac24ff9700 10 client.16302 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.380+0000 7fac24ff9700 1 -- 172.21.15.66:0/856312927 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e59bfdeb0 con 0x558e59ccd4f0 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.380+0000 7fac24ff9700 10 client.16302 handle_command_reply: tid=0 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 client.16302 shutdown 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 msgr2=0x558e59cbce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cbce70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fac38011d10 tx=0x7fac3800f6a0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 shutdown_connections 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59ccd4f0 0x558e59ccf900 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb8ac0 0x558e59cbd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/856312927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cbce70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac24ff9700 0 client.16302 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 >> 172.21.15.66:0/856312927 conn(0x558e59ca8ef0 msgr2=0x558e59cae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 shutdown_connections 2023-07-21T22:25:11.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 1 -- 172.21.15.66:0/856312927 wait complete. 2023-07-21T22:25:11.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.381+0000 7fac56ef9b80 20 client.16302 trim_cache size 0 max 16384 2023-07-21T22:25:11.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 Processor -- start 2023-07-21T22:25:11.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 -- start start 2023-07-21T22:25:11.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59ca8ef0 0x558e59cbf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb81b0 0x558e59cbc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59cb46d0 con 0x558e59ca8ef0 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59cbdb90 con 0x558e59cb3930 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59cbe090 con 0x558e59cb81b0 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59ca8ef0 0x558e59cbf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.395+0000 7fac4139b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34066/0 (socket says 172.21.15.66:34066) 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59ca8ef0 0x558e59cbf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:49660/0 (socket says 172.21.15.66:49660) 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 -- 172.21.15.66:0/3423940034 learned_addr learned my addr 172.21.15.66:0/3423940034 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac40b9a700 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb81b0 0x558e59cbc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 -- 172.21.15.66:0/3423940034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb81b0 msgr2=0x558e59cbc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb81b0 0x558e59cbc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 -- 172.21.15.66:0/3423940034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59ca8ef0 msgr2=0x558e59cbf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59ca8ef0 0x558e59cbf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 -- 172.21.15.66:0/3423940034 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e59a59250 con 0x558e59cb3930 2023-07-21T22:25:11.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac40b9a700 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb81b0 0x558e59cbc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:25:11.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.396+0000 7fac4139b700 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fac380096b0 tx=0x7fac38009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af0dca3ea8a0d78 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.397+0000 7fac27fff700 1 -- 172.21.15.66:0/3423940034 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38016040 con 0x558e59cb3930 2023-07-21T22:25:11.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.397+0000 7fac27fff700 1 -- 172.21.15.66:0/3423940034 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3801a410 con 0x558e59cb3930 2023-07-21T22:25:11.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.397+0000 7fac27fff700 1 -- 172.21.15.66:0/3423940034 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38004700 con 0x558e59cb3930 2023-07-21T22:25:11.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3423940034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 msgr2=0x558e59cbfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fac380096b0 tx=0x7fac38009d40 comp rx=0 tx=0).stop 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3423940034 shutdown_connections 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb81b0 0x558e59cbc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59ca8ef0 0x558e59cbf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/3423940034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59cb3930 0x558e59cbfcd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3423940034 >> 172.21.15.66:0/3423940034 conn(0x558e59cc9b50 msgr2=0x558e59cb1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3423940034 shutdown_connections 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 -- 172.21.15.66:0/3423940034 wait complete. 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 -- start start 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.398+0000 7fac56ef9b80 1 Processor -- start 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59ca8ef0 0x558e59cc45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb3930 0x558e59cc29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cc2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59cc3610 con 0x558e59cb81b0 2023-07-21T22:25:11.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59cc3780 con 0x558e59ca8ef0 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e59d213c0 con 0x558e59cb3930 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac56ef9b80 10 client.0 _ll_register_callbacks cb 0x7fac380117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59ca8ef0 0x558e59cc45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac40b9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59ca8ef0 0x558e59cc45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34076/0 (socket says 172.21.15.66:34076) 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac4139b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb3930 0x558e59cc29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac40b9a700 1 -- 172.21.15.66:0/2792216343 learned_addr learned my addr 172.21.15.66:0/2792216343 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:11.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.399+0000 7fac40b9a700 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cc2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac4139b700 1 -- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59ca8ef0 msgr2=0x558e59cc45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac4139b700 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59ca8ef0 0x558e59cc45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac4139b700 1 -- 172.21.15.66:0/2792216343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 msgr2=0x558e59cc2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac4139b700 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cc2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac4139b700 1 -- 172.21.15.66:0/2792216343 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e59a1bc60 con 0x558e59cb3930 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac4139b700 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb3930 0x558e59cc29b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac3801acc0 tx=0x7fac38005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c26e61f2b36ac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:11.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.400+0000 7fac24ff9700 1 -- 172.21.15.66:0/2792216343 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38016040 con 0x558e59cb3930 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac2c0044f0 con 0x558e59cb3930 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac24ff9700 1 -- 172.21.15.66:0/2792216343 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fac38000e40 con 0x558e59cb3930 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac24ff9700 1 -- 172.21.15.66:0/2792216343 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac38005270 con 0x558e59cb3930 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 10 client.26335 fetch_fsmap learned FSMap version 31 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558e59a312b0 con 0x558e59cb3930 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac24ff9700 1 -- 172.21.15.66:0/2792216343 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3803a080 con 0x558e59cb3930 2023-07-21T22:25:11.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac24ff9700 1 -- 172.21.15.66:0/2792216343 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fac38030070 con 0x558e59cb3930 2023-07-21T22:25:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:25:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 10 client.26335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 20 client.26335 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59cb8ac0 0x558e59d24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 4 client.26335 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:25:11.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.401+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e59bc0050 con 0x558e59cb8ac0 2023-07-21T22:25:11.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac40b9a700 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59cb8ac0 0x558e59d24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:11.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac40b9a700 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59cb8ac0 0x558e59d24600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48ff03af98cfc2b server_cookie=3b449e983efea1ad in_seq=0 out_seq=0 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac24ff9700 10 client.26335 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac24ff9700 1 -- 172.21.15.66:0/2792216343 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e59bc0050 con 0x558e59cb8ac0 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac24ff9700 10 client.26335 handle_command_reply: tid=0 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac56ef9b80 1 client.26335 shutdown 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb3930 msgr2=0x558e59cc29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.402+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb3930 0x558e59cc29b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac3801acc0 tx=0x7fac38005740 comp rx=0 tx=0).stop 2023-07-21T22:25:11.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 shutdown_connections 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558e59ca8ef0 0x558e59cc45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558e59cb8ac0 0x558e59d24600 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558e59cb81b0 0x558e59cc2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac24ff9700 0 client.26335 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 --2- 172.21.15.66:0/2792216343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558e59cb3930 0x558e59cc29b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 >> 172.21.15.66:0/2792216343 conn(0x558e59cc9b50 msgr2=0x558e59cb29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 shutdown_connections 2023-07-21T22:25:11.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 1 -- 172.21.15.66:0/2792216343 wait complete. 2023-07-21T22:25:11.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac56ef9b80 20 client.26335 trim_cache size 0 max 16384 2023-07-21T22:25:11.456 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:25:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac280a12c0 msgr2=0x7fac280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac435ff700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac280a12c0 0x7fac280a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac3409b220 tx=0x7fac3400b040 comp rx=0 tx=0).stop 2023-07-21T22:25:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c105ae0 msgr2=0x7fac3c19c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac435ff700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c105ae0 0x7fac3c19c880 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac34009fd0 tx=0x7fac34007930 comp rx=0 tx=0).stop 2023-07-21T22:25:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 shutdown_connections 2023-07-21T22:25:11.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.403+0000 7fac435ff700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac3c105ae0 0x7fac3c19c880 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.404+0000 7fac435ff700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac280a12c0 0x7fac280a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.404+0000 7fac435ff700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac3c0ffad0 0x7fac3c197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.404+0000 7fac435ff700 1 --2- 172.21.15.66:0/499546561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac3c100480 0x7fac3c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.404+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 >> 172.21.15.66:0/499546561 conn(0x7fac3c0fb5f0 msgr2=0x7fac3c10ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.404+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 shutdown_connections 2023-07-21T22:25:11.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:11.404+0000 7fac435ff700 1 -- 172.21.15.66:0/499546561 wait complete. 2023-07-21T22:25:11.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 61119 2.01 MB/sec execute 38 sec latency 3874.386 ms 2023-07-21T22:25:12.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:11 smithi149 ceph-mon[129268]: pgmap v114: 129 pgs: 129 active+clean; 55 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 400 KiB/s wr, 81 op/s 2023-07-21T22:25:12.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:11 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:12.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:11 smithi104 ceph-mon[129021]: pgmap v114: 129 pgs: 129 active+clean; 55 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 400 KiB/s wr, 81 op/s 2023-07-21T22:25:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:11 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:12.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:11 smithi066 ceph-mon[126379]: pgmap v114: 129 pgs: 129 active+clean; 55 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 400 KiB/s wr, 81 op/s 2023-07-21T22:25:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:11 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:12.468 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:25:12.468 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:25:12.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 61119 1.96 MB/sec execute 39 sec latency 4874.439 ms 2023-07-21T22:25:12.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.662+0000 7f5e143ff700 1 -- 172.21.15.66:0/2978680444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0728b0 msgr2=0x7f5e0c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.662+0000 7f5e143ff700 1 --2- 172.21.15.66:0/2978680444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c072c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009f90 tx=0x7f5e0800b330 comp rx=0 tx=0).stop 2023-07-21T22:25:12.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 -- 172.21.15.66:0/2978680444 shutdown_connections 2023-07-21T22:25:12.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 --2- 172.21.15.66:0/2978680444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c10c310 0x7f5e0c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 --2- 172.21.15.66:0/2978680444 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c072c80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 --2- 172.21.15.66:0/2978680444 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c0731c0 0x7f5e0c10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 -- 172.21.15.66:0/2978680444 >> 172.21.15.66:0/2978680444 conn(0x7f5e0c0fb5f0 msgr2=0x7f5e0c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 -- 172.21.15.66:0/2978680444 shutdown_connections 2023-07-21T22:25:12.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.663+0000 7f5e143ff700 1 -- 172.21.15.66:0/2978680444 wait complete. 2023-07-21T22:25:12.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.664+0000 7f5e143ff700 1 Processor -- start 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 -- start start 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0731c0 0x7f5e0c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c10c310 0x7f5e0c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c19cf70 con 0x7f5e0c0731c0 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c19d0e0 con 0x7f5e0c0728b0 2023-07-21T22:25:12.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e143ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c19d2a0 con 0x7f5e0c10c310 2023-07-21T22:25:12.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57624/0 (socket says 172.21.15.66:57624) 2023-07-21T22:25:12.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e1299c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c10c310 0x7f5e0c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e1219b700 1 -- 172.21.15.66:0/725524485 learned_addr learned my addr 172.21.15.66:0/725524485 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:12.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.665+0000 7f5e1199a700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0731c0 0x7f5e0c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e1219b700 1 -- 172.21.15.66:0/725524485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c10c310 msgr2=0x7f5e0c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e1219b700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c10c310 0x7f5e0c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e1219b700 1 -- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0731c0 msgr2=0x7f5e0c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e1219b700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0731c0 0x7f5e0c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e1219b700 1 -- 172.21.15.66:0/725524485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e08009c40 con 0x7f5e0c0728b0 2023-07-21T22:25:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e1219b700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c197f90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800b6c0 tx=0x7f5e08004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e037fe700 1 -- 172.21.15.66:0/725524485 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800e040 con 0x7f5e0c0728b0 2023-07-21T22:25:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c19d520 con 0x7f5e0c0728b0 2023-07-21T22:25:12.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e037fe700 1 -- 172.21.15.66:0/725524485 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e0800bbf0 con 0x7f5e0c0728b0 2023-07-21T22:25:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.666+0000 7f5e037fe700 1 -- 172.21.15.66:0/725524485 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e080052a0 con 0x7f5e0c0728b0 2023-07-21T22:25:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.667+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c1a4dc0 con 0x7f5e0c0728b0 2023-07-21T22:25:12.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.670+0000 7f5e037fe700 1 -- 172.21.15.66:0/725524485 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5e0801d470 con 0x7f5e0c0728b0 2023-07-21T22:25:12.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.671+0000 7f5e27cbbb80 1 Processor -- start 2023-07-21T22:25:12.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 -- start start 2023-07-21T22:25:12.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 0x55efc9582600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582b40 0x55efc957d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc957d9b0 0x55efc957de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc9482d30 con 0x55efc957d9b0 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc94d5ac0 con 0x55efc9582230 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc94cef50 con 0x55efc9582b40 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e1299c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 0x55efc9582600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582b40 0x55efc957d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e1299c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 0x55efc9582600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57628/0 (socket says 172.21.15.66:57628) 2023-07-21T22:25:12.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e1299c700 1 -- 172.21.15.66:0/398384857 learned_addr learned my addr 172.21.15.66:0/398384857 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:12.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e1199a700 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc957d9b0 0x55efc957de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.672+0000 7f5e037fe700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5df80a12c0 0x7f5df80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e037fe700 1 -- 172.21.15.66:0/725524485 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5e080ce4e0 con 0x7f5e0c0728b0 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1219b700 1 -- 172.21.15.66:0/398384857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 msgr2=0x55efc9582600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1219b700 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 0x55efc9582600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1219b700 1 -- 172.21.15.66:0/398384857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc957d9b0 msgr2=0x55efc957de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1219b700 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc957d9b0 0x55efc957de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1219b700 1 -- 172.21.15.66:0/398384857 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc92fce60 con 0x55efc9582b40 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1299c700 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 0x55efc9582600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:12.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1299c700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5df80a12c0 0x7f5df80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e1219b700 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582b40 0x55efc957d3d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e080cb880 tx=0x7f5e08009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545e150987f74e62 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e00bfe700 1 -- 172.21.15.66:0/398384857 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08009ac0 con 0x55efc9582b40 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e00bfe700 1 -- 172.21.15.66:0/398384857 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e0809add0 con 0x55efc9582b40 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.673+0000 7f5e00bfe700 1 -- 172.21.15.66:0/398384857 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e080a3650 con 0x55efc9582b40 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/398384857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582b40 msgr2=0x55efc957d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e1299c700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5df80a12c0 0x7f5df80a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e04004120 tx=0x7f5e04004750 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:12.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582b40 0x55efc957d3d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e080cb880 tx=0x7f5e08009920 comp rx=0 tx=0).stop 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/398384857 shutdown_connections 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582230 0x55efc9582600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc957d9b0 0x55efc957de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/398384857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582b40 0x55efc957d3d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/398384857 >> 172.21.15.66:0/398384857 conn(0x55efc9572f70 msgr2=0x55efc95788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/398384857 shutdown_connections 2023-07-21T22:25:12.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.674+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/398384857 wait complete. 2023-07-21T22:25:12.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.675+0000 7f5e27cbbb80 1 -- start start 2023-07-21T22:25:12.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.675+0000 7f5e27cbbb80 1 Processor -- start 2023-07-21T22:25:12.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 0x55efc9589ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582230 0x55efc9586ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582b40 0x55efc9587630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc946eb40 con 0x55efc9582230 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc946e840 con 0x55efc9582b40 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc93529a0 con 0x55efc957d9b0 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e27cbbb80 10 client.0 _ll_register_callbacks cb 0x55efc946c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 0x55efc9589ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 0x55efc9589ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:48366/0 (socket says 172.21.15.66:48366) 2023-07-21T22:25:12.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e1219b700 1 -- 172.21.15.66:0/1449667476 learned_addr learned my addr 172.21.15.66:0/1449667476 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:12.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e1299c700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582b40 0x55efc9587630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.676+0000 7f5e1199a700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582230 0x55efc9586ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e1219b700 1 -- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582b40 msgr2=0x55efc9587630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e1219b700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582b40 0x55efc9587630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e1219b700 1 -- 172.21.15.66:0/1449667476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582230 msgr2=0x55efc9586ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e1219b700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582230 0x55efc9586ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e1219b700 1 -- 172.21.15.66:0/1449667476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e04009970 con 0x55efc957d9b0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e1219b700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 0x55efc9589ce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e0809b220 tx=0x7f5e0809abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a2a552eb9d61644 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5df5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5df5ffb700 1 -- 172.21.15.66:0/1449667476 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08095ea0 con 0x55efc957d9b0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5df5ffb700 1 -- 172.21.15.66:0/1449667476 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5e0801de40 con 0x55efc957d9b0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efc9428570 con 0x55efc957d9b0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.677+0000 7f5df5ffb700 1 -- 172.21.15.66:0/1449667476 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e0809b060 con 0x55efc957d9b0 2023-07-21T22:25:12.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 10 client.26347 fetch_fsmap learned FSMap version 31 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f5e0c1a4dc0 con 0x55efc957d9b0 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5df5ffb700 1 -- 172.21.15.66:0/1449667476 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55efc9428570 con 0x55efc957d9b0 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5df5ffb700 1 -- 172.21.15.66:0/1449667476 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5e0809b4f0 con 0x55efc957d9b0 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 10 client.26347 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 10 client.26347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 20 client.26347 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:12.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc9596ee0 0x55efc95992f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 4 client.26347 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.678+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efc94c7f30 con 0x55efc9596ee0 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.679+0000 7f5e1199a700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc9596ee0 0x55efc95992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.679+0000 7f5e1199a700 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc9596ee0 0x55efc95992f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33dd539f3fc5831b server_cookie=343c953319dcf29f in_seq=0 out_seq=0 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.679+0000 7f5df5ffb700 10 client.26347 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5df5ffb700 1 -- 172.21.15.66:0/1449667476 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55efc94c7f30 con 0x55efc9596ee0 2023-07-21T22:25:12.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5df5ffb700 10 client.26347 handle_command_reply: tid=0 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 client.26347 shutdown 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 msgr2=0x55efc9589ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 0x55efc9589ce0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e0809b220 tx=0x7f5e0809abf0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 shutdown_connections 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc9582b40 0x55efc9587630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc9596ee0 0x55efc95992f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc957d9b0 0x55efc9589ce0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/1449667476 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582230 0x55efc9586ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5df5ffb700 0 client.26347 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:12.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 >> 172.21.15.66:0/1449667476 conn(0x55efc9572f70 msgr2=0x55efc9578440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:12.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.680+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 shutdown_connections 2023-07-21T22:25:12.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.681+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/1449667476 wait complete. 2023-07-21T22:25:12.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.681+0000 7f5e27cbbb80 20 client.26347 trim_cache size 0 max 16384 2023-07-21T22:25:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.699+0000 7f5e27cbbb80 1 Processor -- start 2023-07-21T22:25:12.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.699+0000 7f5e27cbbb80 1 -- start start 2023-07-21T22:25:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc95861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc957d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc9575100 con 0x55efc9582b40 2023-07-21T22:25:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc9579f00 con 0x55efc957d9b0 2023-07-21T22:25:12.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc9581f00 con 0x55efc9582230 2023-07-21T22:25:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc95861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc95861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57644/0 (socket says 172.21.15.66:57644) 2023-07-21T22:25:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 -- 172.21.15.66:0/4175912455 learned_addr learned my addr 172.21.15.66:0/4175912455 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1199a700 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1299c700 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc957d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 -- 172.21.15.66:0/4175912455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 msgr2=0x55efc958e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 -- 172.21.15.66:0/4175912455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 msgr2=0x55efc957d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc957d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.700+0000 7f5e1219b700 1 -- 172.21.15.66:0/4175912455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efc92e5ce0 con 0x55efc957d9b0 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.701+0000 7f5e1219b700 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc95861d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e080060b0 tx=0x7f5e0800bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649014914c2dd44f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.701+0000 7f5e013ff700 1 -- 172.21.15.66:0/4175912455 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08095ea0 con 0x55efc957d9b0 2023-07-21T22:25:12.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.701+0000 7f5e013ff700 1 -- 172.21.15.66:0/4175912455 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e0809b290 con 0x55efc957d9b0 2023-07-21T22:25:12.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.701+0000 7f5e013ff700 1 -- 172.21.15.66:0/4175912455 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0809b990 con 0x55efc957d9b0 2023-07-21T22:25:12.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/4175912455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 msgr2=0x55efc95861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc95861d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e080060b0 tx=0x7f5e0800bf40 comp rx=0 tx=0).stop 2023-07-21T22:25:12.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/4175912455 shutdown_connections 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc95861d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc957d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/4175912455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/4175912455 >> 172.21.15.66:0/4175912455 conn(0x55efc9572f70 msgr2=0x55efc957ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/4175912455 shutdown_connections 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/4175912455 wait complete. 2023-07-21T22:25:12.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.702+0000 7f5e27cbbb80 1 -- start start 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 Processor -- start 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc958ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc958b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc95eae90 con 0x55efc9582b40 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc95eb000 con 0x55efc957d9b0 2023-07-21T22:25:12.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efc95eb1c0 con 0x55efc9582230 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e27cbbb80 10 client.0 _ll_register_callbacks cb 0x55efc946c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc958ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e1219b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc958ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57654/0 (socket says 172.21.15.66:57654) 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.703+0000 7f5e1219b700 1 -- 172.21.15.66:0/3665472514 learned_addr learned my addr 172.21.15.66:0/3665472514 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1299c700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc958b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1199a700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1219b700 1 -- 172.21.15.66:0/3665472514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 msgr2=0x55efc958b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1219b700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1219b700 1 -- 172.21.15.66:0/3665472514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 msgr2=0x55efc958b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1219b700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc958b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1219b700 1 -- 172.21.15.66:0/3665472514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c1a4dc0 con 0x55efc957d9b0 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e1219b700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc958ac40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e0809af90 tx=0x7f5e0809abf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b58f955a74f14316 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5df5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:25:12.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5df5ffb700 1 -- 172.21.15.66:0/3665472514 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08095ea0 con 0x55efc957d9b0 2023-07-21T22:25:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5df5ffb700 1 -- 172.21.15.66:0/3665472514 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5e0801de40 con 0x55efc957d9b0 2023-07-21T22:25:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.704+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e0809add0 con 0x55efc957d9b0 2023-07-21T22:25:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5df5ffb700 1 -- 172.21.15.66:0/3665472514 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e0809b060 con 0x55efc957d9b0 2023-07-21T22:25:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 10 client.26378 fetch_fsmap learned FSMap version 31 2023-07-21T22:25:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55efc93232d0 con 0x55efc957d9b0 2023-07-21T22:25:12.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5df5ffb700 1 -- 172.21.15.66:0/3665472514 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e0809add0 con 0x55efc957d9b0 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5df5ffb700 1 -- 172.21.15.66:0/3665472514 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 31) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5e080ce040 con 0x55efc957d9b0 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 10 client.26378 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 20 client.26378 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc95ec430 0x55efc95ee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 4 client.26378 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:25:12.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55efc948a0d0 con 0x55efc95ec430 2023-07-21T22:25:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.705+0000 7f5e1199a700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc95ec430 0x55efc95ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.706+0000 7f5e1199a700 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc95ec430 0x55efc95ee810 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4dc17ffc7e964a6 server_cookie=cb3dce1be059490f in_seq=0 out_seq=0 2023-07-21T22:25:12.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.706+0000 7f5df5ffb700 10 client.26378 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:12.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5df5ffb700 1 -- 172.21.15.66:0/3665472514 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55efc948a0d0 con 0x55efc95ec430 2023-07-21T22:25:12.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5df5ffb700 10 client.26378 handle_command_reply: tid=0 2023-07-21T22:25:12.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 client.26378 shutdown 2023-07-21T22:25:12.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 msgr2=0x55efc958ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc958ac40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e0809af90 tx=0x7f5e0809abf0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 shutdown_connections 2023-07-21T22:25:12.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55efc957d9b0 0x55efc958ac40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55efc95ec430 0x55efc95ee810 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55efc9582b40 0x55efc958b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 --2- 172.21.15.66:0/3665472514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55efc9582230 0x55efc958b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5df5ffb700 0 client.26378 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:12.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.707+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 >> 172.21.15.66:0/3665472514 conn(0x55efc9572f70 msgr2=0x55efc95837a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 shutdown_connections 2023-07-21T22:25:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e27cbbb80 1 -- 172.21.15.66:0/3665472514 wait complete. 2023-07-21T22:25:12.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e27cbbb80 20 client.26378 trim_cache size 0 max 16384 2023-07-21T22:25:12.761 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:25:12.761 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:25:12.762 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "923da864-9e48-49dd-a0d7-88fa8c6cafb6", 2023-07-21T22:25:12.762 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:25:12.762 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:25:12.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5df80a12c0 msgr2=0x7f5df80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e143ff700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5df80a12c0 0x7f5df80a3770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5e04004120 tx=0x7f5e04004750 comp rx=0 tx=0).stop 2023-07-21T22:25:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 msgr2=0x7f5e0c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.708+0000 7f5e143ff700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c197f90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800b6c0 tx=0x7f5e08004a30 comp rx=0 tx=0).stop 2023-07-21T22:25:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 shutdown_connections 2023-07-21T22:25:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e0c0728b0 0x7f5e0c197f90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5df80a12c0 0x7f5df80a3770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e0c10c310 0x7f5e0c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 --2- 172.21.15.66:0/725524485 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5e0c0731c0 0x7f5e0c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 >> 172.21.15.66:0/725524485 conn(0x7f5e0c0fb5f0 msgr2=0x7f5e0c10e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 shutdown_connections 2023-07-21T22:25:12.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.709+0000 7f5e143ff700 1 -- 172.21.15.66:0/725524485 wait complete. 2023-07-21T22:25:12.773 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:25:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.955+0000 7f88361ff700 1 -- 172.21.15.66:0/2416065478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f883010a070 msgr2=0x7f883010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.955+0000 7f88361ff700 1 --2- 172.21.15.66:0/2416065478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f883010a070 0x7f883010c4b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f882800b0a0 tx=0x7f882800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 -- 172.21.15.66:0/2416065478 shutdown_connections 2023-07-21T22:25:13.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 --2- 172.21.15.66:0/2416065478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 --2- 172.21.15.66:0/2416065478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f883010a070 0x7f883010c4b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 --2- 172.21.15.66:0/2416065478 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88301015b0 0x7f8830109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 -- 172.21.15.66:0/2416065478 >> 172.21.15.66:0/2416065478 conn(0x7f88300fa4b0 msgr2=0x7f88300fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 -- 172.21.15.66:0/2416065478 shutdown_connections 2023-07-21T22:25:13.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.956+0000 7f88361ff700 1 -- 172.21.15.66:0/2416065478 wait complete. 2023-07-21T22:25:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 Processor -- start 2023-07-21T22:25:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 -- start start 2023-07-21T22:25:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88301015b0 0x7f8830198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f883010a070 0x7f883019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883019ced0 con 0x7f883010a070 2023-07-21T22:25:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883019d040 con 0x7f8830100ca0 2023-07-21T22:25:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.957+0000 7f88361ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883019d200 con 0x7f88301015b0 2023-07-21T22:25:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f883010a070 0x7f883019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57670/0 (socket says 172.21.15.66:57670) 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 -- 172.21.15.66:0/1319429271 learned_addr learned my addr 172.21.15.66:0/1319429271 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882effd700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88301015b0 0x7f8830198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 -- 172.21.15.66:0/1319429271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88301015b0 msgr2=0x7f8830198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88301015b0 0x7f8830198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 -- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f883010a070 msgr2=0x7f883019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f883010a070 0x7f883019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 -- 172.21.15.66:0/1319429271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828009d00 con 0x7f8830100ca0 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882ffff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f883010a070 0x7f883019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.958+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830198050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f882000e9a0 tx=0x7f882000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.959+0000 7f882cff9700 1 -- 172.21.15.66:0/1319429271 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820009790 con 0x7f8830100ca0 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.959+0000 7f882cff9700 1 -- 172.21.15.66:0/1319429271 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8820010430 con 0x7f8830100ca0 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.959+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883019d4e0 con 0x7f8830100ca0 2023-07-21T22:25:13.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.959+0000 7f882cff9700 1 -- 172.21.15.66:0/1319429271 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820018680 con 0x7f8830100ca0 2023-07-21T22:25:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.959+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88301a4d20 con 0x7f8830100ca0 2023-07-21T22:25:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.962+0000 7f882cff9700 1 -- 172.21.15.66:0/1319429271 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f88200105a0 con 0x7f8830100ca0 2023-07-21T22:25:13.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.964+0000 7f882cff9700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88180a1340 0x7f88180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.964+0000 7f882cff9700 1 -- 172.21.15.66:0/1319429271 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8820014070 con 0x7f8830100ca0 2023-07-21T22:25:13.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.964+0000 7f882effd700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88180a1340 0x7f88180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 Processor -- start 2023-07-21T22:25:13.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 -- start start 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f882effd700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88180a1340 0x7f88180a37f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8830199500 tx=0x7f8824009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd0315e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03159350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd03159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0305ecb0 con 0x55dd03159930 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd030b1a40 con 0x55dd0315e1b0 2023-07-21T22:25:13.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f8849b50b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd030aaed0 con 0x55dd0315eac0 2023-07-21T22:25:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd0315e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd0315e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57676/0 (socket says 172.21.15.66:57676) 2023-07-21T22:25:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f882f7fe700 1 -- 172.21.15.66:0/1843845394 learned_addr learned my addr 172.21.15.66:0/1843845394 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f882effd700 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.965+0000 7f882ffff700 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd03159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882f7fe700 1 -- 172.21.15.66:0/1843845394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 msgr2=0x55dd03159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882f7fe700 1 -- 172.21.15.66:0/1843845394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 msgr2=0x55dd03159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd03159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882f7fe700 1 -- 172.21.15.66:0/1843845394 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd02ed8de0 con 0x55dd0315e1b0 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882effd700 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd0315e580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f88200a1e60 tx=0x7f8820099b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96cceb7373ba901d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f8815ffb700 1 -- 172.21.15.66:0/1843845394 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882009df70 con 0x55dd0315e1b0 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f8815ffb700 1 -- 172.21.15.66:0/1843845394 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88200a2390 con 0x55dd0315e1b0 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.966+0000 7f8815ffb700 1 -- 172.21.15.66:0/1843845394 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820099690 con 0x55dd0315e1b0 2023-07-21T22:25:13.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 -- 172.21.15.66:0/1843845394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 msgr2=0x55dd0315e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd0315e580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f88200a1e60 tx=0x7f8820099b70 comp rx=0 tx=0).stop 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 -- 172.21.15.66:0/1843845394 shutdown_connections 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd0315e580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd03159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1843845394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 -- 172.21.15.66:0/1843845394 >> 172.21.15.66:0/1843845394 conn(0x55dd0314eef0 msgr2=0x55dd03154830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:13.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 -- 172.21.15.66:0/1843845394 shutdown_connections 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.967+0000 7f8849b50b80 1 -- 172.21.15.66:0/1843845394 wait complete. 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.968+0000 7f8849b50b80 1 -- start start 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.968+0000 7f8849b50b80 1 Processor -- start 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 0x55dd03165bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd0315e1b0 0x55dd03162e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd031634e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0304aac0 con 0x55dd0315e1b0 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd0304a7c0 con 0x55dd03159930 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd02f2e920 con 0x55dd0315eac0 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f8849b50b80 10 client.0 _ll_register_callbacks cb 0x55dd03048760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 0x55dd03165bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd031634e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 0x55dd03165bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57692/0 (socket says 172.21.15.66:57692) 2023-07-21T22:25:13.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882f7fe700 1 -- 172.21.15.66:0/1598548220 learned_addr learned my addr 172.21.15.66:0/1598548220 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:13.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882effd700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd0315e1b0 0x55dd03162e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882ffff700 1 -- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 msgr2=0x55dd03165bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882ffff700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 0x55dd03165bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882ffff700 1 -- 172.21.15.66:0/1598548220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd0315e1b0 msgr2=0x55dd03162e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882ffff700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd0315e1b0 0x55dd03162e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882ffff700 1 -- 172.21.15.66:0/1598548220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824017070 con 0x55dd0315eac0 2023-07-21T22:25:13.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.969+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 0x55dd03165bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f882ffff700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd031634e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f88280062a0 tx=0x7f8828004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38dd00e30757283e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f880effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f880effd700 1 -- 172.21.15.66:0/1598548220 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800e040 con 0x55dd0315eac0 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd030044f0 con 0x55dd0315eac0 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f880effd700 1 -- 172.21.15.66:0/1598548220 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8828023070 con 0x55dd0315eac0 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f880effd700 1 -- 172.21.15.66:0/1598548220 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8828004e50 con 0x55dd0315eac0 2023-07-21T22:25:13.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f8849b50b80 10 client.26365 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.970+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f88301a4d20 con 0x55dd0315eac0 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f880effd700 1 -- 172.21.15.66:0/1598548220 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dd030044f0 con 0x55dd0315eac0 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f880effd700 1 -- 172.21.15.66:0/1598548220 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f882800b0d0 con 0x55dd0315eac0 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f8849b50b80 10 client.26365 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f8849b50b80 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f8849b50b80 20 client.26365 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:13.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031734e0 0x55dd031758f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f8849b50b80 4 client.26365 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd030a3eb0 con 0x55dd031734e0 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.971+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031734e0 0x55dd031758f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.972+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031734e0 0x55dd031758f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556b53a71cd6017d server_cookie=610ae56020881989 in_seq=0 out_seq=0 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.972+0000 7f880effd700 10 client.26365 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f880effd700 1 -- 172.21.15.66:0/1598548220 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd030a3eb0 con 0x55dd031734e0 2023-07-21T22:25:13.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f880effd700 10 client.26365 handle_command_reply: tid=0 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 client.26365 shutdown 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 msgr2=0x55dd031634e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd031634e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f88280062a0 tx=0x7f8828004a10 comp rx=0 tx=0).stop 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 shutdown_connections 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd03159930 0x55dd03165bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031734e0 0x55dd031758f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd031634e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1598548220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd0315e1b0 0x55dd03162e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 >> 172.21.15.66:0/1598548220 conn(0x55dd0314eef0 msgr2=0x55dd03154420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:13.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f880effd700 0 client.26365 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:13.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 shutdown_connections 2023-07-21T22:25:13.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.973+0000 7f8849b50b80 1 -- 172.21.15.66:0/1598548220 wait complete. 2023-07-21T22:25:13.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.974+0000 7f8849b50b80 20 client.26365 trim_cache size 0 max 16384 2023-07-21T22:25:13.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 Processor -- start 2023-07-21T22:25:13.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 -- start start 2023-07-21T22:25:13.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03165760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03158980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd03155ea0 con 0x55dd03159930 2023-07-21T22:25:13.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd030a3eb0 con 0x55dd0315e1b0 2023-07-21T22:25:13.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f8849b50b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd03170bf0 con 0x55dd0315eac0 2023-07-21T22:25:13.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.992+0000 7f882effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03165760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03165760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57704/0 (socket says 172.21.15.66:57704) 2023-07-21T22:25:13.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 -- 172.21.15.66:0/1134214124 learned_addr learned my addr 172.21.15.66:0/1134214124 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882ffff700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 -- 172.21.15.66:0/1134214124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 msgr2=0x55dd03158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 -- 172.21.15.66:0/1134214124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 msgr2=0x55dd0316a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 -- 172.21.15.66:0/1134214124 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd02ec1c60 con 0x55dd0315e1b0 2023-07-21T22:25:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882ffff700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882f7fe700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:13.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f882effd700 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03165760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8824009fd0 tx=0x7f8824014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20817edda95aa92b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:13.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.993+0000 7f88167fc700 1 -- 172.21.15.66:0/1134214124 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824024070 con 0x55dd0315e1b0 2023-07-21T22:25:13.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f88167fc700 1 -- 172.21.15.66:0/1134214124 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8824004510 con 0x55dd0315e1b0 2023-07-21T22:25:13.044 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f88167fc700 1 -- 172.21.15.66:0/1134214124 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824003af0 con 0x55dd0315e1b0 2023-07-21T22:25:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 -- 172.21.15.66:0/1134214124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 msgr2=0x55dd03165760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03165760 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8824009fd0 tx=0x7f8824014000 comp rx=0 tx=0).stop 2023-07-21T22:25:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 -- 172.21.15.66:0/1134214124 shutdown_connections 2023-07-21T22:25:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.045 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 --2- 172.21.15.66:0/1134214124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03165760 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 -- 172.21.15.66:0/1134214124 >> 172.21.15.66:0/1134214124 conn(0x55dd0314eef0 msgr2=0x55dd03157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 -- 172.21.15.66:0/1134214124 shutdown_connections 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.994+0000 7f8849b50b80 1 -- 172.21.15.66:0/1134214124 wait complete. 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.995+0000 7f8849b50b80 1 -- start start 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.995+0000 7f8849b50b80 1 Processor -- start 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.995+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03161610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f8849b50b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f8849b50b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd03162090 con 0x55dd03159930 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f8849b50b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd03162200 con 0x55dd0315e1b0 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f8849b50b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd031623c0 con 0x55dd0315eac0 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f8849b50b80 10 client.0 _ll_register_callbacks cb 0x7f882800bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57718/0 (socket says 172.21.15.66:57718) 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 -- 172.21.15.66:0/2200532641 learned_addr learned my addr 172.21.15.66:0/2200532641 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882f7fe700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 -- 172.21.15.66:0/2200532641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 msgr2=0x55dd03161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 -- 172.21.15.66:0/2200532641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 msgr2=0x55dd0316a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:13.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 -- 172.21.15.66:0/2200532641 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88301a4d20 con 0x55dd0315e1b0 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882f7fe700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f882effd700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03161610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8824011c30 tx=0x7f88240040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189a835e2de4331 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.996+0000 7f880effd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f880effd700 1 -- 172.21.15.66:0/2200532641 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824024040 con 0x55dd0315e1b0 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88200a2390 con 0x55dd0315e1b0 2023-07-21T22:25:13.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f880effd700 1 -- 172.21.15.66:0/2200532641 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8824003c80 con 0x55dd0315e1b0 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f880effd700 1 -- 172.21.15.66:0/2200532641 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8824004370 con 0x55dd0315e1b0 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f8849b50b80 10 client.26396 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f8824011c90 con 0x55dd0315e1b0 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f880effd700 1 -- 172.21.15.66:0/2200532641 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88200a2390 con 0x55dd0315e1b0 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f880effd700 1 -- 172.21.15.66:0/2200532641 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8824004b90 con 0x55dd0315e1b0 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f8849b50b80 10 client.26396 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:13.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.997+0000 7f8849b50b80 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f8849b50b80 20 client.26396 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f8849b50b80 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031c8810 0x55dd031cac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f8849b50b80 4 client.26396 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dd03066050 con 0x55dd031c8810 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f882f7fe700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031c8810 0x55dd031cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f880effd700 10 client.26396 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:13.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f882f7fe700 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031c8810 0x55dd031cac20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328987c97c3e13e2 server_cookie=7178b279903b8d8 in_seq=0 out_seq=0 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f880effd700 1 -- 172.21.15.66:0/2200532641 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dd03066050 con 0x55dd031c8810 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f880effd700 10 client.26396 handle_command_reply: tid=0 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.998+0000 7f8849b50b80 1 client.26396 shutdown 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 msgr2=0x55dd03161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03161610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8824011c30 tx=0x7f88240040d0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 shutdown_connections 2023-07-21T22:25:13.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55dd031c8810 0x55dd031cac20 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dd0315eac0 0x55dd03161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f880effd700 0 client.26396 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55dd03159930 0x55dd0316a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 --2- 172.21.15.66:0/2200532641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd0315e1b0 0x55dd03161610 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 >> 172.21.15.66:0/2200532641 conn(0x55dd0314eef0 msgr2=0x55dd03158980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 shutdown_connections 2023-07-21T22:25:13.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 1 -- 172.21.15.66:0/2200532641 wait complete. 2023-07-21T22:25:13.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:12.999+0000 7f8849b50b80 20 client.26396 trim_cache size 0 max 16384 2023-07-21T22:25:13.054 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:25:13.054 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:25:13.054 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:25:13.054 INFO:teuthology.orchestra.run.smithi066.stdout: "923da864-9e48-49dd-a0d7-88fa8c6cafb6": { 2023-07-21T22:25:13.055 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:25:13.055 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "923da864-9e48-49dd-a0d7-88fa8c6cafb6", 2023-07-21T22:25:13.055 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:25:13.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:25:13.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:25:13.055 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:25:13.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88180a1340 msgr2=0x7f88180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88180a1340 0x7f88180a37f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8830199500 tx=0x7f8824009260 comp rx=0 tx=0).stop 2023-07-21T22:25:13.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 msgr2=0x7f8830198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:13.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830198050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f882000e9a0 tx=0x7f882000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 shutdown_connections 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8830100ca0 0x7f8830198050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88180a1340 0x7f88180a37f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f883010a070 0x7f883019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 --2- 172.21.15.66:0/1319429271 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88301015b0 0x7f8830198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 >> 172.21.15.66:0/1319429271 conn(0x7f88300fa4b0 msgr2=0x7f88300fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 shutdown_connections 2023-07-21T22:25:13.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:13.000+0000 7f88361ff700 1 -- 172.21.15.66:0/1319429271 wait complete. 2023-07-21T22:25:13.064 INFO:tasks.cephfs.filesystem:scrub status for tag:923da864-9e48-49dd-a0d7-88fa8c6cafb6 - {'path': '/', 'tag': '923da864-9e48-49dd-a0d7-88fa8c6cafb6', 'options': 'recursive,force'} 2023-07-21T22:25:13.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:12 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:25:13.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:12 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:13.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:12 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:13.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:12 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:13.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:12 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:25:13.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:12 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:13.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:12 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:13.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:12 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:13.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:12 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:25:13.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:12 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:13.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:12 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:13.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:12 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:13.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 63535 2.23 MB/sec execute 40 sec latency 4878.838 ms 2023-07-21T22:25:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: pgmap v115: 129 pgs: 129 active+clean; 55 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 379 KiB/s wr, 74 op/s 2023-07-21T22:25:14.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:13 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:14.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: pgmap v115: 129 pgs: 129 active+clean; 55 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 379 KiB/s wr, 74 op/s 2023-07-21T22:25:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:13 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:14.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:14.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:14.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: pgmap v115: 129 pgs: 129 active+clean; 55 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 379 KiB/s wr, 74 op/s 2023-07-21T22:25:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:13 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:14.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64536 2.32 MB/sec execute 41 sec latency 203.305 ms 2023-07-21T22:25:15.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64536 2.27 MB/sec execute 42 sec latency 1203.389 ms 2023-07-21T22:25:16.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:15 smithi149 ceph-mon[129268]: pgmap v116: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 604 KiB/s wr, 107 op/s 2023-07-21T22:25:16.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:15 smithi104 ceph-mon[129021]: pgmap v116: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 604 KiB/s wr, 107 op/s 2023-07-21T22:25:16.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:15 smithi066 ceph-mon[126379]: pgmap v116: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 967 KiB/s rd, 604 KiB/s wr, 107 op/s 2023-07-21T22:25:16.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64536 2.22 MB/sec execute 43 sec latency 2203.461 ms 2023-07-21T22:25:17.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64536 2.17 MB/sec execute 44 sec latency 3203.560 ms 2023-07-21T22:25:18.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:17 smithi149 ceph-mon[129268]: pgmap v117: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 278 KiB/s wr, 41 op/s 2023-07-21T22:25:18.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:17 smithi104 ceph-mon[129021]: pgmap v117: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 278 KiB/s wr, 41 op/s 2023-07-21T22:25:18.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:17 smithi066 ceph-mon[126379]: pgmap v117: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 278 KiB/s wr, 41 op/s 2023-07-21T22:25:18.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64592 2.12 MB/sec execute 45 sec latency 3217.298 ms 2023-07-21T22:25:19.524 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64592 2.07 MB/sec execute 46 sec latency 1933.772 ms 2023-07-21T22:25:20.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:19 smithi149 ceph-mon[129268]: pgmap v118: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 278 KiB/s wr, 41 op/s 2023-07-21T22:25:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:19 smithi104 ceph-mon[129021]: pgmap v118: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 278 KiB/s wr, 41 op/s 2023-07-21T22:25:20.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:19 smithi066 ceph-mon[126379]: pgmap v118: 129 pgs: 129 active+clean; 57 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 278 KiB/s wr, 41 op/s 2023-07-21T22:25:20.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64592 2.03 MB/sec execute 47 sec latency 2933.959 ms 2023-07-21T22:25:21.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:20 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:25:21.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:20 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:25:21.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:20 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:25:21.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64592 1.99 MB/sec execute 48 sec latency 3934.101 ms 2023-07-21T22:25:22.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:21 smithi149 ceph-mon[129268]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 314 KiB/s wr, 49 op/s 2023-07-21T22:25:22.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:21 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:25:22.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:21 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:25:22.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:21 smithi104 ceph-mon[129021]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 314 KiB/s wr, 49 op/s 2023-07-21T22:25:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:21 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:25:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:21 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:25:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:21 smithi066 ceph-mon[126379]: pgmap v119: 129 pgs: 129 active+clean; 58 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 314 KiB/s wr, 49 op/s 2023-07-21T22:25:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:21 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:25:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:21 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:25:22.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 64592 1.95 MB/sec execute 49 sec latency 4934.178 ms 2023-07-21T22:25:23.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 66785 2.15 MB/sec execute 50 sec latency 4936.751 ms 2023-07-21T22:25:24.239 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:23 smithi149 ceph-mon[129268]: pgmap v120: 129 pgs: 129 active+clean; 58 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 262 KiB/s wr, 40 op/s 2023-07-21T22:25:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:23 smithi104 ceph-mon[129021]: pgmap v120: 129 pgs: 129 active+clean; 58 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 262 KiB/s wr, 40 op/s 2023-07-21T22:25:24.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:23 smithi066 ceph-mon[126379]: pgmap v120: 129 pgs: 129 active+clean; 58 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 262 KiB/s wr, 40 op/s 2023-07-21T22:25:24.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 68087 2.24 MB/sec execute 51 sec latency 82.190 ms 2023-07-21T22:25:25.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 68087 2.20 MB/sec execute 52 sec latency 1082.255 ms 2023-07-21T22:25:26.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:25 smithi149 ceph-mon[129268]: pgmap v121: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 597 KiB/s wr, 82 op/s 2023-07-21T22:25:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:25 smithi104 ceph-mon[129021]: pgmap v121: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 597 KiB/s wr, 82 op/s 2023-07-21T22:25:26.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:25 smithi066 ceph-mon[126379]: pgmap v121: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 597 KiB/s wr, 82 op/s 2023-07-21T22:25:26.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 68087 2.16 MB/sec execute 53 sec latency 2082.402 ms 2023-07-21T22:25:27.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 68087 2.12 MB/sec execute 54 sec latency 3082.472 ms 2023-07-21T22:25:28.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:27 smithi104 ceph-mon[129021]: pgmap v122: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 371 KiB/s wr, 49 op/s 2023-07-21T22:25:28.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:27 smithi066 ceph-mon[126379]: pgmap v122: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 371 KiB/s wr, 49 op/s 2023-07-21T22:25:28.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:27 smithi149 ceph-mon[129268]: pgmap v122: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 371 KiB/s wr, 49 op/s 2023-07-21T22:25:28.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 70395 2.31 MB/sec execute 55 sec latency 3087.545 ms 2023-07-21T22:25:29.525 INFO:tasks.workunit.client.0.smithi066.stdout: 1 71651 2.39 MB/sec execute 56 sec latency 22.065 ms 2023-07-21T22:25:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:29 smithi104 ceph-mon[129021]: pgmap v123: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 371 KiB/s wr, 49 op/s 2023-07-21T22:25:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:29 smithi066 ceph-mon[126379]: pgmap v123: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 371 KiB/s wr, 49 op/s 2023-07-21T22:25:30.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:29 smithi149 ceph-mon[129268]: pgmap v123: 129 pgs: 129 active+clean; 61 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 371 KiB/s wr, 49 op/s 2023-07-21T22:25:30.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 74005 2.56 MB/sec execute 57 sec latency 12.534 ms 2023-07-21T22:25:31.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 75265 2.64 MB/sec execute 58 sec latency 13.417 ms 2023-07-21T22:25:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:31 smithi104 ceph-mon[129021]: pgmap v124: 129 pgs: 129 active+clean; 64 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 902 KiB/s wr, 96 op/s 2023-07-21T22:25:32.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:31 smithi066 ceph-mon[126379]: pgmap v124: 129 pgs: 129 active+clean; 64 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 902 KiB/s wr, 96 op/s 2023-07-21T22:25:32.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:31 smithi149 ceph-mon[129268]: pgmap v124: 129 pgs: 129 active+clean; 64 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 902 KiB/s wr, 96 op/s 2023-07-21T22:25:32.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 77701 2.81 MB/sec execute 59 sec latency 23.365 ms 2023-07-21T22:25:33.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 78892 2.87 MB/sec execute 60 sec latency 36.083 ms 2023-07-21T22:25:34.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:33 smithi104 ceph-mon[129021]: pgmap v125: 129 pgs: 129 active+clean; 64 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 866 KiB/s wr, 89 op/s 2023-07-21T22:25:34.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:33 smithi066 ceph-mon[126379]: pgmap v125: 129 pgs: 129 active+clean; 64 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 866 KiB/s wr, 89 op/s 2023-07-21T22:25:34.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:33 smithi149 ceph-mon[129268]: pgmap v125: 129 pgs: 129 active+clean; 64 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 866 KiB/s wr, 89 op/s 2023-07-21T22:25:34.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 81447 3.05 MB/sec execute 61 sec latency 17.224 ms 2023-07-21T22:25:35.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 82498 3.09 MB/sec execute 62 sec latency 11.633 ms 2023-07-21T22:25:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:35 smithi104 ceph-mon[129021]: pgmap v126: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:25:36.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:35 smithi066 ceph-mon[126379]: pgmap v126: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:25:36.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:35 smithi149 ceph-mon[129268]: pgmap v126: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:25:36.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 85146 3.31 MB/sec execute 63 sec latency 19.030 ms 2023-07-21T22:25:37.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 86245 3.31 MB/sec execute 64 sec latency 13.026 ms 2023-07-21T22:25:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:37 smithi104 ceph-mon[129021]: pgmap v127: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-07-21T22:25:38.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:37 smithi066 ceph-mon[126379]: pgmap v127: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-07-21T22:25:38.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:37 smithi149 ceph-mon[129268]: pgmap v127: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-07-21T22:25:38.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 88855 3.50 MB/sec execute 65 sec latency 36.192 ms 2023-07-21T22:25:39.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 89896 3.50 MB/sec execute 66 sec latency 24.434 ms 2023-07-21T22:25:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:39 smithi104 ceph-mon[129021]: pgmap v128: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-07-21T22:25:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:39 smithi066 ceph-mon[126379]: pgmap v128: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-07-21T22:25:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:39 smithi149 ceph-mon[129268]: pgmap v128: 129 pgs: 129 active+clean; 70 MiB data, 638 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 137 op/s 2023-07-21T22:25:40.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 92496 3.70 MB/sec execute 67 sec latency 15.179 ms 2023-07-21T22:25:41.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 93742 3.72 MB/sec execute 68 sec latency 12.796 ms 2023-07-21T22:25:42.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:41 smithi104 ceph-mon[129021]: pgmap v129: 129 pgs: 129 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2023-07-21T22:25:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:41 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:25:42.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:41 smithi066 ceph-mon[126379]: pgmap v129: 129 pgs: 129 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2023-07-21T22:25:42.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:41 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:25:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:41 smithi149 ceph-mon[129268]: pgmap v129: 129 pgs: 129 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.0 MiB/s wr, 234 op/s 2023-07-21T22:25:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:25:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:25:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:25:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:41 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:25:42.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 96063 3.87 MB/sec execute 69 sec latency 16.940 ms 2023-07-21T22:25:43.065 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:25:43.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 -- 172.21.15.66:0/3998668331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc101f60 msgr2=0x7feddc102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 --2- 172.21.15.66:0/3998668331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc101f60 0x7feddc102330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0099c0 tx=0x7fedcc009cd0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 -- 172.21.15.66:0/3998668331 shutdown_connections 2023-07-21T22:25:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 --2- 172.21.15.66:0/3998668331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc106fa0 0x7feddc10eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 --2- 172.21.15.66:0/3998668331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc101f60 0x7feddc102330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 --2- 172.21.15.66:0/3998668331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc102900 0x7feddc1068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 -- 172.21.15.66:0/3998668331 >> 172.21.15.66:0/3998668331 conn(0x7feddc0fd7d0 msgr2=0x7feddc0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 -- 172.21.15.66:0/3998668331 shutdown_connections 2023-07-21T22:25:43.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.242+0000 7fede23ff700 1 -- 172.21.15.66:0/3998668331 wait complete. 2023-07-21T22:25:43.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.243+0000 7fede23ff700 1 Processor -- start 2023-07-21T22:25:43.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 -- start start 2023-07-21T22:25:43.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc101f60 0x7feddc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc102900 0x7feddc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 0x7feddc19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc19ce40 con 0x7feddc101f60 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc19cfb0 con 0x7feddc102900 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede23ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc19d170 con 0x7feddc106fa0 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc102900 0x7feddc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede099c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 0x7feddc19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede099c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 0x7feddc19c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39506/0 (socket says 172.21.15.66:39506) 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.244+0000 7fede099c700 1 -- 172.21.15.66:0/707627664 learned_addr learned my addr 172.21.15.66:0/707627664 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7feddbfff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc101f60 0x7feddc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fede099c700 1 -- 172.21.15.66:0/707627664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc102900 msgr2=0x7feddc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fede099c700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc102900 0x7feddc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fede099c700 1 -- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc101f60 msgr2=0x7feddc197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fede099c700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc101f60 0x7feddc197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fede099c700 1 -- 172.21.15.66:0/707627664 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0095f0 con 0x7feddc106fa0 2023-07-21T22:25:43.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7feddb7fe700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc102900 0x7feddc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:43.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7feddbfff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc101f60 0x7feddc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:43.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fede099c700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 0x7feddc19c800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fedd4007ee0 tx=0x7fedd400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fedd97fa700 1 -- 172.21.15.66:0/707627664 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4017070 con 0x7feddc106fa0 2023-07-21T22:25:43.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fedd97fa700 1 -- 172.21.15.66:0/707627664 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedd400f040 con 0x7feddc106fa0 2023-07-21T22:25:43.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.245+0000 7fedd97fa700 1 -- 172.21.15.66:0/707627664 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4005740 con 0x7feddc106fa0 2023-07-21T22:25:43.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.246+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc19d450 con 0x7feddc106fa0 2023-07-21T22:25:43.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.246+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc1a4c90 con 0x7feddc106fa0 2023-07-21T22:25:43.296 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.250+0000 7fedd97fa700 1 -- 172.21.15.66:0/707627664 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fedd400dc60 con 0x7feddc106fa0 2023-07-21T22:25:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 Processor -- start 2023-07-21T22:25:43.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 -- start start 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39acac0 0x559ab39a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab38accb0 con 0x559ab39ac1b0 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab38ffa40 con 0x559ab39a7930 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab38f8ed0 con 0x559ab39acac0 2023-07-21T22:25:43.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.251+0000 7fedd97fa700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fedc40a1340 0x7fedc40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7fedd97fa700 1 -- 172.21.15.66:0/707627664 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fedd40cc7a0 con 0x7feddc106fa0 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47610/0 (socket says 172.21.15.66:47610) 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddb7fe700 1 -- 172.21.15.66:0/3920747406 learned_addr learned my addr 172.21.15.66:0/3920747406 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddbfff700 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39acac0 0x559ab39a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7fede099c700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fedc40a1340 0x7fedc40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddbfff700 1 -- 172.21.15.66:0/3920747406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 msgr2=0x559ab39a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddbfff700 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddbfff700 1 -- 172.21.15.66:0/3920747406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 msgr2=0x559ab39ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddbfff700 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddbfff700 1 -- 172.21.15.66:0/3920747406 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab3726de0 con 0x559ab39acac0 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.252+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.253+0000 7fede099c700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fedc40a1340 0x7fedc40a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fedd40959c0 tx=0x7fedd400b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.253+0000 7feddbfff700 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39acac0 0x559ab39a7350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0098e0 tx=0x7fedcc005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476d42f18eda8d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.253+0000 7fedc27fc700 1 -- 172.21.15.66:0/3920747406 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01d040 con 0x559ab39acac0 2023-07-21T22:25:43.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.253+0000 7fedc27fc700 1 -- 172.21.15.66:0/3920747406 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedcc0037d0 con 0x559ab39acac0 2023-07-21T22:25:43.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.253+0000 7fedc27fc700 1 -- 172.21.15.66:0/3920747406 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00ba30 con 0x559ab39acac0 2023-07-21T22:25:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3920747406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39acac0 msgr2=0x559ab39a7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39acac0 0x559ab39a7350 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0098e0 tx=0x7fedcc005f00 comp rx=0 tx=0).stop 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3920747406 shutdown_connections 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3920747406 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39acac0 0x559ab39a7350 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3920747406 >> 172.21.15.66:0/3920747406 conn(0x559ab399cef0 msgr2=0x559ab39a2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3920747406 shutdown_connections 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3920747406 wait complete. 2023-07-21T22:25:43.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.254+0000 7fedf5d46b80 1 -- start start 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 Processor -- start 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab3898ac0 con 0x559ab39ac1b0 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab38987c0 con 0x559ab39acac0 2023-07-21T22:25:43.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab377c920 con 0x559ab39a7930 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7fedf5d46b80 10 client.0 _ll_register_callbacks cb 0x559ab3896760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.255+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39526/0 (socket says 172.21.15.66:39526) 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 -- 172.21.15.66:0/3102598707 learned_addr learned my addr 172.21.15.66:0/3102598707 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddbfff700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 -- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 msgr2=0x559ab39b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 -- 172.21.15.66:0/3102598707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 msgr2=0x559ab39b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 -- 172.21.15.66:0/3102598707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd001b040 con 0x559ab39a7930 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddbfff700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b3c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fedd0013410 tx=0x7fedd0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8762957e9501487b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7fedbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.256+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3102598707 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd0010ea0 con 0x559ab39a7930 2023-07-21T22:25:43.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ab38524f0 con 0x559ab39a7930 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3102598707 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fedd0004030 con 0x559ab39a7930 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3102598707 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd00041f0 con 0x559ab39a7930 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 10 client.26383 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7feddc10f3b0 con 0x559ab39a7930 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3102598707 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ab38524f0 con 0x559ab39a7930 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3102598707 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fedd000f5d0 con 0x559ab39a7930 2023-07-21T22:25:43.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 10 client.26383 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 10 client.26383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 20 client.26383 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab39c1470 0x559ab39c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 4 client.26383 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.257+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ab38f1eb0 con 0x559ab39c1470 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.258+0000 7feddbfff700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab39c1470 0x559ab39c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.258+0000 7feddbfff700 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab39c1470 0x559ab39c3880 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e1861ab2d40cfe server_cookie=5367aa8028382526 in_seq=0 out_seq=0 2023-07-21T22:25:43.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.258+0000 7fedbb7fe700 10 client.26383 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3102598707 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ab38f1eb0 con 0x559ab39c1470 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedbb7fe700 10 client.26383 handle_command_reply: tid=0 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 client.26383 shutdown 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 msgr2=0x559ab39b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b3c20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fedd0013410 tx=0x7fedd0009260 comp rx=0 tx=0).stop 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 shutdown_connections 2023-07-21T22:25:43.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab39c1470 0x559ab39c3880 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b3c20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3102598707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 >> 172.21.15.66:0/3102598707 conn(0x559ab399cef0 msgr2=0x559ab39a2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedbb7fe700 0 client.26383 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:43.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 shutdown_connections 2023-07-21T22:25:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.259+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3102598707 wait complete. 2023-07-21T22:25:43.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.260+0000 7fedf5d46b80 20 client.26383 trim_cache size 0 max 16384 2023-07-21T22:25:43.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.278+0000 7fedf5d46b80 1 Processor -- start 2023-07-21T22:25:43.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 -- start start 2023-07-21T22:25:43.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab39b1eb0 con 0x559ab39ac1b0 2023-07-21T22:25:43.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab39b2460 con 0x559ab39acac0 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab39abd30 con 0x559ab39a7930 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7feddbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7feddb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39538/0 (socket says 172.21.15.66:39538) 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7feddb7fe700 1 -- 172.21.15.66:0/811346235 learned_addr learned my addr 172.21.15.66:0/811346235 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.279+0000 7feddb7fe700 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddbfff700 1 -- 172.21.15.66:0/811346235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 msgr2=0x559ab39b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddbfff700 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddbfff700 1 -- 172.21.15.66:0/811346235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 msgr2=0x559ab39a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddbfff700 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39a22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddbfff700 1 -- 172.21.15.66:0/811346235 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab370fc60 con 0x559ab39ac1b0 2023-07-21T22:25:43.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddb7fe700 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:43.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7feddbfff700 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b37c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x559ab39a85b0 tx=0x7fedcc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f71e8f7e125a830 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7fedc2ffd700 1 -- 172.21.15.66:0/811346235 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01d040 con 0x559ab39ac1b0 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7fedc2ffd700 1 -- 172.21.15.66:0/811346235 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedcc003ac0 con 0x559ab39ac1b0 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.280+0000 7fedc2ffd700 1 -- 172.21.15.66:0/811346235 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc00b790 con 0x559ab39ac1b0 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 -- 172.21.15.66:0/811346235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 msgr2=0x559ab39b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b37c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x559ab39a85b0 tx=0x7fedcc000bc0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 -- 172.21.15.66:0/811346235 shutdown_connections 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39acac0 0x559ab39a22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39a7930 0x559ab39b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/811346235 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39ac1b0 0x559ab39b37c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 -- 172.21.15.66:0/811346235 >> 172.21.15.66:0/811346235 conn(0x559ab399cef0 msgr2=0x559ab39bccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 -- 172.21.15.66:0/811346235 shutdown_connections 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.281+0000 7fedf5d46b80 1 -- 172.21.15.66:0/811346235 wait complete. 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 -- start start 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 Processor -- start 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 0x559ab39b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39acac0 0x559ab3a0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab3a0e840 con 0x559ab39acac0 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab3a0e9b0 con 0x559ab39a7930 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab3a0eb70 con 0x559ab39ac1b0 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.282+0000 7fedf5d46b80 10 client.0 _ll_register_callbacks cb 0x7fedd0010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 0x559ab39b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 0x559ab39b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39542/0 (socket says 172.21.15.66:39542) 2023-07-21T22:25:43.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 -- 172.21.15.66:0/3401654145 learned_addr learned my addr 172.21.15.66:0/3401654145 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39acac0 0x559ab3a0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 -- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 msgr2=0x559ab39b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 -- 172.21.15.66:0/3401654145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39acac0 msgr2=0x559ab3a0e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39acac0 0x559ab3a0e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 -- 172.21.15.66:0/3401654145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc10f3b0 con 0x559ab39ac1b0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39acac0 0x559ab3a0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7feddbfff700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 0x559ab39b6c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fedcc003c50 tx=0x7fedcc005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7424eebd78d9baa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.283+0000 7fedbb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3401654145 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01d040 con 0x559ab39ac1b0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3401654145 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fedcc005190 con 0x559ab39ac1b0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedcc0037d0 con 0x559ab39ac1b0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3401654145 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedcc02d0d0 con 0x559ab39ac1b0 2023-07-21T22:25:43.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 10 client.26389 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:43.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x559ab3726de0 con 0x559ab39ac1b0 2023-07-21T22:25:43.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3401654145 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedcc0037d0 con 0x559ab39ac1b0 2023-07-21T22:25:43.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3401654145 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fedcc00b9f0 con 0x559ab39ac1b0 2023-07-21T22:25:43.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 10 client.26389 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:43.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 10 client.26389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:43.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 20 client.26389 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:43.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.284+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab3a15e80 0x559ab3a18290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7fedf5d46b80 4 client.26389 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:25:43.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ab38b4050 con 0x559ab3a15e80 2023-07-21T22:25:43.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab3a15e80 0x559ab3a18290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7fedbb7fe700 10 client.26389 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:43.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7feddb7fe700 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab3a15e80 0x559ab3a18290 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46f18c891cbe09f8 server_cookie=ccd8fbd53f9a2d2e in_seq=0 out_seq=0 2023-07-21T22:25:43.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7fedbb7fe700 1 -- 172.21.15.66:0/3401654145 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559ab38b4050 con 0x559ab3a15e80 2023-07-21T22:25:43.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7fedbb7fe700 10 client.26389 handle_command_reply: tid=0 2023-07-21T22:25:43.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.285+0000 7fedf5d46b80 1 client.26389 shutdown 2023-07-21T22:25:43.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 msgr2=0x559ab39b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 0x559ab39b6c50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fedcc003c50 tx=0x7fedcc005e80 comp rx=0 tx=0).stop 2023-07-21T22:25:43.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 shutdown_connections 2023-07-21T22:25:43.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ab39a7930 0x559ab39b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559ab3a15e80 0x559ab3a18290 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559ab39acac0 0x559ab3a0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 --2- 172.21.15.66:0/3401654145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559ab39ac1b0 0x559ab39b6c50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedbb7fe700 0 client.26389 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 >> 172.21.15.66:0/3401654145 conn(0x559ab399cef0 msgr2=0x559ab39a22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 shutdown_connections 2023-07-21T22:25:43.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 1 -- 172.21.15.66:0/3401654145 wait complete. 2023-07-21T22:25:43.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.286+0000 7fedf5d46b80 20 client.26389 trim_cache size 0 max 16384 2023-07-21T22:25:43.341 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:25:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:25:43.341 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:25:43.342 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:25:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fedc40a1340 msgr2=0x7fedc40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fedc40a1340 0x7fedc40a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fedd40959c0 tx=0x7fedd400b040 comp rx=0 tx=0).stop 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 msgr2=0x7feddc19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 0x7feddc19c800 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fedd4007ee0 tx=0x7fedd400d3b0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 shutdown_connections 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fedc40a1340 0x7fedc40a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feddc106fa0 0x7feddc19c800 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7feddc101f60 0x7feddc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 --2- 172.21.15.66:0/707627664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feddc102900 0x7feddc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 >> 172.21.15.66:0/707627664 conn(0x7feddc0fd7d0 msgr2=0x7feddc1058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 shutdown_connections 2023-07-21T22:25:43.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.287+0000 7fede23ff700 1 -- 172.21.15.66:0/707627664 wait complete. 2023-07-21T22:25:43.353 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:25:43.353 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:25:43.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 97335 3.89 MB/sec execute 70 sec latency 13.976 ms 2023-07-21T22:25:43.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 -- 172.21.15.66:0/3723627292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106340 msgr2=0x7f9f641067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 --2- 172.21.15.66:0/3723627292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106340 0x7f9f641067b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9f580098f0 tx=0x7f9f58009c00 comp rx=0 tx=0).stop 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 -- 172.21.15.66:0/3723627292 shutdown_connections 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 --2- 172.21.15.66:0/3723627292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106cf0 0x7f9f64112f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 --2- 172.21.15.66:0/3723627292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6410c1c0 0x7f9f6410c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 --2- 172.21.15.66:0/3723627292 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106340 0x7f9f641067b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 -- 172.21.15.66:0/3723627292 >> 172.21.15.66:0/3723627292 conn(0x7f9f640759e0 msgr2=0x7f9f64077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 -- 172.21.15.66:0/3723627292 shutdown_connections 2023-07-21T22:25:43.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.543+0000 7f9f68dff700 1 -- 172.21.15.66:0/3723627292 wait complete. 2023-07-21T22:25:43.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.544+0000 7f9f68dff700 1 Processor -- start 2023-07-21T22:25:43.592 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.546+0000 7f9f68dff700 1 -- start start 2023-07-21T22:25:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.547+0000 7f9f68dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 0x7f9f6419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.547+0000 7f9f68dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106cf0 0x7f9f6419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.547+0000 7f9f68dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6410c1c0 0x7f9f641a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.547+0000 7f9f68dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641a1170 con 0x7f9f64106cf0 2023-07-21T22:25:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.547+0000 7f9f68dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641a12e0 con 0x7f9f64106340 2023-07-21T22:25:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.547+0000 7f9f68dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641a14a0 con 0x7f9f6410c1c0 2023-07-21T22:25:43.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.548+0000 7f9f6259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 0x7f9f6419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.548+0000 7f9f6259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 0x7f9f6419c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56086/0 (socket says 172.21.15.66:56086) 2023-07-21T22:25:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.548+0000 7f9f6259c700 1 -- 172.21.15.66:0/846582146 learned_addr learned my addr 172.21.15.66:0/846582146 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.548+0000 7f9f62d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6410c1c0 0x7f9f641a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.548+0000 7f9f61d9b700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106cf0 0x7f9f6419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f6259c700 1 -- 172.21.15.66:0/846582146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6410c1c0 msgr2=0x7f9f641a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f6259c700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6410c1c0 0x7f9f641a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f6259c700 1 -- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106cf0 msgr2=0x7f9f6419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f6259c700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106cf0 0x7f9f6419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f6259c700 1 -- 172.21.15.66:0/846582146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f58009520 con 0x7f9f64106340 2023-07-21T22:25:43.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f6259c700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 0x7f9f6419c190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e9c0 tx=0x7f9f5400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.549+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400cbd0 con 0x7f9f64106340 2023-07-21T22:25:43.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.550+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f641a1780 con 0x7f9f64106340 2023-07-21T22:25:43.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.550+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f5400eeb0 con 0x7f9f64106340 2023-07-21T22:25:43.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.550+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54005580 con 0x7f9f64106340 2023-07-21T22:25:43.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.550+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641a8fc0 con 0x7f9f64106340 2023-07-21T22:25:43.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.552+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f64067750 con 0x7f9f64106340 2023-07-21T22:25:43.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.554+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9f54004750 con 0x7f9f64106340 2023-07-21T22:25:43.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.555+0000 7f9f537fe700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9f4c0a1380 0x7f9f4c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.556+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9f54014070 con 0x7f9f64106340 2023-07-21T22:25:43.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.556+0000 7f9f61d9b700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9f4c0a1380 0x7f9f4c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:43.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.557+0000 7f9f61d9b700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9f4c0a1380 0x7f9f4c0a3830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9f6419d7a0 tx=0x7f9f58005b80 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:43.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.560+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9f54096420 con 0x7f9f64106340 2023-07-21T22:25:43.912 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.866+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f641a9310 con 0x7f9f64106340 2023-07-21T22:25:43.913 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.867+0000 7f9f537fe700 1 -- 172.21.15.66:0/846582146 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10823 (secure 0 0 0) 0x7f9f54099a40 con 0x7f9f64106340 2023-07-21T22:25:43.914 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:25:43.914 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:25:12.909399+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:25:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9f4c0a1380 msgr2=0x7f9f4c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9f4c0a1380 0x7f9f4c0a3830 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9f6419d7a0 tx=0x7f9f58005b80 comp rx=0 tx=0).stop 2023-07-21T22:25:43.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 msgr2=0x7f9f6419c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 0x7f9f6419c190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e9c0 tx=0x7f9f5400ecd0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 shutdown_connections 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f64106340 0x7f9f6419c190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9f4c0a1380 0x7f9f4c0a3830 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f6410c1c0 0x7f9f641a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 --2- 172.21.15.66:0/846582146 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9f64106cf0 0x7f9f6419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:43.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 >> 172.21.15.66:0/846582146 conn(0x7f9f640759e0 msgr2=0x7f9f64076f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 shutdown_connections 2023-07-21T22:25:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:43.870+0000 7f9f68dff700 1 -- 172.21.15.66:0/846582146 wait complete. 2023-07-21T22:25:43.918 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 32 2023-07-21T22:25:43.936 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:25:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 -- 172.21.15.66:0/1838384891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0fe820 msgr2=0x7f4e5c0febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 --2- 172.21.15.66:0/1838384891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c0febf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009990 tx=0x7f4e4c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 -- 172.21.15.66:0/1838384891 shutdown_connections 2023-07-21T22:25:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 --2- 172.21.15.66:0/1838384891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 0x7f4e5c10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 --2- 172.21.15.66:0/1838384891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c0febf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 --2- 172.21.15.66:0/1838384891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0ff1c0 0x7f4e5c10be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 -- 172.21.15.66:0/1838384891 >> 172.21.15.66:0/1838384891 conn(0x7f4e5c0fa4b0 msgr2=0x7f4e5c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 -- 172.21.15.66:0/1838384891 shutdown_connections 2023-07-21T22:25:44.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.111+0000 7f4e627ff700 1 -- 172.21.15.66:0/1838384891 wait complete. 2023-07-21T22:25:44.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.112+0000 7f4e627ff700 1 Processor -- start 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 -- start start 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0ff1c0 0x7f4e5c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 0x7f4e5c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19ce20 con 0x7f4e5c0fe820 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19cf90 con 0x7f4e5c10c3d0 2023-07-21T22:25:44.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e627ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19d150 con 0x7f4e5c0ff1c0 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47682/0 (socket says 172.21.15.66:47682) 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3421583088 learned_addr learned my addr 172.21.15.66:0/3421583088 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 0x7f4e5c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.113+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0ff1c0 0x7f4e5c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3421583088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0ff1c0 msgr2=0x7f4e5c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0ff1c0 0x7f4e5c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3421583088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 msgr2=0x7f4e5c19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 0x7f4e5c19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3421583088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0095f0 con 0x7f4e5c0fe820 2023-07-21T22:25:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 0x7f4e5c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c197e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005910 tx=0x7f4e4c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e597fa700 1 -- 172.21.15.66:0/3421583088 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c01d070 con 0x7f4e5c0fe820 2023-07-21T22:25:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.114+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c19d3d0 con 0x7f4e5c0fe820 2023-07-21T22:25:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.115+0000 7f4e597fa700 1 -- 172.21.15.66:0/3421583088 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c022470 con 0x7f4e5c0fe820 2023-07-21T22:25:44.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.115+0000 7f4e597fa700 1 -- 172.21.15.66:0/3421583088 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c017650 con 0x7f4e5c0fe820 2023-07-21T22:25:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.115+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1a4c70 con 0x7f4e5c0fe820 2023-07-21T22:25:44.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.119+0000 7f4e597fa700 1 -- 172.21.15.66:0/3421583088 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e4c00fad0 con 0x7f4e5c0fe820 2023-07-21T22:25:44.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.120+0000 7f4e76171b80 1 Processor -- start 2023-07-21T22:25:44.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.120+0000 7f4e76171b80 1 -- start start 2023-07-21T22:25:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.120+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.120+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6a7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e76171b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca5accb0 con 0x55d3ca6a7930 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e76171b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca5ffa40 con 0x55d3ca6acac0 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e76171b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca5f8ed0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39602/0 (socket says 172.21.15.66:39602) 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 -- 172.21.15.66:0/2494586402 learned_addr learned my addr 172.21.15.66:0/2494586402 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 -- 172.21.15.66:0/2494586402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 msgr2=0x55d3ca6a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e597fa700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e440a12c0 0x7f4e440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 -- 172.21.15.66:0/2494586402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 msgr2=0x55d3ca6a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e60d9c700 1 -- 172.21.15.66:0/2494586402 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3ca426de0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.121+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.122+0000 7f4e597fa700 1 -- 172.21.15.66:0/3421583088 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e4c0ce4c0 con 0x7f4e5c0fe820 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.122+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e440a12c0 0x7f4e440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.122+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6ac580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e54003e30 tx=0x7f4e54009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c3b5818873e6c61 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.122+0000 7f4e427fc700 1 -- 172.21.15.66:0/2494586402 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54007e70 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.122+0000 7f4e427fc700 1 -- 172.21.15.66:0/2494586402 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e5400f040 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.122+0000 7f4e427fc700 1 -- 172.21.15.66:0/2494586402 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540157d0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e440a12c0 0x7f4e440a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e54020000 tx=0x7f4e54023000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 -- 172.21.15.66:0/2494586402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 msgr2=0x55d3ca6ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6ac580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e54003e30 tx=0x7f4e54009920 comp rx=0 tx=0).stop 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 -- 172.21.15.66:0/2494586402 shutdown_connections 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6ac580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2494586402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 -- 172.21.15.66:0/2494586402 >> 172.21.15.66:0/2494586402 conn(0x55d3ca69cef0 msgr2=0x55d3ca6a2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 -- 172.21.15.66:0/2494586402 shutdown_connections 2023-07-21T22:25:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.123+0000 7f4e76171b80 1 -- 172.21.15.66:0/2494586402 wait complete. 2023-07-21T22:25:44.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.124+0000 7f4e76171b80 1 -- start start 2023-07-21T22:25:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.124+0000 7f4e76171b80 1 Processor -- start 2023-07-21T22:25:44.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca598ac0 con 0x55d3ca6a7930 2023-07-21T22:25:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca5987c0 con 0x55d3ca6acac0 2023-07-21T22:25:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca47c920 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e76171b80 10 client.0 _ll_register_callbacks cb 0x55d3ca596760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e60d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39618/0 (socket says 172.21.15.66:39618) 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.125+0000 7f4e5b7fe700 1 -- 172.21.15.66:0/3254185130 learned_addr learned my addr 172.21.15.66:0/3254185130 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5b7fe700 1 -- 172.21.15.66:0/3254185130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 msgr2=0x55d3ca6b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5b7fe700 1 -- 172.21.15.66:0/3254185130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 msgr2=0x55d3ca6b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5b7fe700 1 -- 172.21.15.66:0/3254185130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e54003ae0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0e70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b9c0 tx=0x7f4e5000bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=126dbf48fc9bfaa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3254185130 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50003d70 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3254185130 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e50004830 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.126+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3ca5524f0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3254185130 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e500049f0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 10 client.26407 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d3ca40fc60 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3254185130 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3ca5524f0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3254185130 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4e5001a5e0 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 10 client.26407 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:44.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 10 client.26407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 20 client.26407 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca6c1460 0x55d3ca6c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 4 client.26407 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3ca5f1eb0 con 0x55d3ca6c1460 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.127+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca6c1460 0x55d3ca6c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.128+0000 7f4e3b7fe700 10 client.26407 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:44.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.128+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca6c1460 0x55d3ca6c3870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=947f3a1f87874f78 server_cookie=9d72de5acaff445a in_seq=0 out_seq=0 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3254185130 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d3ca5f1eb0 con 0x55d3ca6c1460 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e3b7fe700 10 client.26407 handle_command_reply: tid=0 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 client.26407 shutdown 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 msgr2=0x55d3ca6b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0e70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b9c0 tx=0x7f4e5000bcd0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 shutdown_connections 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca6c1460 0x55d3ca6c3870 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3254185130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0e70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e3b7fe700 0 client.26407 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 >> 172.21.15.66:0/3254185130 conn(0x55d3ca69cef0 msgr2=0x55d3ca6a2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 shutdown_connections 2023-07-21T22:25:44.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.129+0000 7f4e76171b80 1 -- 172.21.15.66:0/3254185130 wait complete. 2023-07-21T22:25:44.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.130+0000 7f4e76171b80 20 client.26407 trim_cache size 0 max 16384 2023-07-21T22:25:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 Processor -- start 2023-07-21T22:25:44.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 -- start start 2023-07-21T22:25:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca6aeea0 con 0x55d3ca6acac0 2023-07-21T22:25:44.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca6b1190 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.144+0000 7f4e76171b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca699fc0 con 0x55d3ca6a7930 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39626/0 (socket says 172.21.15.66:39626) 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 -- 172.21.15.66:0/2940163078 learned_addr learned my addr 172.21.15.66:0/2940163078 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e60d9c700 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 -- 172.21.15.66:0/2940163078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6ac1b0 msgr2=0x55d3ca6b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 -- 172.21.15.66:0/2940163078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6acac0 msgr2=0x55d3ca6b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 -- 172.21.15.66:0/2940163078 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3ca44d250 con 0x55d3ca6a7930 2023-07-21T22:25:44.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000c00 tx=0x7f4e4c0a1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77651bfc408e0f34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e42ffd700 1 -- 172.21.15.66:0/2940163078 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c09f120 con 0x55d3ca6a7930 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.145+0000 7f4e42ffd700 1 -- 172.21.15.66:0/2940163078 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c097610 con 0x55d3ca6a7930 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e42ffd700 1 -- 172.21.15.66:0/2940163078 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c09f300 con 0x55d3ca6a7930 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 -- 172.21.15.66:0/2940163078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 msgr2=0x55d3ca6b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000c00 tx=0x7f4e4c0a1040 comp rx=0 tx=0).stop 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 -- 172.21.15.66:0/2940163078 shutdown_connections 2023-07-21T22:25:44.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b3720 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 --2- 172.21.15.66:0/2940163078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 -- 172.21.15.66:0/2940163078 >> 172.21.15.66:0/2940163078 conn(0x55d3ca69cef0 msgr2=0x55d3ca6a5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 -- 172.21.15.66:0/2940163078 shutdown_connections 2023-07-21T22:25:44.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.146+0000 7f4e76171b80 1 -- 172.21.15.66:0/2940163078 wait complete. 2023-07-21T22:25:44.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.147+0000 7f4e76171b80 1 -- start start 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.147+0000 7f4e76171b80 1 Processor -- start 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.147+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e76171b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e76171b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca6b6e20 con 0x55d3ca6ac1b0 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e76171b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca6b6f90 con 0x55d3ca6acac0 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e76171b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3ca6b7150 con 0x55d3ca6a7930 2023-07-21T22:25:44.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e76171b80 10 client.0 _ll_register_callbacks cb 0x7f4e50004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39628/0 (socket says 172.21.15.66:39628) 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e60d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3369172830 learned_addr learned my addr 172.21.15.66:0/3369172830 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3369172830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 msgr2=0x55d3ca6b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3369172830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6ac1b0 msgr2=0x55d3ca6b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 -- 172.21.15.66:0/3369172830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1a4c70 con 0x55d3ca6a7930 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e5bfff700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b8230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c097050 tx=0x7f4e4c00b490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a76510391aa8f15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.148+0000 7f4e3b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3369172830 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c09f120 con 0x55d3ca6a7930 2023-07-21T22:25:44.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3369172830 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e4c0a3690 con 0x55d3ca6a7930 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e5400f040 con 0x55d3ca6a7930 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3369172830 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e4c09b0a0 con 0x55d3ca6a7930 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 10 client.26413 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d3ca4252b0 con 0x55d3ca6a7930 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3369172830 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e5400f040 con 0x55d3ca6a7930 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3369172830 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4e4c0ce110 con 0x55d3ca6a7930 2023-07-21T22:25:44.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 10 client.26413 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 10 client.26413 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:25:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 20 client.26413 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca71e9c0 0x55d3ca720dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 4 client.26413 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:25:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.149+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3ca5b4050 con 0x55d3ca71e9c0 2023-07-21T22:25:44.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.150+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca71e9c0 0x55d3ca720dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.203 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:43 smithi066 ceph-mon[126379]: pgmap v130: 129 pgs: 129 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2023-07-21T22:25:44.203 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:43 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/846582146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:25:44.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.150+0000 7f4e5b7fe700 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca71e9c0 0x55d3ca720dd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bedf96b516c341f2 server_cookie=804b91b5416a0c76 in_seq=0 out_seq=0 2023-07-21T22:25:44.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.150+0000 7f4e3b7fe700 10 client.26413 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:44.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.150+0000 7f4e3b7fe700 1 -- 172.21.15.66:0/3369172830 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3ca5b4050 con 0x55d3ca71e9c0 2023-07-21T22:25:44.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.150+0000 7f4e3b7fe700 10 client.26413 handle_command_reply: tid=0 2023-07-21T22:25:44.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.150+0000 7f4e76171b80 1 client.26413 shutdown 2023-07-21T22:25:44.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 msgr2=0x55d3ca6b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b8230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c097050 tx=0x7f4e4c00b490 comp rx=0 tx=0).stop 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 shutdown_connections 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d3ca71e9c0 0x55d3ca720dd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3ca6acac0 0x55d3ca6b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d3ca6a7930 0x55d3ca6b8230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 --2- 172.21.15.66:0/3369172830 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d3ca6ac1b0 0x55d3ca6b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e3b7fe700 0 client.26413 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:25:44.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 >> 172.21.15.66:0/3369172830 conn(0x55d3ca69cef0 msgr2=0x55d3ca6b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 shutdown_connections 2023-07-21T22:25:44.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 1 -- 172.21.15.66:0/3369172830 wait complete. 2023-07-21T22:25:44.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e76171b80 20 client.26413 trim_cache size 0 max 16384 2023-07-21T22:25:44.206 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:25:44.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e440a12c0 msgr2=0x7f4e440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e627ff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e440a12c0 0x7f4e440a3770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4e54020000 tx=0x7f4e54023000 comp rx=0 tx=0).stop 2023-07-21T22:25:44.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 msgr2=0x7f4e5c197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.151+0000 7f4e627ff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c197e40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005910 tx=0x7f4e4c005740 comp rx=0 tx=0).stop 2023-07-21T22:25:44.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 shutdown_connections 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e5c10c3d0 0x7f4e5c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e440a12c0 0x7f4e440a3770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e5c0fe820 0x7f4e5c197e40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 --2- 172.21.15.66:0/3421583088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e5c0ff1c0 0x7f4e5c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 >> 172.21.15.66:0/3421583088 conn(0x7f4e5c0fa4b0 msgr2=0x7f4e5c10d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 shutdown_connections 2023-07-21T22:25:44.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.152+0000 7f4e627ff700 1 -- 172.21.15.66:0/3421583088 wait complete. 2023-07-21T22:25:44.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:25:44.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:43 smithi104 ceph-mon[129021]: pgmap v130: 129 pgs: 129 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2023-07-21T22:25:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:43 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/846582146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:25:44.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 -- 172.21.15.66:0/2656950908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c0ffad0 msgr2=0x7f074c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 --2- 172.21.15.66:0/2656950908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c0ffad0 0x7f074c0fff40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0740009990 tx=0x7f0740009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 -- 172.21.15.66:0/2656950908 shutdown_connections 2023-07-21T22:25:44.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 --2- 172.21.15.66:0/2656950908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c105ae0 0x7f074c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 --2- 172.21.15.66:0/2656950908 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c100480 0x7f074c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 --2- 172.21.15.66:0/2656950908 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c0ffad0 0x7f074c0fff40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 -- 172.21.15.66:0/2656950908 >> 172.21.15.66:0/2656950908 conn(0x7f074c0fb5f0 msgr2=0x7f074c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 -- 172.21.15.66:0/2656950908 shutdown_connections 2023-07-21T22:25:44.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.389+0000 7f0751bff700 1 -- 172.21.15.66:0/2656950908 wait complete. 2023-07-21T22:25:44.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.391+0000 7f0751bff700 1 Processor -- start 2023-07-21T22:25:44.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.391+0000 7f0751bff700 1 -- start start 2023-07-21T22:25:44.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f0751bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c0ffad0 0x7f074c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f0751bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 0x7f074c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f0751bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c105ae0 0x7f074c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f0751bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c19ce00 con 0x7f074c0ffad0 2023-07-21T22:25:44.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f0751bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c19cf70 con 0x7f074c100480 2023-07-21T22:25:44.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f0751bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c19d130 con 0x7f074c105ae0 2023-07-21T22:25:44.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 0x7f074c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c0ffad0 0x7f074c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 0x7f074c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56154/0 (socket says 172.21.15.66:56154) 2023-07-21T22:25:44.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 -- 172.21.15.66:0/3401759145 learned_addr learned my addr 172.21.15.66:0/3401759145 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074bfff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c105ae0 0x7f074c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 -- 172.21.15.66:0/3401759145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c105ae0 msgr2=0x7f074c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c105ae0 0x7f074c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 -- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c0ffad0 msgr2=0x7f074c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c0ffad0 0x7f074c197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074affd700 1 -- 172.21.15.66:0/3401759145 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400095f0 con 0x7f074c100480 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074b7fe700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c0ffad0 0x7f074c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.392+0000 7f074bfff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c105ae0 0x7f074c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.393+0000 7f074affd700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 0x7f074c198360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0740005910 tx=0x7f0740004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.393+0000 7f0748ff9700 1 -- 172.21.15.66:0/3401759145 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074001d070 con 0x7f074c100480 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.393+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c19d3b0 con 0x7f074c100480 2023-07-21T22:25:44.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.393+0000 7f0748ff9700 1 -- 172.21.15.66:0/3401759145 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0740022470 con 0x7f074c100480 2023-07-21T22:25:44.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.393+0000 7f0748ff9700 1 -- 172.21.15.66:0/3401759145 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740017650 con 0x7f074c100480 2023-07-21T22:25:44.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.393+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c1a4c50 con 0x7f074c100480 2023-07-21T22:25:44.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.396+0000 7f0748ff9700 1 -- 172.21.15.66:0/3401759145 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f074000fad0 con 0x7f074c100480 2023-07-21T22:25:44.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.398+0000 7f0765476b80 1 Processor -- start 2023-07-21T22:25:44.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.398+0000 7f0765476b80 1 -- start start 2023-07-21T22:25:44.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.398+0000 7f0748ff9700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07340a12c0 0x7f07340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.398+0000 7f0748ff9700 1 -- 172.21.15.66:0/3401759145 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f07400ce4d0 con 0x7f074c100480 2023-07-21T22:25:44.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.398+0000 7f074bfff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07340a12c0 0x7f07340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f0765476b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x564857947580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f0765476b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x564857942350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f0765476b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857942da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f0765476b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564857847cb0 con 0x5648579471b0 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f0765476b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485789aa40 con 0x564857947ac0 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f0765476b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564857893ed0 con 0x564857942930 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x564857947580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x564857947580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47742/0 (socket says 172.21.15.66:47742) 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074b7fe700 1 -- 172.21.15.66:0/2338827209 learned_addr learned my addr 172.21.15.66:0/2338827209 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074affd700 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x564857942350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074bfff700 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857942da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074affd700 1 -- 172.21.15.66:0/2338827209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 msgr2=0x564857942da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074affd700 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857942da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074affd700 1 -- 172.21.15.66:0/2338827209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 msgr2=0x564857947580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074affd700 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x564857947580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.399+0000 7f074affd700 1 -- 172.21.15.66:0/2338827209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648576c1de0 con 0x564857947ac0 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.400+0000 7f074bfff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07340a12c0 0x7f07340a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07440076f0 tx=0x7f0744009130 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.400+0000 7f074affd700 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x564857942350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07400cb8d0 tx=0x7f074009b1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27f79894c0a6c4b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.400+0000 7f0731ffb700 1 -- 172.21.15.66:0/2338827209 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074009b2e0 con 0x564857947ac0 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.400+0000 7f0731ffb700 1 -- 172.21.15.66:0/2338827209 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074009f310 con 0x564857947ac0 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.400+0000 7f0731ffb700 1 -- 172.21.15.66:0/2338827209 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074009f020 con 0x564857947ac0 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 -- 172.21.15.66:0/2338827209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 msgr2=0x564857942350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x564857942350 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f07400cb8d0 tx=0x7f074009b1a0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 -- 172.21.15.66:0/2338827209 shutdown_connections 2023-07-21T22:25:44.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857942da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x564857947580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 --2- 172.21.15.66:0/2338827209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x564857942350 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 -- 172.21.15.66:0/2338827209 >> 172.21.15.66:0/2338827209 conn(0x564857937ef0 msgr2=0x56485793d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 -- 172.21.15.66:0/2338827209 shutdown_connections 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 -- 172.21.15.66:0/2338827209 wait complete. 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.401+0000 7f0765476b80 1 -- start start 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 Processor -- start 2023-07-21T22:25:44.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 0x56485794ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648579471b0 0x56485794bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x56485794c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564857833ac0 con 0x564857947ac0 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648578337c0 con 0x564857942930 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564857717920 con 0x5648579471b0 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f0765476b80 10 client.0 _ll_register_callbacks cb 0x564857831760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:44.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 0x56485794ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f074affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648579471b0 0x56485794bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 0x56485794ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56172/0 (socket says 172.21.15.66:56172) 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f074b7fe700 1 -- 172.21.15.66:0/2703471664 learned_addr learned my addr 172.21.15.66:0/2703471664 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.402+0000 7f074bfff700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x56485794c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074b7fe700 1 -- 172.21.15.66:0/2703471664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648579471b0 msgr2=0x56485794bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074b7fe700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648579471b0 0x56485794bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074b7fe700 1 -- 172.21.15.66:0/2703471664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 msgr2=0x56485794c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074b7fe700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x56485794c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074b7fe700 1 -- 172.21.15.66:0/2703471664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c1a4c50 con 0x564857942930 2023-07-21T22:25:44.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074affd700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648579471b0 0x56485794bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f074b7fe700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 0x56485794ebd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f073c011ce0 tx=0x7f073c009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a217fc67211a5e3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f072affd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f072affd700 1 -- 172.21.15.66:0/2703471664 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c016070 con 0x564857942930 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f072affd700 1 -- 172.21.15.66:0/2703471664 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f073c00fb00 con 0x564857942930 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.403+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648577ed4f0 con 0x564857942930 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f072affd700 1 -- 172.21.15.66:0/2703471664 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f073c00fcc0 con 0x564857942930 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 10 client.26450 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:44.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5648576aac60 con 0x564857942930 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f072affd700 1 -- 172.21.15.66:0/2703471664 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648577ed4f0 con 0x564857942930 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f072affd700 1 -- 172.21.15.66:0/2703471664 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f073c019390 con 0x564857942930 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 10 client.26450 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 10 client.26450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 20 client.26450 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56485795c340 0x56485795e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 4 client.26450 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56485788ceb0 con 0x56485795c340 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.404+0000 7f074affd700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56485795c340 0x56485795e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.405+0000 7f072affd700 10 client.26450 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.405+0000 7f074affd700 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56485795c340 0x56485795e750 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79fd1177c31c0c4 server_cookie=219989e347f1aa61 in_seq=0 out_seq=0 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.405+0000 7f072affd700 1 -- 172.21.15.66:0/2703471664 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56485788ceb0 con 0x56485795c340 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f072affd700 10 client.26450 handle_command_reply: tid=0 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 client.26450 shutdown 2023-07-21T22:25:44.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 msgr2=0x56485794ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 0x56485794ebd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f073c011ce0 tx=0x7f073c009ea0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 shutdown_connections 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56485795c340 0x56485795e750 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857942930 0x56485794ebd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x56485794c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 --2- 172.21.15.66:0/2703471664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648579471b0 0x56485794bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f072affd700 0 client.26450 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:44.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 >> 172.21.15.66:0/2703471664 conn(0x564857937ef0 msgr2=0x56485793d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 shutdown_connections 2023-07-21T22:25:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 1 -- 172.21.15.66:0/2703471664 wait complete. 2023-07-21T22:25:44.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.406+0000 7f0765476b80 20 client.26450 trim_cache size 0 max 16384 2023-07-21T22:25:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 Processor -- start 2023-07-21T22:25:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 -- start start 2023-07-21T22:25:44.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857953380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648579471b0 0x56485794e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x5648579418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648579379b0 con 0x564857947ac0 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648579436d0 con 0x5648579471b0 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.421+0000 7f0765476b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485793a010 con 0x564857942930 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857953380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648579471b0 0x56485794e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074bfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x5648579418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857953380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39692/0 (socket says 172.21.15.66:39692) 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 -- 172.21.15.66:0/977205234 learned_addr learned my addr 172.21.15.66:0/977205234 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 -- 172.21.15.66:0/977205234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648579471b0 msgr2=0x56485794e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648579471b0 0x56485794e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 -- 172.21.15.66:0/977205234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 msgr2=0x5648579418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x5648579418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 -- 172.21.15.66:0/977205234 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648576e8250 con 0x564857942930 2023-07-21T22:25:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074affd700 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648579471b0 0x56485794e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074bfff700 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x5648579418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f074b7fe700 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857953380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f073c0097c0 tx=0x7f073c009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b03efccd2dffee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.422+0000 7f07327fc700 1 -- 172.21.15.66:0/977205234 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c016070 con 0x564857942930 2023-07-21T22:25:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f07327fc700 1 -- 172.21.15.66:0/977205234 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f073c019910 con 0x564857942930 2023-07-21T22:25:44.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f07327fc700 1 -- 172.21.15.66:0/977205234 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c004c20 con 0x564857942930 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 -- 172.21.15.66:0/977205234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 msgr2=0x564857953380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857953380 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f073c0097c0 tx=0x7f073c009400 comp rx=0 tx=0).stop 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 -- 172.21.15.66:0/977205234 shutdown_connections 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564857947ac0 0x5648579418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x564857953380 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 --2- 172.21.15.66:0/977205234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648579471b0 0x56485794e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 -- 172.21.15.66:0/977205234 >> 172.21.15.66:0/977205234 conn(0x564857937ef0 msgr2=0x564857940920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 -- 172.21.15.66:0/977205234 shutdown_connections 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.423+0000 7f0765476b80 1 -- 172.21.15.66:0/977205234 wait complete. 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.424+0000 7f0765476b80 1 -- start start 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.424+0000 7f0765476b80 1 Processor -- start 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x56485794b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x56485794bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x56485794a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485794a880 con 0x5648579471b0 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485794a9f0 con 0x564857947ac0 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56485794abb0 con 0x564857942930 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f0765476b80 10 client.0 _ll_register_callbacks cb 0x7f073c0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f074affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x56485794bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f074bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x56485794a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f074b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x56485794b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f074affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x56485794bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:47764/0 (socket says 172.21.15.66:47764) 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.425+0000 7f074affd700 1 -- 172.21.15.66:0/329876913 learned_addr learned my addr 172.21.15.66:0/329876913 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074affd700 1 -- 172.21.15.66:0/329876913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 msgr2=0x56485794b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074affd700 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x56485794b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074affd700 1 -- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 msgr2=0x56485794a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074affd700 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x56485794a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074affd700 1 -- 172.21.15.66:0/329876913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648576c1de0 con 0x5648579471b0 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074bfff700 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x56485794a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f074affd700 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x56485794bcc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f07400cbe50 tx=0x7f074000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652cfdf8c4f9435b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f072affd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f072affd700 1 -- 172.21.15.66:0/329876913 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074009b2e0 con 0x5648579471b0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f074009f310 con 0x5648579471b0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f072affd700 1 -- 172.21.15.66:0/329876913 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0740099680 con 0x5648579471b0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f072affd700 1 -- 172.21.15.66:0/329876913 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07400a10d0 con 0x5648579471b0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f0765476b80 10 client.16401 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.426+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f07440171d0 con 0x5648579471b0 2023-07-21T22:25:44.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f072affd700 1 -- 172.21.15.66:0/329876913 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f074009f310 con 0x5648579471b0 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f072affd700 1 -- 172.21.15.66:0/329876913 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f074009b540 con 0x5648579471b0 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f0765476b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f0765476b80 10 client.16401 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f0765476b80 20 client.16401 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f0765476b80 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5648579b0c70 0x5648579b3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f0765476b80 4 client.16401 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56485784f050 con 0x5648579b0c70 2023-07-21T22:25:44.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.427+0000 7f074b7fe700 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5648579b0c70 0x5648579b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f072affd700 10 client.16401 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f074b7fe700 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5648579b0c70 0x5648579b3050 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df25954b3c463fb5 server_cookie=d8149ff503d409f4 in_seq=0 out_seq=0 2023-07-21T22:25:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f072affd700 1 -- 172.21.15.66:0/329876913 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56485784f050 con 0x5648579b0c70 2023-07-21T22:25:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f072affd700 10 client.16401 handle_command_reply: tid=0 2023-07-21T22:25:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 client.16401 shutdown 2023-07-21T22:25:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 msgr2=0x56485794bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x56485794bcc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f07400cbe50 tx=0x7f074000f840 comp rx=0 tx=0).stop 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 shutdown_connections 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5648579b0c70 0x5648579b3050 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564857947ac0 0x56485794a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f072affd700 0 client.16401 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564857942930 0x56485794b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 --2- 172.21.15.66:0/329876913 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648579471b0 0x56485794bcc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 >> 172.21.15.66:0/329876913 conn(0x564857937ef0 msgr2=0x564857947e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 shutdown_connections 2023-07-21T22:25:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.428+0000 7f0765476b80 1 -- 172.21.15.66:0/329876913 wait complete. 2023-07-21T22:25:44.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0765476b80 20 client.16401 trim_cache size 0 max 16384 2023-07-21T22:25:44.483 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:25:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07340a12c0 msgr2=0x7f07340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07340a12c0 0x7f07340a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f07440076f0 tx=0x7f0744009130 comp rx=0 tx=0).stop 2023-07-21T22:25:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 msgr2=0x7f074c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 0x7f074c198360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0740005910 tx=0x7f0740004730 comp rx=0 tx=0).stop 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 shutdown_connections 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f07340a12c0 0x7f07340a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f074c105ae0 0x7f074c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f074c0ffad0 0x7f074c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 --2- 172.21.15.66:0/3401759145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f074c100480 0x7f074c198360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 >> 172.21.15.66:0/3401759145 conn(0x7f074c0fb5f0 msgr2=0x7f074c10ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.429+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 shutdown_connections 2023-07-21T22:25:44.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.430+0000 7f0751bff700 1 -- 172.21.15.66:0/3401759145 wait complete. 2023-07-21T22:25:44.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:43 smithi149 ceph-mon[129268]: pgmap v130: 129 pgs: 129 active+clean; 75 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2023-07-21T22:25:44.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:43 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/846582146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:25:44.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:25:44.526 INFO:tasks.workunit.client.0.smithi066.stdout: 1 99585 4.03 MB/sec execute 71 sec latency 30.281 ms 2023-07-21T22:25:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 -- 172.21.15.66:0/4280251230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 msgr2=0x7f5ae81010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 --2- 172.21.15.66:0/4280251230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae81010f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009990 tx=0x7f5ae4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 -- 172.21.15.66:0/4280251230 shutdown_connections 2023-07-21T22:25:44.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 --2- 172.21.15.66:0/4280251230 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae810a060 0x7f5ae810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 --2- 172.21.15.66:0/4280251230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae81010f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 --2- 172.21.15.66:0/4280251230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae8101630 0x7f5ae8109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 -- 172.21.15.66:0/4280251230 >> 172.21.15.66:0/4280251230 conn(0x7f5ae80fa4b0 msgr2=0x7f5ae80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 -- 172.21.15.66:0/4280251230 shutdown_connections 2023-07-21T22:25:44.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.686+0000 7f5aefbff700 1 -- 172.21.15.66:0/4280251230 wait complete. 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 Processor -- start 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 -- start start 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae80fe970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae8101630 0x7f5ae80feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae810a060 0x7f5ae81a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80ff6e0 con 0x7f5ae8101630 2023-07-21T22:25:44.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81a43a0 con 0x7f5ae810a060 2023-07-21T22:25:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.687+0000 7f5aefbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81a44e0 con 0x7f5ae8100d20 2023-07-21T22:25:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae80fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae810a060 0x7f5ae81a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae80fe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39702/0 (socket says 172.21.15.66:39702) 2023-07-21T22:25:44.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aed99b700 1 -- 172.21.15.66:0/1353241373 learned_addr learned my addr 172.21.15.66:0/1353241373 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae8101630 0x7f5ae80feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 -- 172.21.15.66:0/1353241373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 msgr2=0x7f5ae80fe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae80fe970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 -- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae8101630 msgr2=0x7f5ae80feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae8101630 0x7f5ae80feeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 -- 172.21.15.66:0/1353241373 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40095f0 con 0x7f5ae810a060 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae80fe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae8101630 0x7f5ae80feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.688+0000 7f5aee19c700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae810a060 0x7f5ae81a3e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000efc0 tx=0x7f5ae000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.689+0000 7f5adeffd700 1 -- 172.21.15.66:0/1353241373 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0018070 con 0x7f5ae810a060 2023-07-21T22:25:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.689+0000 7f5adeffd700 1 -- 172.21.15.66:0/1353241373 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae000f040 con 0x7f5ae810a060 2023-07-21T22:25:44.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.689+0000 7f5adeffd700 1 -- 172.21.15.66:0/1353241373 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000cdf0 con 0x7f5ae810a060 2023-07-21T22:25:44.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.689+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae81a46e0 con 0x7f5ae810a060 2023-07-21T22:25:44.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.689+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81a4b70 con 0x7f5ae810a060 2023-07-21T22:25:44.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.692+0000 7f5adeffd700 1 -- 172.21.15.66:0/1353241373 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5ae0007500 con 0x7f5ae810a060 2023-07-21T22:25:44.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 Processor -- start 2023-07-21T22:25:44.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5adeffd700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5ad40a12c0 0x7f5ad40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 -- start start 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5adeffd700 1 -- 172.21.15.66:0/1353241373 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ae00ccc70 con 0x7f5ae810a060 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5ad40a12c0 0x7f5ad40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 0x555815c89580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c84930 0x555815c84da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815b89cb0 con 0x555815c84930 2023-07-21T22:25:44.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815bdca40 con 0x555815c89ac0 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.694+0000 7f5b0349db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815bd5ed0 con 0x555815c891b0 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 0x555815c89580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 0x555815c89580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39710/0 (socket says 172.21.15.66:39710) 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 -- 172.21.15.66:0/1392915790 learned_addr learned my addr 172.21.15.66:0/1392915790 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aee19c700 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c84930 0x555815c84da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 -- 172.21.15.66:0/1392915790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 msgr2=0x555815c84350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c84350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 -- 172.21.15.66:0/1392915790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c84930 msgr2=0x555815c84da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c84930 0x555815c84da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed99b700 1 -- 172.21.15.66:0/1392915790 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555815a03de0 con 0x555815c891b0 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c84350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.695+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5ad40a12c0 0x7f5ad40a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8100160 tx=0x7f5ad80138a0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.696+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 0x555815c89580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005f30 tx=0x7f5ae4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f6db504f959ad66 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.696+0000 7f5ad3fff700 1 -- 172.21.15.66:0/1392915790 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4027070 con 0x555815c891b0 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.696+0000 7f5ad3fff700 1 -- 172.21.15.66:0/1392915790 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae401e040 con 0x555815c891b0 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.696+0000 7f5ad3fff700 1 -- 172.21.15.66:0/1392915790 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae401d050 con 0x555815c891b0 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.696+0000 7f5b0349db80 1 -- 172.21.15.66:0/1392915790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 msgr2=0x555815c89580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.696+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 0x555815c89580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005f30 tx=0x7f5ae4005c00 comp rx=0 tx=0).stop 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 -- 172.21.15.66:0/1392915790 shutdown_connections 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c84930 0x555815c84da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c891b0 0x555815c89580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1392915790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c84350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 -- 172.21.15.66:0/1392915790 >> 172.21.15.66:0/1392915790 conn(0x555815c79ef0 msgr2=0x555815c7f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 -- 172.21.15.66:0/1392915790 shutdown_connections 2023-07-21T22:25:44.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 -- 172.21.15.66:0/1392915790 wait complete. 2023-07-21T22:25:44.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.697+0000 7f5b0349db80 1 -- start start 2023-07-21T22:25:44.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 Processor -- start 2023-07-21T22:25:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c90be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c8ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c8e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815b75ac0 con 0x555815c891b0 2023-07-21T22:25:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815b757c0 con 0x555815c89ac0 2023-07-21T22:25:44.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815a59920 con 0x555815c84930 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5b0349db80 10 client.0 _ll_register_callbacks cb 0x555815b73760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c90be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5aed19a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c8ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5aee19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c8e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c90be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39724/0 (socket says 172.21.15.66:39724) 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.698+0000 7f5aed99b700 1 -- 172.21.15.66:0/1631802629 learned_addr learned my addr 172.21.15.66:0/1631802629 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aed99b700 1 -- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 msgr2=0x555815c8e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c8e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aed99b700 1 -- 172.21.15.66:0/1631802629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 msgr2=0x555815c8ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c8ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aed99b700 1 -- 172.21.15.66:0/1631802629 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae810d180 con 0x555815c84930 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aee19c700 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5aed99b700 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c90be0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400bc40 tx=0x7f5ae4024c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25f1a3671e779afe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5ad17fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:44.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.699+0000 7f5ad17fa700 1 -- 172.21.15.66:0/1631802629 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4027070 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555815b2f4f0 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5ad17fa700 1 -- 172.21.15.66:0/1631802629 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ae4005240 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5ad17fa700 1 -- 172.21.15.66:0/1631802629 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae400bca0 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5b0349db80 10 client.26443 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5558159ecc60 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5ad17fa700 1 -- 172.21.15.66:0/1631802629 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555815b2f4f0 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5ad17fa700 1 -- 172.21.15.66:0/1631802629 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5ae402c470 con 0x555815c84930 2023-07-21T22:25:44.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5b0349db80 10 client.26443 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.700+0000 7f5b0349db80 10 client.26443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5b0349db80 20 client.26443 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815c9dfd0 0x555815ca03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5b0349db80 4 client.26443 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555815bceeb0 con 0x555815c9dfd0 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815c9dfd0 0x555815ca03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5aed19a700 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815c9dfd0 0x555815ca03e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb22e61af3d921f3 server_cookie=10c2a384f1f0a64a in_seq=0 out_seq=0 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.701+0000 7f5ad17fa700 10 client.26443 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5ad17fa700 1 -- 172.21.15.66:0/1631802629 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555815bceeb0 con 0x555815c9dfd0 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5ad17fa700 10 client.26443 handle_command_reply: tid=0 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 client.26443 shutdown 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 msgr2=0x555815c90be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c90be0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400bc40 tx=0x7f5ae4024c40 comp rx=0 tx=0).stop 2023-07-21T22:25:44.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 shutdown_connections 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815c9dfd0 0x555815ca03e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c90be0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 --2- 172.21.15.66:0/1631802629 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c8ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5ad17fa700 0 client.26443 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 >> 172.21.15.66:0/1631802629 conn(0x555815c79ef0 msgr2=0x555815c7f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.702+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 shutdown_connections 2023-07-21T22:25:44.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.703+0000 7f5b0349db80 1 -- 172.21.15.66:0/1631802629 wait complete. 2023-07-21T22:25:44.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.703+0000 7f5b0349db80 20 client.26443 trim_cache size 0 max 16384 2023-07-21T22:25:44.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 Processor -- start 2023-07-21T22:25:44.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 -- start start 2023-07-21T22:25:44.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c95520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c90710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c82a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815c9a3d0 con 0x555815c891b0 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815c7e5e0 con 0x555815c89ac0 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.717+0000 7f5b0349db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815c8c0f0 con 0x555815c84930 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c82a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c95520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c95520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39736/0 (socket says 172.21.15.66:39736) 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aed99b700 1 -- 172.21.15.66:0/3625369121 learned_addr learned my addr 172.21.15.66:0/3625369121 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aed19a700 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c90710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 -- 172.21.15.66:0/3625369121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 msgr2=0x555815c95520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c95520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 -- 172.21.15.66:0/3625369121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 msgr2=0x555815c90710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c90710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 -- 172.21.15.66:0/3625369121 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555815a2a250 con 0x555815c89ac0 2023-07-21T22:25:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.718+0000 7f5aee19c700 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c82a60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00ca010 tx=0x7f5ae00b9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ea0545ee59b2db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5adcbff700 1 -- 172.21.15.66:0/3625369121 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0099380 con 0x555815c89ac0 2023-07-21T22:25:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5adcbff700 1 -- 172.21.15.66:0/3625369121 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae0004ad0 con 0x555815c89ac0 2023-07-21T22:25:44.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5adcbff700 1 -- 172.21.15.66:0/3625369121 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00a1950 con 0x555815c89ac0 2023-07-21T22:25:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 -- 172.21.15.66:0/3625369121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 msgr2=0x555815c82a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c82a60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00ca010 tx=0x7f5ae00b9040 comp rx=0 tx=0).stop 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 -- 172.21.15.66:0/3625369121 shutdown_connections 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c89ac0 0x555815c82a60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c84930 0x555815c95520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3625369121 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c90710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 -- 172.21.15.66:0/3625369121 >> 172.21.15.66:0/3625369121 conn(0x555815c79ef0 msgr2=0x555815c99cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 -- 172.21.15.66:0/3625369121 shutdown_connections 2023-07-21T22:25:44.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.719+0000 7f5b0349db80 1 -- 172.21.15.66:0/3625369121 wait complete. 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.720+0000 7f5b0349db80 1 -- start start 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.720+0000 7f5b0349db80 1 Processor -- start 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 0x555815c953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c93660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c89ac0 0x555815c93ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815c940e0 con 0x555815c891b0 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815c94220 con 0x555815c84930 2023-07-21T22:25:44.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555815c943e0 con 0x555815c89ac0 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5b0349db80 10 client.0 _ll_register_callbacks cb 0x7f5ae0009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 0x555815c953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 0x555815c953e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56248/0 (socket says 172.21.15.66:56248) 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed19a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c93660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 -- 172.21.15.66:0/3659928087 learned_addr learned my addr 172.21.15.66:0/3659928087 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aee19c700 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c89ac0 0x555815c93ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 -- 172.21.15.66:0/3659928087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c89ac0 msgr2=0x555815c93ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c89ac0 0x555815c93ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 -- 172.21.15.66:0/3659928087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 msgr2=0x555815c93660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c93660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.721+0000 7f5aed99b700 1 -- 172.21.15.66:0/3659928087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81a4b70 con 0x555815c84930 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5aed99b700 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 0x555815c953e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ae402c880 tx=0x7f5ae402df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81fbf90d834a3747 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5ad17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5ad17fa700 1 -- 172.21.15.66:0/3659928087 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae401d040 con 0x555815c84930 2023-07-21T22:25:44.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5ad17fa700 1 -- 172.21.15.66:0/3659928087 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5ae4000e40 con 0x555815c84930 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ae401e040 con 0x555815c84930 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5ad17fa700 1 -- 172.21.15.66:0/3659928087 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ae4027040 con 0x555815c84930 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5b0349db80 10 client.26474 fetch_fsmap learned FSMap version 32 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555815a022b0 con 0x555815c84930 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5ad17fa700 1 -- 172.21.15.66:0/3659928087 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae401e040 con 0x555815c84930 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.722+0000 7f5ad17fa700 1 -- 172.21.15.66:0/3659928087 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5ae402b3d0 con 0x555815c84930 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5b0349db80 10 client.26474 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T22:25:44.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5b0349db80 10 client.26474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5b0349db80 20 client.26474 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815cf2e80 0x555815cf5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5b0349db80 4 client.26474 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555815b91050 con 0x555815cf2e80 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5aed19a700 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815cf2e80 0x555815cf5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5aed19a700 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815cf2e80 0x555815cf5290 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297cb76a4fe8f07e server_cookie=fbf2d5ad978da2df in_seq=0 out_seq=0 2023-07-21T22:25:44.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5ad17fa700 10 client.26474 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:44.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5ad17fa700 1 -- 172.21.15.66:0/3659928087 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555815b91050 con 0x555815cf2e80 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.723+0000 7f5ad17fa700 10 client.26474 handle_command_reply: tid=0 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 client.26474 shutdown 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 msgr2=0x555815c953e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 0x555815c953e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5ae402c880 tx=0x7f5ae402df70 comp rx=0 tx=0).stop 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 shutdown_connections 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555815c84930 0x555815c953e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555815cf2e80 0x555815cf5290 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555815c89ac0 0x555815c93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 --2- 172.21.15.66:0/3659928087 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555815c891b0 0x555815c93660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5ad17fa700 0 client.26474 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 >> 172.21.15.66:0/3659928087 conn(0x555815c79ef0 msgr2=0x555815c8d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 shutdown_connections 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 1 -- 172.21.15.66:0/3659928087 wait complete. 2023-07-21T22:25:44.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.724+0000 7f5b0349db80 20 client.26474 trim_cache size 0 max 16384 2023-07-21T22:25:44.779 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:25:44.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5ad40a12c0 msgr2=0x7f5ad40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5ad40a12c0 0x7f5ad40a3770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8100160 tx=0x7f5ad80138a0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae810a060 msgr2=0x7f5ae81a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae810a060 0x7f5ae81a3e60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000efc0 tx=0x7f5ae000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 shutdown_connections 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ae810a060 0x7f5ae81a3e60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5ad40a12c0 0x7f5ad40a3770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ae8100d20 0x7f5ae80fe970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 --2- 172.21.15.66:0/1353241373 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5ae8101630 0x7f5ae80feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 >> 172.21.15.66:0/1353241373 conn(0x7f5ae80fa4b0 msgr2=0x7f5ae810c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 shutdown_connections 2023-07-21T22:25:44.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:44.725+0000 7f5aefbff700 1 -- 172.21.15.66:0/1353241373 wait complete. 2023-07-21T22:25:45.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 100880 4.05 MB/sec execute 72 sec latency 16.245 ms 2023-07-21T22:25:45.791 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:25:45.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:25:46.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.967+0000 7ff9913ff700 1 -- 172.21.15.66:0/593005242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c073140 msgr2=0x7ff98c10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.967+0000 7ff9913ff700 1 --2- 172.21.15.66:0/593005242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c073140 0x7ff98c10be80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff980009990 tx=0x7ff980009ca0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 -- 172.21.15.66:0/593005242 shutdown_connections 2023-07-21T22:25:46.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 --2- 172.21.15.66:0/593005242 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c10c3c0 0x7ff98c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 --2- 172.21.15.66:0/593005242 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 --2- 172.21.15.66:0/593005242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c073140 0x7ff98c10be80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 -- 172.21.15.66:0/593005242 >> 172.21.15.66:0/593005242 conn(0x7ff98c0fb600 msgr2=0x7ff98c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 -- 172.21.15.66:0/593005242 shutdown_connections 2023-07-21T22:25:46.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.968+0000 7ff9913ff700 1 -- 172.21.15.66:0/593005242 wait complete. 2023-07-21T22:25:46.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 Processor -- start 2023-07-21T22:25:46.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 -- start start 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c073140 0x7ff98c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 0x7ff98c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c19cdb0 con 0x7ff98c073140 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c19cf20 con 0x7ff98c10c3c0 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.969+0000 7ff9913ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c19d0e0 con 0x7ff98c072830 2023-07-21T22:25:46.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 0x7ff98c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39774/0 (socket says 172.21.15.66:39774) 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 0x7ff98c19c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56276/0 (socket says 172.21.15.66:56276) 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98affd700 1 -- 172.21.15.66:0/3555194393 learned_addr learned my addr 172.21.15.66:0/3555194393 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c073140 0x7ff98c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 -- 172.21.15.66:0/3555194393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 msgr2=0x7ff98c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 -- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c073140 msgr2=0x7ff98c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c073140 0x7ff98c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 -- 172.21.15.66:0/3555194393 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800095f0 con 0x7ff98c10c3c0 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c073140 0x7ff98c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98affd700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.970+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 0x7ff98c19c770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff98400efc0 tx=0x7ff98400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.971+0000 7ff973fff700 1 -- 172.21.15.66:0/3555194393 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984018070 con 0x7ff98c10c3c0 2023-07-21T22:25:46.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.971+0000 7ff973fff700 1 -- 172.21.15.66:0/3555194393 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff98400f040 con 0x7ff98c10c3c0 2023-07-21T22:25:46.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.971+0000 7ff973fff700 1 -- 172.21.15.66:0/3555194393 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400cd40 con 0x7ff98c10c3c0 2023-07-21T22:25:46.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.971+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c19d360 con 0x7ff98c10c3c0 2023-07-21T22:25:46.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.971+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c1a4c00 con 0x7ff98c10c3c0 2023-07-21T22:25:46.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.976+0000 7ff9a4caab80 1 Processor -- start 2023-07-21T22:25:46.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.976+0000 7ff9a4caab80 1 -- start start 2023-07-21T22:25:46.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc18920 0x55695cc18cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc1de10 0x55695cc1e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cb14d30 con 0x55695cc18920 2023-07-21T22:25:46.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cb67ac0 con 0x55695cc19230 2023-07-21T22:25:46.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cb60f50 con 0x55695cc1de10 2023-07-21T22:25:46.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc1de10 0x55695cc1e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc1de10 0x55695cc1e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39782/0 (socket says 172.21.15.66:39782) 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.977+0000 7ff98a7fc700 1 -- 172.21.15.66:0/1963254765 learned_addr learned my addr 172.21.15.66:0/1963254765 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc18920 0x55695cc18cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 -- 172.21.15.66:0/1963254765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc1de10 msgr2=0x55695cc1e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc1de10 0x55695cc1e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 -- 172.21.15.66:0/1963254765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc18920 msgr2=0x55695cc18cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc18920 0x55695cc18cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 -- 172.21.15.66:0/1963254765 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695c98ee60 con 0x55695cc19230 2023-07-21T22:25:46.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.978+0000 7ff98affd700 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1d8d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff97c011570 tx=0x7ff97c011880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b478331fae3640 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.980+0000 7ff973fff700 1 -- 172.21.15.66:0/3555194393 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff984003a00 con 0x7ff98c10c3c0 2023-07-21T22:25:46.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.981+0000 7ff973fff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9740a1340 0x7ff9740a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.981+0000 7ff973fff700 1 -- 172.21.15.66:0/3555194393 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff9840ccb40 con 0x7ff98c10c3c0 2023-07-21T22:25:46.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.981+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9740a1340 0x7ff9740a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.982+0000 7ff9717fa700 1 -- 172.21.15.66:0/1963254765 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c004230 con 0x55695cc19230 2023-07-21T22:25:46.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.982+0000 7ff9717fa700 1 -- 172.21.15.66:0/1963254765 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97c004390 con 0x55695cc19230 2023-07-21T22:25:46.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.982+0000 7ff9717fa700 1 -- 172.21.15.66:0/1963254765 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c019920 con 0x55695cc19230 2023-07-21T22:25:46.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1963254765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 msgr2=0x55695cc1d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9740a1340 0x7ff9740a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff980009990 tx=0x7ff980005fd0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1d8d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff97c011570 tx=0x7ff97c011880 comp rx=0 tx=0).stop 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1963254765 shutdown_connections 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc1de10 0x55695cc1e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc18920 0x55695cc18cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1963254765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1d8d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1963254765 >> 172.21.15.66:0/1963254765 conn(0x55695cc04f70 msgr2=0x55695cc0a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1963254765 shutdown_connections 2023-07-21T22:25:46.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1963254765 wait complete. 2023-07-21T22:25:46.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.983+0000 7ff9a4caab80 1 -- start start 2023-07-21T22:25:46.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 Processor -- start 2023-07-21T22:25:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc21ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc22220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cb00b40 con 0x55695cc1de10 2023-07-21T22:25:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cb00840 con 0x55695cc19230 2023-07-21T22:25:46.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695c9e49a0 con 0x55695cc18920 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff9a4caab80 10 client.0 _ll_register_callbacks cb 0x55695cafe7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.984+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc21ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc21ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56286/0 (socket says 172.21.15.66:56286) 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98a7fc700 1 -- 172.21.15.66:0/1557199042 learned_addr learned my addr 172.21.15.66:0/1557199042 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc22220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 -- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 msgr2=0x55695cc21ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc21ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 -- 172.21.15.66:0/1557199042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 msgr2=0x55695cc22220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc22220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 -- 172.21.15.66:0/1557199042 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97c009d60 con 0x55695cc18920 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc22220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff98affd700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc217a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff97c000c00 tx=0x7ff97c005d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=135d1ca5be938f80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.985+0000 7ff96eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff96eff5700 1 -- 172.21.15.66:0/1557199042 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c003b90 con 0x55695cc18920 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55695caba570 con 0x55695cc18920 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff96eff5700 1 -- 172.21.15.66:0/1557199042 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff97c005040 con 0x55695cc18920 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff96eff5700 1 -- 172.21.15.66:0/1557199042 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97c005200 con 0x55695cc18920 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff9a4caab80 10 client.26461 fetch_fsmap learned FSMap version 33 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55695c977ce0 con 0x55695cc18920 2023-07-21T22:25:46.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff96eff5700 1 -- 172.21.15.66:0/1557199042 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55695caba570 con 0x55695cc18920 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff96eff5700 1 -- 172.21.15.66:0/1557199042 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff97c021040 con 0x55695cc18920 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff9a4caab80 10 client.26461 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff9a4caab80 10 client.26461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.986+0000 7ff9a4caab80 20 client.26461 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.987+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc31b60 0x55695cc33f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.987+0000 7ff9a4caab80 4 client.26461 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.987+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55695cb59f30 con 0x55695cc31b60 2023-07-21T22:25:46.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.987+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc31b60 0x55695cc33f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.987+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc31b60 0x55695cc33f70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ce3a61837d69a2 server_cookie=ca1be8d034371bed in_seq=0 out_seq=0 2023-07-21T22:25:46.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.987+0000 7ff96eff5700 10 client.26461 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff96eff5700 1 -- 172.21.15.66:0/1557199042 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55695cb59f30 con 0x55695cc31b60 2023-07-21T22:25:46.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff96eff5700 10 client.26461 handle_command_reply: tid=0 2023-07-21T22:25:46.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 client.26461 shutdown 2023-07-21T22:25:46.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 msgr2=0x55695cc217a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc217a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff97c000c00 tx=0x7ff97c005d90 comp rx=0 tx=0).stop 2023-07-21T22:25:46.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 shutdown_connections 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc31b60 0x55695cc33f70 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc22220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc217a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff96eff5700 0 client.26461 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1557199042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc21ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 >> 172.21.15.66:0/1557199042 conn(0x55695cc04f70 msgr2=0x55695cc0a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 shutdown_connections 2023-07-21T22:25:46.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.988+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1557199042 wait complete. 2023-07-21T22:25:46.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:45.989+0000 7ff9a4caab80 20 client.26461 trim_cache size 0 max 16384 2023-07-21T22:25:46.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.009+0000 7ff9a4caab80 1 Processor -- start 2023-07-21T22:25:46.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.009+0000 7ff9a4caab80 1 -- start start 2023-07-21T22:25:46.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc1d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cc248d0 con 0x55695cc1de10 2023-07-21T22:25:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cc1fd80 con 0x55695cc19230 2023-07-21T22:25:46.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cc09660 con 0x55695cc18920 2023-07-21T22:25:46.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc28d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39808/0 (socket says 172.21.15.66:39808) 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56298/0 (socket says 172.21.15.66:56298) 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 -- 172.21.15.66:0/1257117842 learned_addr learned my addr 172.21.15.66:0/1257117842 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc1d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 -- 172.21.15.66:0/1257117842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 msgr2=0x55695cc1f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 -- 172.21.15.66:0/1257117842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 msgr2=0x55695cc1d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc1d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.010+0000 7ff98affd700 1 -- 172.21.15.66:0/1257117842 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695cc1b8e0 con 0x55695cc18920 2023-07-21T22:25:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.011+0000 7ff98affd700 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc28d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0114f0 tx=0x7ff97c011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=914892ca221a2cf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.011+0000 7ff971ffb700 1 -- 172.21.15.66:0/1257117842 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c003b90 con 0x55695cc18920 2023-07-21T22:25:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.011+0000 7ff971ffb700 1 -- 172.21.15.66:0/1257117842 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97c0196d0 con 0x55695cc18920 2023-07-21T22:25:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.011+0000 7ff971ffb700 1 -- 172.21.15.66:0/1257117842 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c004c40 con 0x55695cc18920 2023-07-21T22:25:46.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.011+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1257117842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 msgr2=0x55695cc28d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.011+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc28d10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff97c0114f0 tx=0x7ff97c011880 comp rx=0 tx=0).stop 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1257117842 shutdown_connections 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc1d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc28d10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/1257117842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc1f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1257117842 >> 172.21.15.66:0/1257117842 conn(0x55695cc04f70 msgr2=0x55695cc1cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1257117842 shutdown_connections 2023-07-21T22:25:46.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 -- 172.21.15.66:0/1257117842 wait complete. 2023-07-21T22:25:46.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 -- start start 2023-07-21T22:25:46.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.012+0000 7ff9a4caab80 1 Processor -- start 2023-07-21T22:25:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc7f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc7f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cc79f90 con 0x55695cc1de10 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cc7a100 con 0x55695cc19230 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695cc7a2c0 con 0x55695cc18920 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff9a4caab80 10 client.0 _ll_register_callbacks cb 0x7ff980009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc7f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc7f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39824/0 (socket says 172.21.15.66:39824) 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 -- 172.21.15.66:0/3690135744 learned_addr learned my addr 172.21.15.66:0/3690135744 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc7f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98b7fe700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 -- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 msgr2=0x55695cc7f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc7f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 -- 172.21.15.66:0/3690135744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 msgr2=0x55695cc799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.013+0000 7ff98affd700 1 -- 172.21.15.66:0/3690135744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55695c977ce0 con 0x55695cc18920 2023-07-21T22:25:46.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff98affd700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc7f460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003f80 tx=0x7ff97c011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9856575db7acc2fa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff96eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff96eff5700 1 -- 172.21.15.66:0/3690135744 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c004ac0 con 0x55695cc18920 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff96eff5700 1 -- 172.21.15.66:0/3690135744 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff97c000e40 con 0x55695cc18920 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff97c004390 con 0x55695cc18920 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff96eff5700 1 -- 172.21.15.66:0/3690135744 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff97c005c80 con 0x55695cc18920 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff9a4caab80 10 client.26467 fetch_fsmap learned FSMap version 33 2023-07-21T22:25:46.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.014+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55695c9b52d0 con 0x55695cc18920 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff96eff5700 1 -- 172.21.15.66:0/3690135744 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff97c004390 con 0x55695cc18920 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff96eff5700 1 -- 172.21.15.66:0/3690135744 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff97c021070 con 0x55695cc18920 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff9a4caab80 10 client.26467 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff9a4caab80 10 client.26467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff9a4caab80 20 client.26467 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc86e20 0x55695cc89200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff9a4caab80 4 client.26467 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55695cb1c0d0 con 0x55695cc86e20 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.015+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc86e20 0x55695cc89200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.016+0000 7ff98a7fc700 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc86e20 0x55695cc89200 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a8e35f0d2dbe98 server_cookie=bb116bfb7d4db646 in_seq=0 out_seq=0 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.016+0000 7ff96eff5700 10 client.26467 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.016+0000 7ff96eff5700 1 -- 172.21.15.66:0/3690135744 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55695cb1c0d0 con 0x55695cc86e20 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.016+0000 7ff96eff5700 10 client.26467 handle_command_reply: tid=0 2023-07-21T22:25:46.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.016+0000 7ff9a4caab80 1 client.26467 shutdown 2023-07-21T22:25:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 msgr2=0x55695cc7f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc7f460 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff97c003f80 tx=0x7ff97c011d10 comp rx=0 tx=0).stop 2023-07-21T22:25:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 shutdown_connections 2023-07-21T22:25:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55695cc86e20 0x55695cc89200 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55695cc1de10 0x55695cc799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55695cc18920 0x55695cc7f460 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 --2- 172.21.15.66:0/3690135744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55695cc19230 0x55695cc7f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 >> 172.21.15.66:0/3690135744 conn(0x55695cc04f70 msgr2=0x55695cc16bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff96eff5700 0 client.26467 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 shutdown_connections 2023-07-21T22:25:46.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 1 -- 172.21.15.66:0/3690135744 wait complete. 2023-07-21T22:25:46.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.017+0000 7ff9a4caab80 20 client.26467 trim_cache size 0 max 16384 2023-07-21T22:25:46.073 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:25:46.073 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:25:46.074 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "f996705e-bc97-4251-b632-1a299122b0f3", 2023-07-21T22:25:46.074 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:25:46.074 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:25:46.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9740a1340 msgr2=0x7ff9740a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9740a1340 0x7ff9740a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff980009990 tx=0x7ff980005fd0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 msgr2=0x7ff98c19c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 0x7ff98c19c770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff98400efc0 tx=0x7ff98400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 shutdown_connections 2023-07-21T22:25:46.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff98c10c3c0 0x7ff98c19c770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff9740a1340 0x7ff9740a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff98c072830 0x7ff98c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 --2- 172.21.15.66:0/3555194393 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff98c073140 0x7ff98c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 >> 172.21.15.66:0/3555194393 conn(0x7ff98c0fb600 msgr2=0x7ff98c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 shutdown_connections 2023-07-21T22:25:46.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.018+0000 7ff9913ff700 1 -- 172.21.15.66:0/3555194393 wait complete. 2023-07-21T22:25:46.084 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:25:46.148 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:45 smithi066 ceph-mon[126379]: pgmap v131: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 278 op/s 2023-07-21T22:25:46.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 -- 172.21.15.66:0/1334228838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f899010c370 msgr2=0x7f899010e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 --2- 172.21.15.66:0/1334228838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f899010c370 0x7f899010e750 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f898800b0a0 tx=0x7f898800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 -- 172.21.15.66:0/1334228838 shutdown_connections 2023-07-21T22:25:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 --2- 172.21.15.66:0/1334228838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f899010c370 0x7f899010e750 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 --2- 172.21.15.66:0/1334228838 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8990072850 0x7f8990072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 --2- 172.21.15.66:0/1334228838 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 0x7f899010be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 -- 172.21.15.66:0/1334228838 >> 172.21.15.66:0/1334228838 conn(0x7f89900fb610 msgr2=0x7f89900fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 -- 172.21.15.66:0/1334228838 shutdown_connections 2023-07-21T22:25:46.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.256+0000 7f89959ff700 1 -- 172.21.15.66:0/1334228838 wait complete. 2023-07-21T22:25:46.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.258+0000 7f89959ff700 1 Processor -- start 2023-07-21T22:25:46.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.258+0000 7f89959ff700 1 -- start start 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f89959ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 0x7f8990197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f89959ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 0x7f89901984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f89959ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f899010c370 0x7f899019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f89959ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899019cf40 con 0x7f8990073160 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f89959ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899019d0b0 con 0x7f8990072850 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f89959ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899019d270 con 0x7f899010c370 2023-07-21T22:25:46.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 0x7f8990197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 0x7f8990197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56330/0 (socket says 172.21.15.66:56330) 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898effd700 1 -- 172.21.15.66:0/2206568223 learned_addr learned my addr 172.21.15.66:0/2206568223 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898f7fe700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f899010c370 0x7f899019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898e7fc700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 0x7f89901984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898f7fe700 1 -- 172.21.15.66:0/2206568223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 msgr2=0x7f8990197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898f7fe700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 0x7f8990197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898f7fe700 1 -- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 msgr2=0x7f89901984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898f7fe700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 0x7f89901984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898f7fe700 1 -- 172.21.15.66:0/2206568223 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988009d00 con 0x7f899010c370 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898e7fc700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 0x7f89901984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.259+0000 7f898effd700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 0x7f8990197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.260+0000 7f898f7fe700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f899010c370 0x7f899019c900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f898800b070 tx=0x7f8988007930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.260+0000 7f89949fd700 1 -- 172.21.15.66:0/2206568223 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800e040 con 0x7f899010c370 2023-07-21T22:25:46.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.260+0000 7f89949fd700 1 -- 172.21.15.66:0/2206568223 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8988004c50 con 0x7f899010c370 2023-07-21T22:25:46.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.260+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899019d4f0 con 0x7f899010c370 2023-07-21T22:25:46.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.260+0000 7f89949fd700 1 -- 172.21.15.66:0/2206568223 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988005040 con 0x7f899010c370 2023-07-21T22:25:46.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.260+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89901a4d90 con 0x7f899010c370 2023-07-21T22:25:46.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.265+0000 7f89949fd700 1 -- 172.21.15.66:0/2206568223 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f898801d470 con 0x7f899010c370 2023-07-21T22:25:46.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 Processor -- start 2023-07-21T22:25:46.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 -- start start 2023-07-21T22:25:46.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x563251430580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 0x56325142b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56325142b930 0x56325142bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251330cb0 con 0x56325142b930 2023-07-21T22:25:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251383a40 con 0x5632514301b0 2023-07-21T22:25:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89a930bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325137ced0 con 0x563251430ac0 2023-07-21T22:25:46.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89949fd700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f89780a12c0 0x7f89780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f89949fd700 1 -- 172.21.15.66:0/2206568223 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f89880ce670 con 0x7f899010c370 2023-07-21T22:25:46.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f898effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 0x56325142b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f898e7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x563251430580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.266+0000 7f898effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 0x56325142b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39858/0 (socket says 172.21.15.66:39858) 2023-07-21T22:25:46.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898effd700 1 -- 172.21.15.66:0/42097243 learned_addr learned my addr 172.21.15.66:0/42097243 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56325142b930 0x56325142bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898f7fe700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f89780a12c0 0x7f89780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 -- 172.21.15.66:0/42097243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 msgr2=0x56325142b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 0x56325142b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 -- 172.21.15.66:0/42097243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56325142b930 msgr2=0x56325142bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56325142b930 0x56325142bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 -- 172.21.15.66:0/42097243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632511aade0 con 0x5632514301b0 2023-07-21T22:25:46.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898effd700 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 0x56325142b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898e7fc700 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x563251430580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8990199570 tx=0x7f8984009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b1d8fcf17485d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f898f7fe700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f89780a12c0 0x7f89780a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f898809b3b0 tx=0x7f89880097e0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.267+0000 7f8975ffb700 1 -- 172.21.15.66:0/42097243 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984012d80 con 0x5632514301b0 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f8975ffb700 1 -- 172.21.15.66:0/42097243 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89840044f0 con 0x5632514301b0 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f8975ffb700 1 -- 172.21.15.66:0/42097243 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400f650 con 0x5632514301b0 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 -- 172.21.15.66:0/42097243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 msgr2=0x563251430580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x563251430580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8990199570 tx=0x7f8984009690 comp rx=0 tx=0).stop 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 -- 172.21.15.66:0/42097243 shutdown_connections 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x563251430580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56325142b930 0x56325142bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 --2- 172.21.15.66:0/42097243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563251430ac0 0x56325142b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 -- 172.21.15.66:0/42097243 >> 172.21.15.66:0/42097243 conn(0x563251420ef0 msgr2=0x563251426830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.268+0000 7f89a930bb80 1 -- 172.21.15.66:0/42097243 shutdown_connections 2023-07-21T22:25:46.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.269+0000 7f89a930bb80 1 -- 172.21.15.66:0/42097243 wait complete. 2023-07-21T22:25:46.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.269+0000 7f89a930bb80 1 -- start start 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.269+0000 7f89a930bb80 1 Processor -- start 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56325142b930 0x563251437c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 0x563251434e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x5632514354e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325131cac0 con 0x563251430ac0 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325131c7c0 con 0x56325142b930 2023-07-21T22:25:46.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251200920 con 0x5632514301b0 2023-07-21T22:25:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f89a930bb80 10 client.0 _ll_register_callbacks cb 0x56325131a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f898effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 0x563251434e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f898effd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 0x563251434e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39870/0 (socket says 172.21.15.66:39870) 2023-07-21T22:25:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f898effd700 1 -- 172.21.15.66:0/225340372 learned_addr learned my addr 172.21.15.66:0/225340372 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f898e7fc700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56325142b930 0x563251437c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.270+0000 7f898e7fc700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x5632514354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898effd700 1 -- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56325142b930 msgr2=0x563251437c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898effd700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56325142b930 0x563251437c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898effd700 1 -- 172.21.15.66:0/225340372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 msgr2=0x5632514354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898effd700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x5632514354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898effd700 1 -- 172.21.15.66:0/225340372 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8980011440 con 0x5632514301b0 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898e7fc700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x5632514354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:25:46.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f898effd700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 0x563251434e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8980012e80 tx=0x7f898000f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=332d78080f91821b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f896effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f896effd700 1 -- 172.21.15.66:0/225340372 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980017070 con 0x5632514301b0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632512d64f0 con 0x5632514301b0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.271+0000 7f896effd700 1 -- 172.21.15.66:0/225340372 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8980004830 con 0x5632514301b0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f896effd700 1 -- 172.21.15.66:0/225340372 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f898000fea0 con 0x5632514301b0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 10 client.26479 fetch_fsmap learned FSMap version 33 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f898401a040 con 0x5632514301b0 2023-07-21T22:25:46.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f896effd700 1 -- 172.21.15.66:0/225340372 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632512d64f0 con 0x5632514301b0 2023-07-21T22:25:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f896effd700 1 -- 172.21.15.66:0/225340372 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f898001a550 con 0x5632514301b0 2023-07-21T22:25:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 10 client.26479 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:25:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 10 client.26479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 20 client.26479 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x5632514479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 4 client.26479 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563251375eb0 con 0x5632514455d0 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.272+0000 7f898e7fc700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x5632514479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.273+0000 7f896effd700 10 client.26479 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.273+0000 7f898e7fc700 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x5632514479e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5faf72527038ac18 server_cookie=975236144f728630 in_seq=0 out_seq=0 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.273+0000 7f896effd700 1 -- 172.21.15.66:0/225340372 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563251375eb0 con 0x5632514455d0 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f896effd700 10 client.26479 handle_command_reply: tid=0 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 client.26479 shutdown 2023-07-21T22:25:46.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 msgr2=0x563251434e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 0x563251434e80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8980012e80 tx=0x7f898000f6a0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 shutdown_connections 2023-07-21T22:25:46.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56325142b930 0x563251437c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x5632514479e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f896effd700 0 client.26479 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x5632514354e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 --2- 172.21.15.66:0/225340372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5632514301b0 0x563251434e80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 >> 172.21.15.66:0/225340372 conn(0x563251420ef0 msgr2=0x563251426420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 shutdown_connections 2023-07-21T22:25:46.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 1 -- 172.21.15.66:0/225340372 wait complete. 2023-07-21T22:25:46.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.274+0000 7f89a930bb80 20 client.26479 trim_cache size 0 max 16384 2023-07-21T22:25:46.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.292+0000 7f89a930bb80 1 Processor -- start 2023-07-21T22:25:46.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.292+0000 7f89a930bb80 1 -- start start 2023-07-21T22:25:46.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x56325142a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x56325142af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f89a930bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251423010 con 0x563251430ac0 2023-07-21T22:25:46.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f89a930bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56325141e050 con 0x5632514301b0 2023-07-21T22:25:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f89a930bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251427e50 con 0x56325142b930 2023-07-21T22:25:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x56325142a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x56325142a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56368/0 (socket says 172.21.15.66:56368) 2023-07-21T22:25:46.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 -- 172.21.15.66:0/3983532155 learned_addr learned my addr 172.21.15.66:0/3983532155 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898e7fc700 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 -- 172.21.15.66:0/3983532155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 msgr2=0x56325143c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898e7fc700 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x56325142af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 -- 172.21.15.66:0/3983532155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 msgr2=0x56325142af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x56325142af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898effd700 1 -- 172.21.15.66:0/3983532155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563251193c60 con 0x5632514301b0 2023-07-21T22:25:46.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.293+0000 7f898e7fc700 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x56325142af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:25:46.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.294+0000 7f898effd700 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x56325142a9e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8980009340 tx=0x7f898000faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f27184502b9bb25 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.294+0000 7f89767fc700 1 -- 172.21.15.66:0/3983532155 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980017070 con 0x5632514301b0 2023-07-21T22:25:46.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.294+0000 7f89767fc700 1 -- 172.21.15.66:0/3983532155 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f898001ae10 con 0x5632514301b0 2023-07-21T22:25:46.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.294+0000 7f89767fc700 1 -- 172.21.15.66:0/3983532155 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980022300 con 0x5632514301b0 2023-07-21T22:25:46.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 -- 172.21.15.66:0/3983532155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 msgr2=0x56325142a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x56325142a9e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8980009340 tx=0x7f898000faa0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 -- 172.21.15.66:0/3983532155 shutdown_connections 2023-07-21T22:25:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563251430ac0 0x56325142af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 --2- 172.21.15.66:0/3983532155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632514301b0 0x56325142a9e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 -- 172.21.15.66:0/3983532155 >> 172.21.15.66:0/3983532155 conn(0x563251420ef0 msgr2=0x563251429a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 -- 172.21.15.66:0/3983532155 shutdown_connections 2023-07-21T22:25:46.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 -- 172.21.15.66:0/3983532155 wait complete. 2023-07-21T22:25:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 -- start start 2023-07-21T22:25:46.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.295+0000 7f89a930bb80 1 Processor -- start 2023-07-21T22:25:46.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5632514301b0 0x563251433aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563251430ac0 0x5632514341e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632514348f0 con 0x5632514301b0 2023-07-21T22:25:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251499450 con 0x563251430ac0 2023-07-21T22:25:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563251499590 con 0x56325142b930 2023-07-21T22:25:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f89a930bb80 10 client.0 _ll_register_callbacks cb 0x7f8980012980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:25:46.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f898e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f898e7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:39900/0 (socket says 172.21.15.66:39900) 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f898e7fc700 1 -- 172.21.15.66:0/2547418293 learned_addr learned my addr 172.21.15.66:0/2547418293 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f898e7fc700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563251430ac0 0x5632514341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.296+0000 7f898effd700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5632514301b0 0x563251433aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f898e7fc700 1 -- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563251430ac0 msgr2=0x5632514341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f898e7fc700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563251430ac0 0x5632514341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f898e7fc700 1 -- 172.21.15.66:0/2547418293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5632514301b0 msgr2=0x563251433aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f898e7fc700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5632514301b0 0x563251433aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f898e7fc700 1 -- 172.21.15.66:0/2547418293 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898401a040 con 0x56325142b930 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f898e7fc700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8984003aa0 tx=0x7f8984021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0831c257d4fc387 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f896effd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.297+0000 7f896effd700 1 -- 172.21.15.66:0/2547418293 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89840037e0 con 0x56325142b930 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f896effd700 1 -- 172.21.15.66:0/2547418293 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8984012250 con 0x56325142b930 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89840044f0 con 0x56325142b930 2023-07-21T22:25:46.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f896effd700 1 -- 172.21.15.66:0/2547418293 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f898402a270 con 0x56325142b930 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 10 client.26485 fetch_fsmap learned FSMap version 33 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5632511d1250 con 0x56325142b930 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f896effd700 1 -- 172.21.15.66:0/2547418293 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89840044f0 con 0x56325142b930 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f896effd700 1 -- 172.21.15.66:0/2547418293 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f898402a880 con 0x56325142b930 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 10 client.26485 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:25:46.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 20 client.26485 populate_metadata read hostname 'smithi066' 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x56325149cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 4 client.26485 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.298+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563251338050 con 0x5632514455d0 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f898effd700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x56325149cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f898effd700 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x56325149cc60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac01d1c8a706a04 server_cookie=9e86c6c64aaf7708 in_seq=0 out_seq=0 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f896effd700 10 client.26485 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f896effd700 1 -- 172.21.15.66:0/2547418293 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563251338050 con 0x5632514455d0 2023-07-21T22:25:46.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f896effd700 10 client.26485 handle_command_reply: tid=0 2023-07-21T22:25:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f89a930bb80 1 client.26485 shutdown 2023-07-21T22:25:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 msgr2=0x56325143c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.299+0000 7f89a930bb80 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8984003aa0 tx=0x7f8984021040 comp rx=0 tx=0).stop 2023-07-21T22:25:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 shutdown_connections 2023-07-21T22:25:46.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563251430ac0 0x5632514341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5632514455d0 0x56325149cc60 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56325142b930 0x56325143c770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 --2- 172.21.15.66:0/2547418293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5632514301b0 0x563251433aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f896effd700 0 client.26485 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 >> 172.21.15.66:0/2547418293 conn(0x563251420ef0 msgr2=0x56325142a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 shutdown_connections 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 1 -- 172.21.15.66:0/2547418293 wait complete. 2023-07-21T22:25:46.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89a930bb80 20 client.26485 trim_cache size 0 max 16384 2023-07-21T22:25:46.364 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:25:46.364 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:25:46.364 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:25:46.364 INFO:teuthology.orchestra.run.smithi066.stdout: "f996705e-bc97-4251-b632-1a299122b0f3": { 2023-07-21T22:25:46.365 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:25:46.365 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "f996705e-bc97-4251-b632-1a299122b0f3", 2023-07-21T22:25:46.365 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:25:46.365 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:25:46.365 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:25:46.365 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:25:46.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f89780a12c0 msgr2=0x7f89780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89959ff700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f89780a12c0 0x7f89780a3770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f898809b3b0 tx=0x7f89880097e0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f899010c370 msgr2=0x7f899019c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.300+0000 7f89959ff700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f899010c370 0x7f899019c900 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f898800b070 tx=0x7f8988007930 comp rx=0 tx=0).stop 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 shutdown_connections 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8990072850 0x7f8990197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f89780a12c0 0x7f89780a3770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f899010c370 0x7f899019c900 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 --2- 172.21.15.66:0/2206568223 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8990073160 0x7f89901984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:25:46.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 >> 172.21.15.66:0/2206568223 conn(0x7f89900fb610 msgr2=0x7f899010e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:25:46.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 shutdown_connections 2023-07-21T22:25:46.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:25:46.301+0000 7f89959ff700 1 -- 172.21.15.66:0/2206568223 wait complete. 2023-07-21T22:25:46.368 INFO:tasks.cephfs.filesystem:scrub status for tag:f996705e-bc97-4251-b632-1a299122b0f3 - {'path': '/', 'tag': 'f996705e-bc97-4251-b632-1a299122b0f3', 'options': 'recursive,force'} 2023-07-21T22:25:46.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:45 smithi104 ceph-mon[129021]: pgmap v131: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 278 op/s 2023-07-21T22:25:46.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:45 smithi149 ceph-mon[129268]: pgmap v131: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 278 op/s 2023-07-21T22:25:46.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 103176 4.18 MB/sec execute 73 sec latency 22.989 ms 2023-07-21T22:25:47.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:46 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:47.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:46 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:25:47.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:25:47.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:47.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:25:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:46 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:25:47.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 104550 4.20 MB/sec execute 74 sec latency 15.065 ms 2023-07-21T22:25:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:47 smithi104 ceph-mon[129021]: pgmap v132: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-07-21T22:25:48.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:47 smithi066 ceph-mon[126379]: pgmap v132: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-07-21T22:25:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:47 smithi149 ceph-mon[129268]: pgmap v132: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-07-21T22:25:48.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 106827 4.33 MB/sec execute 75 sec latency 10.483 ms 2023-07-21T22:25:49.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 108294 4.34 MB/sec execute 76 sec latency 21.211 ms 2023-07-21T22:25:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:49 smithi104 ceph-mon[129021]: pgmap v133: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-07-21T22:25:50.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:49 smithi066 ceph-mon[126379]: pgmap v133: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-07-21T22:25:50.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:49 smithi149 ceph-mon[129268]: pgmap v133: 129 pgs: 129 active+clean; 81 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2023-07-21T22:25:50.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 110385 4.47 MB/sec execute 77 sec latency 15.771 ms 2023-07-21T22:25:51.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 112204 4.48 MB/sec execute 78 sec latency 12.605 ms 2023-07-21T22:25:52.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:51 smithi104 ceph-mon[129021]: pgmap v134: 129 pgs: 129 active+clean; 88 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2023-07-21T22:25:52.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:51 smithi066 ceph-mon[126379]: pgmap v134: 129 pgs: 129 active+clean; 88 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2023-07-21T22:25:52.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:51 smithi149 ceph-mon[129268]: pgmap v134: 129 pgs: 129 active+clean; 88 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 277 op/s 2023-07-21T22:25:52.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 113959 4.59 MB/sec execute 79 sec latency 12.106 ms 2023-07-21T22:25:53.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 115961 4.63 MB/sec execute 80 sec latency 20.214 ms 2023-07-21T22:25:54.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:53 smithi104 ceph-mon[129021]: pgmap v135: 129 pgs: 129 active+clean; 88 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-07-21T22:25:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:53 smithi066 ceph-mon[126379]: pgmap v135: 129 pgs: 129 active+clean; 88 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-07-21T22:25:54.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:53 smithi149 ceph-mon[129268]: pgmap v135: 129 pgs: 129 active+clean; 88 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2023-07-21T22:25:54.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 117549 4.72 MB/sec execute 81 sec latency 28.627 ms 2023-07-21T22:25:55.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 118916 4.72 MB/sec execute 82 sec latency 149.092 ms 2023-07-21T22:25:56.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:55 smithi104 ceph-mon[129021]: pgmap v136: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2023-07-21T22:25:56.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:55 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:25:56.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:55 smithi066 ceph-mon[126379]: pgmap v136: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2023-07-21T22:25:56.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:55 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:25:56.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:55 smithi149 ceph-mon[129268]: pgmap v136: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 271 op/s 2023-07-21T22:25:56.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:55 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:25:56.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 121029 4.83 MB/sec execute 83 sec latency 29.177 ms 2023-07-21T22:25:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:57 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:25:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:57 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:25:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:57 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:25:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:57 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:25:57.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:57 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:25:57.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:57 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:25:57.527 INFO:tasks.workunit.client.0.smithi066.stdout: 1 122693 4.84 MB/sec execute 84 sec latency 26.138 ms 2023-07-21T22:25:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:25:58 smithi104 ceph-mon[129021]: pgmap v137: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:25:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:25:58 smithi066 ceph-mon[126379]: pgmap v137: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:25:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:25:58 smithi149 ceph-mon[129268]: pgmap v137: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:25:58.528 INFO:tasks.workunit.client.0.smithi066.stdout: 1 124592 4.94 MB/sec execute 85 sec latency 10.603 ms 2023-07-21T22:25:59.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 126327 4.95 MB/sec execute 86 sec latency 28.890 ms 2023-07-21T22:26:00.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:00 smithi104 ceph-mon[129021]: pgmap v138: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:26:00.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:00 smithi066 ceph-mon[126379]: pgmap v138: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:26:00.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:00 smithi149 ceph-mon[129268]: pgmap v138: 129 pgs: 129 active+clean; 93 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 179 op/s 2023-07-21T22:26:00.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 128134 5.05 MB/sec execute 87 sec latency 52.535 ms 2023-07-21T22:26:01.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:01 smithi104 ceph-mon[129021]: pgmap v139: 129 pgs: 129 active+clean; 100 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2023-07-21T22:26:01.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:01 smithi066 ceph-mon[126379]: pgmap v139: 129 pgs: 129 active+clean; 100 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2023-07-21T22:26:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:01 smithi149 ceph-mon[129268]: pgmap v139: 129 pgs: 129 active+clean; 100 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 276 op/s 2023-07-21T22:26:01.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 128757 5.01 MB/sec execute 88 sec latency 59.585 ms 2023-07-21T22:26:02.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 130750 5.08 MB/sec execute 89 sec latency 48.792 ms 2023-07-21T22:26:03.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 132079 5.10 MB/sec execute 90 sec latency 10.849 ms 2023-07-21T22:26:04.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:03 smithi066 ceph-mon[126379]: pgmap v140: 129 pgs: 129 active+clean; 100 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2023-07-21T22:26:04.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:03 smithi149 ceph-mon[129268]: pgmap v140: 129 pgs: 129 active+clean; 100 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2023-07-21T22:26:04.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:03 smithi104 ceph-mon[129021]: pgmap v140: 129 pgs: 129 active+clean; 100 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2023-07-21T22:26:04.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 134298 5.18 MB/sec execute 91 sec latency 79.175 ms 2023-07-21T22:26:05.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 135558 5.20 MB/sec execute 92 sec latency 24.824 ms 2023-07-21T22:26:06.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:05 smithi066 ceph-mon[126379]: pgmap v141: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-07-21T22:26:06.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:05 smithi149 ceph-mon[129268]: pgmap v141: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-07-21T22:26:06.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:05 smithi104 ceph-mon[129021]: pgmap v141: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 317 op/s 2023-07-21T22:26:06.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138014 5.28 MB/sec execute 93 sec latency 22.297 ms 2023-07-21T22:26:07.530 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 5.29 MB/sec execute 94 sec latency 191.697 ms 2023-07-21T22:26:08.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:07 smithi066 ceph-mon[126379]: pgmap v142: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:26:08.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:07 smithi149 ceph-mon[129268]: pgmap v142: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:26:08.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:07 smithi104 ceph-mon[129021]: pgmap v142: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:26:08.531 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 5.23 MB/sec execute 95 sec latency 1191.839 ms 2023-07-21T22:26:09.531 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 5.17 MB/sec execute 96 sec latency 2191.985 ms 2023-07-21T22:26:10.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:09 smithi066 ceph-mon[126379]: pgmap v143: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:26:10.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:09 smithi149 ceph-mon[129268]: pgmap v143: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:26:10.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:09 smithi104 ceph-mon[129021]: pgmap v143: 129 pgs: 129 active+clean; 105 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 226 op/s 2023-07-21T22:26:10.531 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 5.12 MB/sec execute 97 sec latency 3192.255 ms 2023-07-21T22:26:11.531 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 5.07 MB/sec execute 98 sec latency 4192.419 ms 2023-07-21T22:26:12.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:11 smithi066 ceph-mon[126379]: pgmap v144: 129 pgs: 129 active+clean; 114 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2023-07-21T22:26:12.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:11 smithi149 ceph-mon[129268]: pgmap v144: 129 pgs: 129 active+clean; 114 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2023-07-21T22:26:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:11 smithi104 ceph-mon[129021]: pgmap v144: 129 pgs: 129 active+clean; 114 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 285 op/s 2023-07-21T22:26:12.531 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 5.02 MB/sec execute 99 sec latency 5192.502 ms 2023-07-21T22:26:13.531 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.97 MB/sec execute 100 sec latency 6192.622 ms 2023-07-21T22:26:14.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:13 smithi066 ceph-mon[126379]: pgmap v145: 129 pgs: 129 active+clean; 114 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 188 op/s 2023-07-21T22:26:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:13 smithi149 ceph-mon[129268]: pgmap v145: 129 pgs: 129 active+clean; 114 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 188 op/s 2023-07-21T22:26:14.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:13 smithi104 ceph-mon[129021]: pgmap v145: 129 pgs: 129 active+clean; 114 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 188 op/s 2023-07-21T22:26:14.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.92 MB/sec execute 101 sec latency 7192.759 ms 2023-07-21T22:26:15.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.87 MB/sec execute 102 sec latency 8192.857 ms 2023-07-21T22:26:16.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:15 smithi066 ceph-mon[126379]: pgmap v146: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2023-07-21T22:26:16.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:15 smithi149 ceph-mon[129268]: pgmap v146: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2023-07-21T22:26:16.369 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:26:16.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:15 smithi104 ceph-mon[129021]: pgmap v146: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.0 MiB/s wr, 218 op/s 2023-07-21T22:26:16.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.82 MB/sec execute 103 sec latency 9192.930 ms 2023-07-21T22:26:16.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 -- 172.21.15.66:0/3639224862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89200ffad0 msgr2=0x7f89200fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 --2- 172.21.15.66:0/3639224862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89200ffad0 0x7f89200fff40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8910009990 tx=0x7f8910009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 -- 172.21.15.66:0/3639224862 shutdown_connections 2023-07-21T22:26:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 --2- 172.21.15.66:0/3639224862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8920100480 0x7f892010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 --2- 172.21.15.66:0/3639224862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8920105ae0 0x7f8920105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 --2- 172.21.15.66:0/3639224862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89200ffad0 0x7f89200fff40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 -- 172.21.15.66:0/3639224862 >> 172.21.15.66:0/3639224862 conn(0x7f89200fb5f0 msgr2=0x7f89200fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 -- 172.21.15.66:0/3639224862 shutdown_connections 2023-07-21T22:26:16.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.563+0000 7f89287ff700 1 -- 172.21.15.66:0/3639224862 wait complete. 2023-07-21T22:26:16.612 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.564+0000 7f89287ff700 1 Processor -- start 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.564+0000 7f89287ff700 1 -- start start 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f89287ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 0x7f8920197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f89287ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8920100480 0x7f8920198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f89287ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8920105ae0 0x7f892019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f89287ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019cdf0 con 0x7f89200ffad0 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f89287ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019cf60 con 0x7f8920105ae0 2023-07-21T22:26:16.613 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f89287ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892019d120 con 0x7f8920100480 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f892659b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 0x7f8920197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f892659b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 0x7f8920197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50436/0 (socket says 172.21.15.66:50436) 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f892659b700 1 -- 172.21.15.66:0/4038288654 learned_addr learned my addr 172.21.15.66:0/4038288654 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f8925d9a700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8920100480 0x7f8920198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f8926d9c700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8920105ae0 0x7f892019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.565+0000 7f892659b700 1 -- 172.21.15.66:0/4038288654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8920100480 msgr2=0x7f8920198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f892659b700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8920100480 0x7f8920198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f892659b700 1 -- 172.21.15.66:0/4038288654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8920105ae0 msgr2=0x7f892019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f892659b700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8920105ae0 0x7f892019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f892659b700 1 -- 172.21.15.66:0/4038288654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100095f0 con 0x7f89200ffad0 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f892659b700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 0x7f8920197e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e9a0 tx=0x7f891c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f89177fe700 1 -- 172.21.15.66:0/4038288654 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c009790 con 0x7f89200ffad0 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f89177fe700 1 -- 172.21.15.66:0/4038288654 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891c010430 con 0x7f89200ffad0 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892019d400 con 0x7f89200ffad0 2023-07-21T22:26:16.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f89177fe700 1 -- 172.21.15.66:0/4038288654 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c018680 con 0x7f89200ffad0 2023-07-21T22:26:16.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.566+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89201a4c40 con 0x7f89200ffad0 2023-07-21T22:26:16.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.569+0000 7f89177fe700 1 -- 172.21.15.66:0/4038288654 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f891c0105a0 con 0x7f89200ffad0 2023-07-21T22:26:16.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 Processor -- start 2023-07-21T22:26:16.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 -- start start 2023-07-21T22:26:16.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f89177fe700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f890c0a12c0 0x7f890c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644314580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644314ac0 0x55f64430f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64430fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644214cb0 con 0x55f6443141b0 2023-07-21T22:26:16.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644267a40 con 0x55f64430f930 2023-07-21T22:26:16.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644260ed0 con 0x55f644314ac0 2023-07-21T22:26:16.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f89177fe700 1 -- 172.21.15.66:0/4038288654 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f891c014070 con 0x7f89200ffad0 2023-07-21T22:26:16.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f8926d9c700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f890c0a12c0 0x7f890c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.571+0000 7f892659b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644314580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644314ac0 0x55f64430f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f892659b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644314580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50450/0 (socket says 172.21.15.66:50450) 2023-07-21T22:26:16.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f892659b700 1 -- 172.21.15.66:0/2861349354 learned_addr learned my addr 172.21.15.66:0/2861349354 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8926d9c700 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64430fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 -- 172.21.15.66:0/2861349354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 msgr2=0x55f64430fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64430fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 -- 172.21.15.66:0/2861349354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 msgr2=0x55f644314580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644314580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 -- 172.21.15.66:0/2861349354 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f64408ede0 con 0x55f644314ac0 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f892659b700 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644314580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8926d9c700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f890c0a12c0 0x7f890c0a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8918009ec0 tx=0x7f8918016450 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.572+0000 7f8925d9a700 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644314ac0 0x55f64430f350 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f891000b2e0 tx=0x7f8910005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c39bdd074d350d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.573+0000 7f8914ff9700 1 -- 172.21.15.66:0/2861349354 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001d070 con 0x55f644314ac0 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.573+0000 7f8914ff9700 1 -- 172.21.15.66:0/2861349354 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891001e040 con 0x55f644314ac0 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.573+0000 7f8914ff9700 1 -- 172.21.15.66:0/2861349354 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910004720 con 0x55f644314ac0 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.573+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2861349354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644314ac0 msgr2=0x55f64430f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.573+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644314ac0 0x55f64430f350 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f891000b2e0 tx=0x7f8910005ea0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.573+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2861349354 shutdown_connections 2023-07-21T22:26:16.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64430fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644314580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2861349354 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644314ac0 0x55f64430f350 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2861349354 >> 172.21.15.66:0/2861349354 conn(0x55f644304ef0 msgr2=0x55f64430a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2861349354 shutdown_connections 2023-07-21T22:26:16.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2861349354 wait complete. 2023-07-21T22:26:16.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 -- start start 2023-07-21T22:26:16.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.574+0000 7f893c1c5b80 1 Processor -- start 2023-07-21T22:26:16.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f64431bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644314ac0 0x55f6443194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644200ac0 con 0x55f6443141b0 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6442007c0 con 0x55f644314ac0 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6440e4920 con 0x55f64430f930 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f893c1c5b80 10 client.0 _ll_register_callbacks cb 0x55f6441fe760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f8925d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f8925d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50458/0 (socket says 172.21.15.66:50458) 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f8925d9a700 1 -- 172.21.15.66:0/4264187350 learned_addr learned my addr 172.21.15.66:0/4264187350 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:16.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f892659b700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f64431bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.575+0000 7f8926d9c700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644314ac0 0x55f6443194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8925d9a700 1 -- 172.21.15.66:0/4264187350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 msgr2=0x55f64431bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8925d9a700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f64431bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8925d9a700 1 -- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644314ac0 msgr2=0x55f6443194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8925d9a700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644314ac0 0x55f6443194d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8925d9a700 1 -- 172.21.15.66:0/4264187350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89201a4c40 con 0x55f6443141b0 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8926d9c700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644314ac0 0x55f6443194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f892659b700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f64431bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8925d9a700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318e30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f89100098e0 tx=0x7f8910005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7956387a31d4c79e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8909ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8909ffb700 1 -- 172.21.15.66:0/4264187350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001d040 con 0x55f6443141b0 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8909ffb700 1 -- 172.21.15.66:0/4264187350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f891000f870 con 0x55f6443141b0 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6441ba4f0 con 0x55f6443141b0 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.576+0000 7f8909ffb700 1 -- 172.21.15.66:0/4264187350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f891000fa30 con 0x55f6443141b0 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 10 client.16464 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:16.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f892010d210 con 0x55f6443141b0 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f8909ffb700 1 -- 172.21.15.66:0/4264187350 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6441ba4f0 con 0x55f6443141b0 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f8909ffb700 1 -- 172.21.15.66:0/4264187350 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8910022400 con 0x55f6443141b0 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 20 client.16464 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644328f50 0x55f64432b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 4 client.16464 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f644259eb0 con 0x55f644328f50 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.577+0000 7f892659b700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644328f50 0x55f64432b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.578+0000 7f892659b700 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644328f50 0x55f64432b360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56e9d10ec5cd764 server_cookie=fffcc90cd9f79d79 in_seq=0 out_seq=0 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.578+0000 7f8909ffb700 10 client.16464 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f8909ffb700 1 -- 172.21.15.66:0/4264187350 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f644259eb0 con 0x55f644328f50 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f8909ffb700 10 client.16464 handle_command_reply: tid=0 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 client.16464 shutdown 2023-07-21T22:26:16.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 msgr2=0x55f644318e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318e30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f89100098e0 tx=0x7f8910005c40 comp rx=0 tx=0).stop 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 shutdown_connections 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644314ac0 0x55f6443194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644328f50 0x55f64432b360 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f64431bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/4264187350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318e30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f8909ffb700 0 client.16464 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:16.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 >> 172.21.15.66:0/4264187350 conn(0x55f644304ef0 msgr2=0x55f64430a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 shutdown_connections 2023-07-21T22:26:16.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 1 -- 172.21.15.66:0/4264187350 wait complete. 2023-07-21T22:26:16.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.579+0000 7f893c1c5b80 20 client.16464 trim_cache size 0 max 16384 2023-07-21T22:26:16.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.597+0000 7f893c1c5b80 1 Processor -- start 2023-07-21T22:26:16.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.597+0000 7f893c1c5b80 1 -- start start 2023-07-21T22:26:16.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.597+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644304ef0 0x55f64431b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.597+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f6443185f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.597+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64430be70 con 0x55f6443141b0 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6443106f0 con 0x55f644304ef0 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f644317390 con 0x55f64430f930 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f8926d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f892659b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644304ef0 0x55f64431b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f8926d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50474/0 (socket says 172.21.15.66:50474) 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f8926d9c700 1 -- 172.21.15.66:0/2496782797 learned_addr learned my addr 172.21.15.66:0/2496782797 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f8925d9a700 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f6443185f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f892659b700 1 -- 172.21.15.66:0/2496782797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 msgr2=0x55f6443185f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f892659b700 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f6443185f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f892659b700 1 -- 172.21.15.66:0/2496782797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 msgr2=0x55f644318b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f892659b700 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f892659b700 1 -- 172.21.15.66:0/2496782797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f644077c60 con 0x55f644304ef0 2023-07-21T22:26:16.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.598+0000 7f8926d9c700 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f892659b700 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644304ef0 0x55f64431b750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f891c096990 tx=0x7f891c0af040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f35e430b97de56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f89157fa700 1 -- 172.21.15.66:0/2496782797 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c099cf0 con 0x55f644304ef0 2023-07-21T22:26:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f89157fa700 1 -- 172.21.15.66:0/2496782797 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891c099880 con 0x55f644304ef0 2023-07-21T22:26:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f89157fa700 1 -- 172.21.15.66:0/2496782797 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0982c0 con 0x55f644304ef0 2023-07-21T22:26:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2496782797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644304ef0 msgr2=0x55f64431b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644304ef0 0x55f64431b750 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f891c096990 tx=0x7f891c0af040 comp rx=0 tx=0).stop 2023-07-21T22:26:16.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2496782797 shutdown_connections 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f644304ef0 0x55f64431b750 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f644318b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/2496782797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f64430f930 0x55f6443185f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.599+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2496782797 >> 172.21.15.66:0/2496782797 conn(0x55f644310060 msgr2=0x55f64430d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.600+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2496782797 shutdown_connections 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.600+0000 7f893c1c5b80 1 -- 172.21.15.66:0/2496782797 wait complete. 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.600+0000 7f893c1c5b80 1 -- start start 2023-07-21T22:26:16.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.600+0000 7f893c1c5b80 1 Processor -- start 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644304ef0 0x55f644320430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64431e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f64431eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64431f280 con 0x55f6443141b0 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64431f3f0 con 0x55f64430f930 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f64431f5b0 con 0x55f644304ef0 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f893c1c5b80 10 client.0 _ll_register_callbacks cb 0x7f891c096260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:16.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f8925d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64431e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f8926d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f64431eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f892659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644304ef0 0x55f644320430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f8925d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64431e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33320/0 (socket says 172.21.15.66:33320) 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.601+0000 7f8925d9a700 1 -- 172.21.15.66:0/477059246 learned_addr learned my addr 172.21.15.66:0/477059246 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8925d9a700 1 -- 172.21.15.66:0/477059246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644304ef0 msgr2=0x55f644320430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8925d9a700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644304ef0 0x55f644320430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8925d9a700 1 -- 172.21.15.66:0/477059246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 msgr2=0x55f64431eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8925d9a700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f64431eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8925d9a700 1 -- 172.21.15.66:0/477059246 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891801abe0 con 0x55f64430f930 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f892659b700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644304ef0 0x55f644320430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8926d9c700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f64431eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8925d9a700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64431e760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8910022790 tx=0x7f891000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5033027949068b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8909ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8909ffb700 1 -- 172.21.15.66:0/477059246 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891001d030 con 0x55f64430f930 2023-07-21T22:26:16.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f891001e040 con 0x55f64430f930 2023-07-21T22:26:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8909ffb700 1 -- 172.21.15.66:0/477059246 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8910000e40 con 0x55f64430f930 2023-07-21T22:26:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.602+0000 7f8909ffb700 1 -- 172.21.15.66:0/477059246 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8910022a80 con 0x55f64430f930 2023-07-21T22:26:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 10 client.26528 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f892019d400 con 0x55f64430f930 2023-07-21T22:26:16.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f8909ffb700 1 -- 172.21.15.66:0/477059246 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891001e040 con 0x55f64430f930 2023-07-21T22:26:16.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f8909ffb700 1 -- 172.21.15.66:0/477059246 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f891000fac0 con 0x55f64430f930 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 10 client.26528 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 20 client.26528 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644314ac0 0x55f64437ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 4 client.26528 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f64421c050 con 0x55f644314ac0 2023-07-21T22:26:16.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.603+0000 7f892659b700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644314ac0 0x55f64437ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f892659b700 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644314ac0 0x55f64437ff60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937581b5c15a0369 server_cookie=82054a8ecab0c99a in_seq=0 out_seq=0 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f8909ffb700 10 client.26528 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f8909ffb700 1 -- 172.21.15.66:0/477059246 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f64421c050 con 0x55f644314ac0 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f8909ffb700 10 client.26528 handle_command_reply: tid=0 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 client.26528 shutdown 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 msgr2=0x55f64431e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64431e760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8910022790 tx=0x7f891000b690 comp rx=0 tx=0).stop 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 shutdown_connections 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55f644314ac0 0x55f64437ff60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55f6443141b0 0x55f64431eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f8909ffb700 0 client.26528 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f644304ef0 0x55f644320430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 --2- 172.21.15.66:0/477059246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f64430f930 0x55f64431e760 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 >> 172.21.15.66:0/477059246 conn(0x55f644310060 msgr2=0x55f64430e940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 shutdown_connections 2023-07-21T22:26:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.604+0000 7f893c1c5b80 1 -- 172.21.15.66:0/477059246 wait complete. 2023-07-21T22:26:16.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f893c1c5b80 20 client.26528 trim_cache size 0 max 16384 2023-07-21T22:26:16.659 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:26:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:26:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:26:16.659 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:26:16.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f890c0a12c0 msgr2=0x7f890c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f890c0a12c0 0x7f890c0a3770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8918009ec0 tx=0x7f8918016450 comp rx=0 tx=0).stop 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 msgr2=0x7f8920197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 0x7f8920197e10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f891c00e9a0 tx=0x7f891c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 shutdown_connections 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8920105ae0 0x7f892019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f890c0a12c0 0x7f890c0a3770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f89200ffad0 0x7f8920197e10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 --2- 172.21.15.66:0/4038288654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8920100480 0x7f8920198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 >> 172.21.15.66:0/4038288654 conn(0x7f89200fb5f0 msgr2=0x7f892010ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.605+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 shutdown_connections 2023-07-21T22:26:16.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.606+0000 7f89287ff700 1 -- 172.21.15.66:0/4038288654 wait complete. 2023-07-21T22:26:16.669 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:26:16.670 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:26:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 -- 172.21.15.66:0/3308369175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5138100480 msgr2=0x7f513810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 --2- 172.21.15.66:0/3308369175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5138100480 0x7f513810c910 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f513000b0a0 tx=0x7f513000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 -- 172.21.15.66:0/3308369175 shutdown_connections 2023-07-21T22:26:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 --2- 172.21.15.66:0/3308369175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 0x7f5138105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 --2- 172.21.15.66:0/3308369175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5138100480 0x7f513810c910 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 --2- 172.21.15.66:0/3308369175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51380ffad0 0x7f51380fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 -- 172.21.15.66:0/3308369175 >> 172.21.15.66:0/3308369175 conn(0x7f51380fb590 msgr2=0x7f51380fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 -- 172.21.15.66:0/3308369175 shutdown_connections 2023-07-21T22:26:16.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.832+0000 7f513e5ff700 1 -- 172.21.15.66:0/3308369175 wait complete. 2023-07-21T22:26:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.834+0000 7f513e5ff700 1 Processor -- start 2023-07-21T22:26:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.834+0000 7f513e5ff700 1 -- start start 2023-07-21T22:26:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513e5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 0x7f5138197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513e5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5138100480 0x7f5138198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513e5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 0x7f513819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513e5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513819ceb0 con 0x7f5138100480 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513e5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513819d020 con 0x7f5138105ae0 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513e5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513819d1e0 con 0x7f51380ffad0 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 0x7f5138197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f51377fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5138100480 0x7f5138198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 0x7f5138197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33890/0 (socket says 172.21.15.66:33890) 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 -- 172.21.15.66:0/527392130 learned_addr learned my addr 172.21.15.66:0/527392130 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513cb9c700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 0x7f513819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 -- 172.21.15.66:0/527392130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 msgr2=0x7f513819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 0x7f513819c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 -- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5138100480 msgr2=0x7f5138198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5138100480 0x7f5138198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f5137fff700 1 -- 172.21.15.66:0/527392130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130009d00 con 0x7f51380ffad0 2023-07-21T22:26:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f51377fe700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5138100480 0x7f5138198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.835+0000 7f513cb9c700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 0x7f513819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.836+0000 7f5137fff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 0x7f5138197ed0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f512800e9a0 tx=0x7f512800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.836+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128009790 con 0x7f51380ffad0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.836+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513819d4c0 con 0x7f51380ffad0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.836+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5128010430 con 0x7f51380ffad0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.836+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128018680 con 0x7f51380ffad0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.836+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51381a4d00 con 0x7f51380ffad0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.837+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5138067300 con 0x7f51380ffad0 2023-07-21T22:26:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.839+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f51280105a0 con 0x7f51380ffad0 2023-07-21T22:26:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.841+0000 7f51357fa700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51200a1300 0x7f51200a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.841+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5128014070 con 0x7f51380ffad0 2023-07-21T22:26:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.841+0000 7f51377fe700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51200a1300 0x7f51200a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.842+0000 7f51377fe700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51200a1300 0x7f51200a37b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f51381994e0 tx=0x7f512c009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:16.843+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5128096430 con 0x7f51380ffad0 2023-07-21T22:26:17.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.098+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5138199220 con 0x7f51380ffad0 2023-07-21T22:26:17.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.099+0000 7f51357fa700 1 -- 172.21.15.66:0/527392130 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10820 (secure 0 0 0) 0x7f5128099a50 con 0x7f51380ffad0 2023-07-21T22:26:17.146 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:26:17.146 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:25:45.971950+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:26:17.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51200a1300 msgr2=0x7f51200a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51200a1300 0x7f51200a37b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f51381994e0 tx=0x7f512c009260 comp rx=0 tx=0).stop 2023-07-21T22:26:17.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 msgr2=0x7f5138197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 0x7f5138197ed0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f512800e9a0 tx=0x7f512800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 shutdown_connections 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5138105ae0 0x7f513819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51200a1300 0x7f51200a37b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51380ffad0 0x7f5138197ed0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 --2- 172.21.15.66:0/527392130 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5138100480 0x7f5138198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 >> 172.21.15.66:0/527392130 conn(0x7f51380fb590 msgr2=0x7f513810ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 shutdown_connections 2023-07-21T22:26:17.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.102+0000 7f513e5ff700 1 -- 172.21.15.66:0/527392130 wait complete. 2023-07-21T22:26:17.150 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 33 2023-07-21T22:26:17.167 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:26:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 -- 172.21.15.66:0/1297155506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 msgr2=0x7fac4c0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 --2- 172.21.15.66:0/1297155506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c0fff40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009990 tx=0x7fac3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 -- 172.21.15.66:0/1297155506 shutdown_connections 2023-07-21T22:26:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 --2- 172.21.15.66:0/1297155506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac4c105ae0 0x7fac4c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 --2- 172.21.15.66:0/1297155506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac4c100480 0x7fac4c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 --2- 172.21.15.66:0/1297155506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c0fff40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 -- 172.21.15.66:0/1297155506 >> 172.21.15.66:0/1297155506 conn(0x7fac4c0fb5f0 msgr2=0x7fac4c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 -- 172.21.15.66:0/1297155506 shutdown_connections 2023-07-21T22:26:17.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.330+0000 7fac533ff700 1 -- 172.21.15.66:0/1297155506 wait complete. 2023-07-21T22:26:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 Processor -- start 2023-07-21T22:26:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 -- start start 2023-07-21T22:26:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac4c100480 0x7fac4c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac4c105ae0 0x7fac4c1a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c073580 con 0x7fac4c0ffad0 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1a42c0 con 0x7fac4c100480 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.331+0000 7fac533ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c1a4400 con 0x7fac4c105ae0 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5199c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac4c105ae0 0x7fac4c1a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50522/0 (socket says 172.21.15.66:50522) 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 -- 172.21.15.66:0/1153157311 learned_addr learned my addr 172.21.15.66:0/1153157311 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5099a700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac4c100480 0x7fac4c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 -- 172.21.15.66:0/1153157311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac4c105ae0 msgr2=0x7fac4c1a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac4c105ae0 0x7fac4c1a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 -- 172.21.15.66:0/1153157311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac4c100480 msgr2=0x7fac4c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac4c100480 0x7fac4c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 -- 172.21.15.66:0/1153157311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c0095f0 con 0x7fac4c0ffad0 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.332+0000 7fac5119b700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c072810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fac4800d980 tx=0x7fac4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.333+0000 7fac427fc700 1 -- 172.21.15.66:0/1153157311 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48004220 con 0x7fac4c0ffad0 2023-07-21T22:26:17.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.333+0000 7fac427fc700 1 -- 172.21.15.66:0/1153157311 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac48007640 con 0x7fac4c0ffad0 2023-07-21T22:26:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.333+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c1a4660 con 0x7fac4c0ffad0 2023-07-21T22:26:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.333+0000 7fac427fc700 1 -- 172.21.15.66:0/1153157311 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48005690 con 0x7fac4c0ffad0 2023-07-21T22:26:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.333+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c1a4b90 con 0x7fac4c0ffad0 2023-07-21T22:26:17.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.336+0000 7fac427fc700 1 -- 172.21.15.66:0/1153157311 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fac48004750 con 0x7fac4c0ffad0 2023-07-21T22:26:17.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.337+0000 7fac66dccb80 1 Processor -- start 2023-07-21T22:26:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac427fc700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac380a1340 0x7fac380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 -- start start 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac427fc700 1 -- 172.21.15.66:0/1153157311 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fac480ccd90 con 0x7fac4c0ffad0 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac5199c700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac380a1340 0x7fac380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3a1b0 0x564422f3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 0x564422f35350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f35da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422e3acb0 con 0x564422f35930 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422e8da40 con 0x564422f3a1b0 2023-07-21T22:26:17.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac66dccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422e86ed0 con 0x564422f3aac0 2023-07-21T22:26:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 0x564422f35350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac5199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f35da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 0x564422f35350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33908/0 (socket says 172.21.15.66:33908) 2023-07-21T22:26:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.338+0000 7fac5099a700 1 -- 172.21.15.66:0/1490903633 learned_addr learned my addr 172.21.15.66:0/1490903633 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5119b700 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3a1b0 0x564422f3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5099a700 1 -- 172.21.15.66:0/1490903633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3a1b0 msgr2=0x564422f3a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5099a700 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3a1b0 0x564422f3a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5099a700 1 -- 172.21.15.66:0/1490903633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 msgr2=0x564422f35da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5099a700 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f35da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5099a700 1 -- 172.21.15.66:0/1490903633 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564422cb4de0 con 0x564422f3aac0 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5199c700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac380a1340 0x7fac380a37f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac4c074920 tx=0x7fac4400fbe0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.339+0000 7fac5099a700 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 0x564422f35350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009990 tx=0x7fac3c005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b7416d25756d65 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.340+0000 7fac377fe700 1 -- 172.21.15.66:0/1490903633 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c01d070 con 0x564422f3aac0 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.340+0000 7fac377fe700 1 -- 172.21.15.66:0/1490903633 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c01e040 con 0x564422f3aac0 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.340+0000 7fac377fe700 1 -- 172.21.15.66:0/1490903633 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c004900 con 0x564422f3aac0 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.340+0000 7fac66dccb80 1 -- 172.21.15.66:0/1490903633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 msgr2=0x564422f35350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.340+0000 7fac66dccb80 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 0x564422f35350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009990 tx=0x7fac3c005f00 comp rx=0 tx=0).stop 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 -- 172.21.15.66:0/1490903633 shutdown_connections 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3a1b0 0x564422f3a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f35da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 --2- 172.21.15.66:0/1490903633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3aac0 0x564422f35350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 -- 172.21.15.66:0/1490903633 >> 172.21.15.66:0/1490903633 conn(0x564422f2aef0 msgr2=0x564422f30830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 -- 172.21.15.66:0/1490903633 shutdown_connections 2023-07-21T22:26:17.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 -- 172.21.15.66:0/1490903633 wait complete. 2023-07-21T22:26:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 -- start start 2023-07-21T22:26:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.341+0000 7fac66dccb80 1 Processor -- start 2023-07-21T22:26:17.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f41b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3a1b0 0x564422f3edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 0x564422f3f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422e26ac0 con 0x564422f35930 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422e267c0 con 0x564422f3aac0 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422d0a920 con 0x564422f3a1b0 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac66dccb80 10 client.0 _ll_register_callbacks cb 0x564422e24760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f41b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5199c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 0x564422f3f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3a1b0 0x564422f3edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3a1b0 0x564422f3edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33910/0 (socket says 172.21.15.66:33910) 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5119b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f41b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50534/0 (socket says 172.21.15.66:50534) 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5099a700 1 -- 172.21.15.66:0/422375605 learned_addr learned my addr 172.21.15.66:0/422375605 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.342+0000 7fac5199c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 0x564422f3f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33360/0 (socket says 172.21.15.66:33360) 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5199c700 1 -- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3a1b0 msgr2=0x564422f3edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5199c700 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3a1b0 0x564422f3edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5199c700 1 -- 172.21.15.66:0/422375605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 msgr2=0x564422f41b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5199c700 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f41b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5199c700 1 -- 172.21.15.66:0/422375605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c10d210 con 0x564422f3aac0 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5119b700 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f41b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac5199c700 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 0x564422f3f470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac440060b0 tx=0x7fac44004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7a5fea2674c8239 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.343+0000 7fac34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac34ff9700 1 -- 172.21.15.66:0/422375605 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4400e040 con 0x564422f3aac0 2023-07-21T22:26:17.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564422de04f0 con 0x564422f3aac0 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac34ff9700 1 -- 172.21.15.66:0/422375605 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fac44003c00 con 0x564422f3aac0 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac34ff9700 1 -- 172.21.15.66:0/422375605 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac44009410 con 0x564422f3aac0 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 10 client.26546 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564422c9dc60 con 0x564422f3aac0 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac34ff9700 1 -- 172.21.15.66:0/422375605 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564422de04f0 con 0x564422f3aac0 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac34ff9700 1 -- 172.21.15.66:0/422375605 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fac4402a030 con 0x564422f3aac0 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 10 client.26546 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:17.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 10 client.26546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 20 client.26546 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.344+0000 7fac66dccb80 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f4f450 0x564422f51860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.345+0000 7fac66dccb80 4 client.26546 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.345+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564422e7feb0 con 0x564422f4f450 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.345+0000 7fac5119b700 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f4f450 0x564422f51860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.345+0000 7fac5119b700 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f4f450 0x564422f51860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdc1683fd243b255 server_cookie=41cac1f6c5de7ed6 in_seq=0 out_seq=0 2023-07-21T22:26:17.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.345+0000 7fac34ff9700 10 client.26546 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:26:17.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac34ff9700 1 -- 172.21.15.66:0/422375605 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564422e7feb0 con 0x564422f4f450 2023-07-21T22:26:17.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac34ff9700 10 client.26546 handle_command_reply: tid=0 2023-07-21T22:26:17.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 client.26546 shutdown 2023-07-21T22:26:17.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 msgr2=0x564422f3f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 0x564422f3f470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac440060b0 tx=0x7fac44004b40 comp rx=0 tx=0).stop 2023-07-21T22:26:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 shutdown_connections 2023-07-21T22:26:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f4f450 0x564422f51860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f3aac0 0x564422f3f470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f35930 0x564422f41b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac34ff9700 0 client.26546 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:26:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 --2- 172.21.15.66:0/422375605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f3a1b0 0x564422f3edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.346+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 >> 172.21.15.66:0/422375605 conn(0x564422f2aef0 msgr2=0x564422f30420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.347+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 shutdown_connections 2023-07-21T22:26:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.347+0000 7fac66dccb80 1 -- 172.21.15.66:0/422375605 wait complete. 2023-07-21T22:26:17.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.347+0000 7fac66dccb80 20 client.26546 trim_cache size 0 max 16384 2023-07-21T22:26:17.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.360+0000 7fac66dccb80 1 Processor -- start 2023-07-21T22:26:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.360+0000 7fac66dccb80 1 -- start start 2023-07-21T22:26:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.360+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f2aef0 0x564422f41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.360+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 0x564422f302d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.360+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422f30810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac66dccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422f31e70 con 0x564422f3a1b0 2023-07-21T22:26:17.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac66dccb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422f366d0 con 0x564422f2aef0 2023-07-21T22:26:17.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac66dccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422f403d0 con 0x564422f35930 2023-07-21T22:26:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 0x564422f302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422f30810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 0x564422f302d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33926/0 (socket says 172.21.15.66:33926) 2023-07-21T22:26:17.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 -- 172.21.15.66:0/602321712 learned_addr learned my addr 172.21.15.66:0/602321712 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5119b700 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f2aef0 0x564422f41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 -- 172.21.15.66:0/602321712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f2aef0 msgr2=0x564422f41710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f2aef0 0x564422f41710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 -- 172.21.15.66:0/602321712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 msgr2=0x564422f30810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422f30810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5099a700 1 -- 172.21.15.66:0/602321712 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564422f35fd0 con 0x564422f35930 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5119b700 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f2aef0 0x564422f41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:26:17.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.361+0000 7fac5199c700 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422f30810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.362+0000 7fac5099a700 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 0x564422f302d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00b3d0 tx=0x7fac3c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=470f4486da7bf1dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.362+0000 7fac37fff700 1 -- 172.21.15.66:0/602321712 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c01d040 con 0x564422f35930 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.362+0000 7fac37fff700 1 -- 172.21.15.66:0/602321712 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c0059d0 con 0x564422f35930 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.362+0000 7fac37fff700 1 -- 172.21.15.66:0/602321712 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c004be0 con 0x564422f35930 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.362+0000 7fac66dccb80 1 -- 172.21.15.66:0/602321712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 msgr2=0x564422f302d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.362+0000 7fac66dccb80 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 0x564422f302d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00b3d0 tx=0x7fac3c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 -- 172.21.15.66:0/602321712 shutdown_connections 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f2aef0 0x564422f41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422f30810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 --2- 172.21.15.66:0/602321712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f35930 0x564422f302d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 -- 172.21.15.66:0/602321712 >> 172.21.15.66:0/602321712 conn(0x564422f4bb40 msgr2=0x564422f33a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 -- 172.21.15.66:0/602321712 shutdown_connections 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 -- 172.21.15.66:0/602321712 wait complete. 2023-07-21T22:26:17.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 -- start start 2023-07-21T22:26:17.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.363+0000 7fac66dccb80 1 Processor -- start 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 0x564422f45690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f35930 0x564422f45bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422fa2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422f464b0 con 0x564422f3a1b0 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422fa2850 con 0x564422f35930 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422fa2990 con 0x564422f2aef0 2023-07-21T22:26:17.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac66dccb80 10 client.0 _ll_register_callbacks cb 0x7fac44009040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 0x564422f45690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5199c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422fa2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 0x564422f45690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33928/0 (socket says 172.21.15.66:33928) 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5099a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f35930 0x564422f45bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 -- 172.21.15.66:0/2038317329 learned_addr learned my addr 172.21.15.66:0/2038317329 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 -- 172.21.15.66:0/2038317329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f35930 msgr2=0x564422f45bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f35930 0x564422f45bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 -- 172.21.15.66:0/2038317329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 msgr2=0x564422fa2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.364+0000 7fac5119b700 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422fa2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac5119b700 1 -- 172.21.15.66:0/2038317329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564422c9dc60 con 0x564422f2aef0 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac5119b700 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 0x564422f45690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac48007fd0 tx=0x7fac4809d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70355458399524c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac34ff9700 1 -- 172.21.15.66:0/2038317329 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac480a1f00 con 0x564422f2aef0 2023-07-21T22:26:17.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac34ff9700 1 -- 172.21.15.66:0/2038317329 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fac480b2070 con 0x564422f2aef0 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac3c01e040 con 0x564422f2aef0 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac34ff9700 1 -- 172.21.15.66:0/2038317329 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac480ca3b0 con 0x564422f2aef0 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac66dccb80 10 client.26527 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x564422cb4de0 con 0x564422f2aef0 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.365+0000 7fac34ff9700 1 -- 172.21.15.66:0/2038317329 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c01e040 con 0x564422f2aef0 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac34ff9700 1 -- 172.21.15.66:0/2038317329 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fac48098fa0 con 0x564422f2aef0 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac66dccb80 10 client.26527 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:17.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac66dccb80 10 client.26527 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac66dccb80 20 client.26527 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac66dccb80 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f3aac0 0x564422fa5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac66dccb80 4 client.26527 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564422e42050 con 0x564422f3aac0 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.366+0000 7fac5099a700 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f3aac0 0x564422fa5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac34ff9700 10 client.26527 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac5099a700 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f3aac0 0x564422fa5e70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b42896ec58f96bda server_cookie=498b5bd612485fe5 in_seq=0 out_seq=0 2023-07-21T22:26:17.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac34ff9700 1 -- 172.21.15.66:0/2038317329 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564422e42050 con 0x564422f3aac0 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac34ff9700 10 client.26527 handle_command_reply: tid=0 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 client.26527 shutdown 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 msgr2=0x564422f45690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 0x564422f45690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac48007fd0 tx=0x7fac4809d040 comp rx=0 tx=0).stop 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 shutdown_connections 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x564422f3aac0 0x564422fa5e70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564422f3a1b0 0x564422fa2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564422f2aef0 0x564422f45690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac34ff9700 0 client.26527 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 --2- 172.21.15.66:0/2038317329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564422f35930 0x564422f45bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 >> 172.21.15.66:0/2038317329 conn(0x564422f4bb40 msgr2=0x564422f302b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 shutdown_connections 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.367+0000 7fac66dccb80 1 -- 172.21.15.66:0/2038317329 wait complete. 2023-07-21T22:26:17.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac66dccb80 20 client.26527 trim_cache size 0 max 16384 2023-07-21T22:26:17.422 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac380a1340 msgr2=0x7fac380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac380a1340 0x7fac380a37f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac4c074920 tx=0x7fac4400fbe0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 msgr2=0x7fac4c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c072810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fac4800d980 tx=0x7fac4800dc90 comp rx=0 tx=0).stop 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 shutdown_connections 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fac380a1340 0x7fac380a37f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fac4c105ae0 0x7fac4c1a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fac4c0ffad0 0x7fac4c072810 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 --2- 172.21.15.66:0/1153157311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac4c100480 0x7fac4c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 >> 172.21.15.66:0/1153157311 conn(0x7fac4c0fb5f0 msgr2=0x7fac4c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 shutdown_connections 2023-07-21T22:26:17.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.368+0000 7fac533ff700 1 -- 172.21.15.66:0/1153157311 wait complete. 2023-07-21T22:26:17.433 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:26:17.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.78 MB/sec execute 104 sec latency 10192.995 ms 2023-07-21T22:26:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.606+0000 7f7489bff700 1 -- 172.21.15.66:0/2911940700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7484100ca0 msgr2=0x7f7484101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.606+0000 7f7489bff700 1 --2- 172.21.15.66:0/2911940700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7484100ca0 0x7f7484101070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7474009990 tx=0x7f7474009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 -- 172.21.15.66:0/2911940700 shutdown_connections 2023-07-21T22:26:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 --2- 172.21.15.66:0/2911940700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f748410a070 0x7f748410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 --2- 172.21.15.66:0/2911940700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7484100ca0 0x7f7484101070 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 --2- 172.21.15.66:0/2911940700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74841015b0 0x7f7484109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 -- 172.21.15.66:0/2911940700 >> 172.21.15.66:0/2911940700 conn(0x7f74840fa4b0 msgr2=0x7f74840fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 -- 172.21.15.66:0/2911940700 shutdown_connections 2023-07-21T22:26:17.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.607+0000 7f7489bff700 1 -- 172.21.15.66:0/2911940700 wait complete. 2023-07-21T22:26:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 Processor -- start 2023-07-21T22:26:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 -- start start 2023-07-21T22:26:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 0x7f748419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74841015b0 0x7f748419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f748410a070 0x7f7484195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484196100 con 0x7f74841015b0 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484196270 con 0x7f7484100ca0 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.608+0000 7f7489bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484196430 con 0x7f748410a070 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 0x7f748419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 0x7f748419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33398/0 (socket says 172.21.15.66:33398) 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 -- 172.21.15.66:0/698189786 learned_addr learned my addr 172.21.15.66:0/698189786 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f7482ffd700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74841015b0 0x7f748419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f7483fff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f748410a070 0x7f7484195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 -- 172.21.15.66:0/698189786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f748410a070 msgr2=0x7f7484195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f748410a070 0x7f7484195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 -- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74841015b0 msgr2=0x7f748419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74841015b0 0x7f748419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 -- 172.21.15.66:0/698189786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740095f0 con 0x7f7484100ca0 2023-07-21T22:26:17.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.609+0000 7f74837fe700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 0x7f748419ba20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7474005910 tx=0x7f7474004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.610+0000 7f7480ff9700 1 -- 172.21.15.66:0/698189786 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747401d070 con 0x7f7484100ca0 2023-07-21T22:26:17.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.610+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74841966b0 con 0x7f7484100ca0 2023-07-21T22:26:17.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.610+0000 7f7480ff9700 1 -- 172.21.15.66:0/698189786 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7474022470 con 0x7f7484100ca0 2023-07-21T22:26:17.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.610+0000 7f7480ff9700 1 -- 172.21.15.66:0/698189786 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474017650 con 0x7f7484100ca0 2023-07-21T22:26:17.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.610+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841a2a10 con 0x7f7484100ca0 2023-07-21T22:26:17.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.613+0000 7f7480ff9700 1 -- 172.21.15.66:0/698189786 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f74740225e0 con 0x7f7484100ca0 2023-07-21T22:26:17.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.614+0000 7f749d504b80 1 Processor -- start 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.614+0000 7f749d504b80 1 -- start start 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f749d504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d6057a0 0x56092d605b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f749d504b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f749d504b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d60fe50 0x56092d612230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f749d504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d506cb0 con 0x56092d6057a0 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f749d504b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d559a40 con 0x56092d60fe50 2023-07-21T22:26:17.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f749d504b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d552ed0 con 0x56092d6060b0 2023-07-21T22:26:17.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7483fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d6057a0 0x56092d605b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7480ff9700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f746c0a12c0 0x7f746c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f74837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7483fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d6057a0 0x56092d605b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50574/0 (socket says 172.21.15.66:50574) 2023-07-21T22:26:17.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7483fff700 1 -- 172.21.15.66:0/1792214602 learned_addr learned my addr 172.21.15.66:0/1792214602 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7482ffd700 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d60fe50 0x56092d612230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7480ff9700 1 -- 172.21.15.66:0/698189786 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f74740ce2e0 con 0x7f7484100ca0 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.615+0000 7f7483fff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f746c0a12c0 0x7f746c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f74837fe700 1 -- 172.21.15.66:0/1792214602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d60fe50 msgr2=0x56092d612230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f74837fe700 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d60fe50 0x56092d612230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f74837fe700 1 -- 172.21.15.66:0/1792214602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d6057a0 msgr2=0x56092d605b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f74837fe700 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d6057a0 0x56092d605b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f74837fe700 1 -- 172.21.15.66:0/1792214602 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56092d380de0 con 0x56092d6060b0 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f74837fe700 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60f910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74740cb650 tx=0x7f7474009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f0ebfb5865dee1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f7483fff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f746c0a12c0 0x7f746c0a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56092d5fdf70 tx=0x7f747c004b40 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f7469ffb700 1 -- 172.21.15.66:0/1792214602 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747409abd0 con 0x56092d6060b0 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f7469ffb700 1 -- 172.21.15.66:0/1792214602 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747409a9f0 con 0x56092d6060b0 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.616+0000 7f7469ffb700 1 -- 172.21.15.66:0/1792214602 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747409b2f0 con 0x56092d6060b0 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 -- 172.21.15.66:0/1792214602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 msgr2=0x56092d60f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60f910 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f74740cb650 tx=0x7f7474009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 -- 172.21.15.66:0/1792214602 shutdown_connections 2023-07-21T22:26:17.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d60fe50 0x56092d612230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d6057a0 0x56092d605b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 --2- 172.21.15.66:0/1792214602 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60f910 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 -- 172.21.15.66:0/1792214602 >> 172.21.15.66:0/1792214602 conn(0x56092d5f6ef0 msgr2=0x56092d5fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 -- 172.21.15.66:0/1792214602 shutdown_connections 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.617+0000 7f749d504b80 1 -- 172.21.15.66:0/1792214602 wait complete. 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.618+0000 7f749d504b80 1 -- start start 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.618+0000 7f749d504b80 1 Processor -- start 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d613e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d614370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d4f2ac0 con 0x56092d60fe50 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d4f27c0 con 0x56092d6060b0 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d3d6920 con 0x56092d6057a0 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f749d504b80 10 client.0 _ll_register_callbacks cb 0x56092d4f0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f7483fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f74837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d613e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f7482ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d614370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f7483fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50582/0 (socket says 172.21.15.66:50582) 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f74837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d613e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33970/0 (socket says 172.21.15.66:33970) 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f7482ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d614370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33416/0 (socket says 172.21.15.66:33416) 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.619+0000 7f7483fff700 1 -- 172.21.15.66:0/1348447319 learned_addr learned my addr 172.21.15.66:0/1348447319 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7483fff700 1 -- 172.21.15.66:0/1348447319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 msgr2=0x56092d613e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7483fff700 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d613e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7483fff700 1 -- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 msgr2=0x56092d614370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7483fff700 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d614370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7483fff700 1 -- 172.21.15.66:0/1348447319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74841a2a10 con 0x56092d60fe50 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f74837fe700 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d613e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7483fff700 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60edf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f747c013fd0 tx=0x7f747c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d399a94dcc114416 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7462ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7462ffd700 1 -- 172.21.15.66:0/1348447319 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e040 con 0x56092d60fe50 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56092d4ac4f0 con 0x56092d60fe50 2023-07-21T22:26:17.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.620+0000 7f7462ffd700 1 -- 172.21.15.66:0/1348447319 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f747c003a00 con 0x56092d60fe50 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f7462ffd700 1 -- 172.21.15.66:0/1348447319 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747c003bc0 con 0x56092d60fe50 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 10 client.16506 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56092d369c60 con 0x56092d60fe50 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f7462ffd700 1 -- 172.21.15.66:0/1348447319 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c030020 con 0x56092d60fe50 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f7462ffd700 1 -- 172.21.15.66:0/1348447319 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f747c027460 con 0x56092d60fe50 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 10 client.16506 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:17.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 10 client.16506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:26:17.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 20 client.16506 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:17.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d61f430 0x56092d621840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 4 client.16506 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:17.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.621+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56092d54beb0 con 0x56092d61f430 2023-07-21T22:26:17.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.622+0000 7f74837fe700 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d61f430 0x56092d621840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.622+0000 7f74837fe700 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d61f430 0x56092d621840 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ddb13481c32c145 server_cookie=43f31c2b0bb171ba in_seq=0 out_seq=0 2023-07-21T22:26:17.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.622+0000 7f7462ffd700 10 client.16506 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:26:17.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f7462ffd700 1 -- 172.21.15.66:0/1348447319 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56092d54beb0 con 0x56092d61f430 2023-07-21T22:26:17.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f7462ffd700 10 client.16506 handle_command_reply: tid=0 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 client.16506 shutdown 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 msgr2=0x56092d60edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60edf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f747c013fd0 tx=0x7f747c004ad0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 shutdown_connections 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60edf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d613e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d61f430 0x56092d621840 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 --2- 172.21.15.66:0/1348447319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d614370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 >> 172.21.15.66:0/1348447319 conn(0x56092d5f6ef0 msgr2=0x56092d5fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f7462ffd700 0 client.16506 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:26:17.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 shutdown_connections 2023-07-21T22:26:17.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 1 -- 172.21.15.66:0/1348447319 wait complete. 2023-07-21T22:26:17.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.623+0000 7f749d504b80 20 client.16506 trim_cache size 0 max 16384 2023-07-21T22:26:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 Processor -- start 2023-07-21T22:26:17.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 -- start start 2023-07-21T22:26:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6057a0 0x56092d6098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d615220 con 0x56092d60fe50 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d5fddc0 con 0x56092d6057a0 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.637+0000 7f749d504b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d60fb30 con 0x56092d6060b0 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50588/0 (socket says 172.21.15.66:50588) 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 -- 172.21.15.66:0/1292664722 learned_addr learned my addr 172.21.15.66:0/1292664722 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7482ffd700 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f74837fe700 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6057a0 0x56092d6098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 -- 172.21.15.66:0/1292664722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 msgr2=0x56092d60a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 -- 172.21.15.66:0/1292664722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6057a0 msgr2=0x56092d6098a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6057a0 0x56092d6098a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 -- 172.21.15.66:0/1292664722 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56092d3a7250 con 0x56092d60fe50 2023-07-21T22:26:17.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.638+0000 7f7483fff700 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60a660 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f747c027810 tx=0x7f747c013a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e483e8bb8e3cf53c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f746a7fc700 1 -- 172.21.15.66:0/1292664722 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e030 con 0x56092d60fe50 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f746a7fc700 1 -- 172.21.15.66:0/1292664722 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c030030 con 0x56092d60fe50 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f746a7fc700 1 -- 172.21.15.66:0/1292664722 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c007ea0 con 0x56092d60fe50 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 -- 172.21.15.66:0/1292664722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 msgr2=0x56092d60a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60a660 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f747c027810 tx=0x7f747c013a10 comp rx=0 tx=0).stop 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 -- 172.21.15.66:0/1292664722 shutdown_connections 2023-07-21T22:26:17.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6057a0 0x56092d6098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d60a660 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 --2- 172.21.15.66:0/1292664722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6060b0 0x56092d60a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 -- 172.21.15.66:0/1292664722 >> 172.21.15.66:0/1292664722 conn(0x56092d5f6ef0 msgr2=0x56092d614b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 -- 172.21.15.66:0/1292664722 shutdown_connections 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.639+0000 7f749d504b80 1 -- 172.21.15.66:0/1292664722 wait complete. 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 -- start start 2023-07-21T22:26:17.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 Processor -- start 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d61b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d617c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d6181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d618700 con 0x56092d60fe50 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d618870 con 0x56092d6060b0 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.640+0000 7f749d504b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56092d618a30 con 0x56092d6057a0 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f749d504b80 10 client.0 _ll_register_callbacks cb 0x56092d4f0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:17.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d61b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d61b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33992/0 (socket says 172.21.15.66:33992) 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 -- 172.21.15.66:0/3200066521 learned_addr learned my addr 172.21.15.66:0/3200066521 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f7482ffd700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d617c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f7483fff700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d6181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 -- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 msgr2=0x56092d617c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d617c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 -- 172.21.15.66:0/3200066521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 msgr2=0x56092d6181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d6181c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 -- 172.21.15.66:0/3200066521 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56092d369c60 con 0x56092d6057a0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f74837fe700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d61b6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7474024000 tx=0x7f747400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdbeffc4961a8dc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f7462ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.641+0000 7f7462ffd700 1 -- 172.21.15.66:0/3200066521 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74740ce120 con 0x56092d6057a0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f7462ffd700 1 -- 172.21.15.66:0/3200066521 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f747409b090 con 0x56092d6057a0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f747409a9f0 con 0x56092d6057a0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f7462ffd700 1 -- 172.21.15.66:0/3200066521 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7474022900 con 0x56092d6057a0 2023-07-21T22:26:17.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 10 client.26545 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56092d37f2b0 con 0x56092d6057a0 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f7462ffd700 1 -- 172.21.15.66:0/3200066521 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747409a9f0 con 0x56092d6057a0 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f7462ffd700 1 -- 172.21.15.66:0/3200066521 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74740a2fc0 con 0x56092d6057a0 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 10 client.26545 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 10 client.26545 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 20 client.26545 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:17.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d6741c0 0x56092d6765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 4 client.26545 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:26:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.642+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56092d50e050 con 0x56092d6741c0 2023-07-21T22:26:17.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f7482ffd700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d6741c0 0x56092d6765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f7482ffd700 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d6741c0 0x56092d6765a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fc78f47ce02aba8 server_cookie=e783f493f2502e65 in_seq=0 out_seq=0 2023-07-21T22:26:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f7462ffd700 10 client.26545 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:26:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f7462ffd700 1 -- 172.21.15.66:0/3200066521 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56092d50e050 con 0x56092d6741c0 2023-07-21T22:26:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f7462ffd700 10 client.26545 handle_command_reply: tid=0 2023-07-21T22:26:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 client.26545 shutdown 2023-07-21T22:26:17.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 msgr2=0x56092d61b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d61b6e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7474024000 tx=0x7f747400b610 comp rx=0 tx=0).stop 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 shutdown_connections 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56092d60fe50 0x56092d6181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56092d6057a0 0x56092d61b6e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x56092d6741c0 0x56092d6765a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 --2- 172.21.15.66:0/3200066521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56092d6060b0 0x56092d617c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f7462ffd700 0 client.26545 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:26:17.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.643+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 >> 172.21.15.66:0/3200066521 conn(0x56092d5f6ef0 msgr2=0x56092d60a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 shutdown_connections 2023-07-21T22:26:17.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f749d504b80 1 -- 172.21.15.66:0/3200066521 wait complete. 2023-07-21T22:26:17.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f749d504b80 20 client.26545 trim_cache size 0 max 16384 2023-07-21T22:26:17.698 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:26:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f746c0a12c0 msgr2=0x7f746c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f746c0a12c0 0x7f746c0a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56092d5fdf70 tx=0x7f747c004b40 comp rx=0 tx=0).stop 2023-07-21T22:26:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 msgr2=0x7f748419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 0x7f748419ba20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7474005910 tx=0x7f7474004710 comp rx=0 tx=0).stop 2023-07-21T22:26:17.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 shutdown_connections 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7484100ca0 0x7f748419ba20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f746c0a12c0 0x7f746c0a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f748410a070 0x7f7484195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 --2- 172.21.15.66:0/698189786 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74841015b0 0x7f748419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.644+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 >> 172.21.15.66:0/698189786 conn(0x7f74840fa4b0 msgr2=0x7f748410c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.645+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 shutdown_connections 2023-07-21T22:26:17.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.645+0000 7f7489bff700 1 -- 172.21.15.66:0/698189786 wait complete. 2023-07-21T22:26:17.708 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:26:17.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.871+0000 7fab339ff700 1 -- 172.21.15.66:0/3449623273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 msgr2=0x7fab2c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.871+0000 7fab339ff700 1 --2- 172.21.15.66:0/3449623273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 0x7fab2c072c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab28009990 tx=0x7fab28009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.871+0000 7fab339ff700 1 -- 172.21.15.66:0/3449623273 shutdown_connections 2023-07-21T22:26:17.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.871+0000 7fab339ff700 1 --2- 172.21.15.66:0/3449623273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab2c10c310 0x7fab2c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.871+0000 7fab339ff700 1 --2- 172.21.15.66:0/3449623273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 0x7fab2c072c80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.872+0000 7fab339ff700 1 --2- 172.21.15.66:0/3449623273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.872+0000 7fab339ff700 1 -- 172.21.15.66:0/3449623273 >> 172.21.15.66:0/3449623273 conn(0x7fab2c0fb5f0 msgr2=0x7fab2c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.872+0000 7fab339ff700 1 -- 172.21.15.66:0/3449623273 shutdown_connections 2023-07-21T22:26:17.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.872+0000 7fab339ff700 1 -- 172.21.15.66:0/3449623273 wait complete. 2023-07-21T22:26:17.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.873+0000 7fab339ff700 1 Processor -- start 2023-07-21T22:26:17.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.873+0000 7fab339ff700 1 -- start start 2023-07-21T22:26:17.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab339ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 0x7fab2c197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab339ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab339ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab2c10c310 0x7fab2c19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab339ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19cdf0 con 0x7fab2c10c310 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab339ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19cf60 con 0x7fab2c0731c0 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab339ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c19d120 con 0x7fab2c0728b0 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab3179b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 0x7fab2c197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33438/0 (socket says 172.21.15.66:33438) 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab30f9a700 1 -- 172.21.15.66:0/1360500017 learned_addr learned my addr 172.21.15.66:0/1360500017 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.874+0000 7fab31f9c700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab2c10c310 0x7fab2c19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.875+0000 7fab30f9a700 1 -- 172.21.15.66:0/1360500017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 msgr2=0x7fab2c197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.875+0000 7fab30f9a700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 0x7fab2c197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.875+0000 7fab30f9a700 1 -- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab2c10c310 msgr2=0x7fab2c19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.875+0000 7fab30f9a700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab2c10c310 0x7fab2c19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.875+0000 7fab30f9a700 1 -- 172.21.15.66:0/1360500017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab280095f0 con 0x7fab2c0731c0 2023-07-21T22:26:17.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.875+0000 7fab30f9a700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c198350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d980 tx=0x7fab1c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.878+0000 7fab227fc700 1 -- 172.21.15.66:0/1360500017 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c004220 con 0x7fab2c0731c0 2023-07-21T22:26:17.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.878+0000 7fab227fc700 1 -- 172.21.15.66:0/1360500017 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c007640 con 0x7fab2c0731c0 2023-07-21T22:26:17.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.878+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c19d400 con 0x7fab2c0731c0 2023-07-21T22:26:17.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.878+0000 7fab227fc700 1 -- 172.21.15.66:0/1360500017 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c005690 con 0x7fab2c0731c0 2023-07-21T22:26:17.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.878+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1a4c40 con 0x7fab2c0731c0 2023-07-21T22:26:17.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.881+0000 7fab227fc700 1 -- 172.21.15.66:0/1360500017 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fab1c004750 con 0x7fab2c0731c0 2023-07-21T22:26:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab227fc700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fab180a1340 0x7fab180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 Processor -- start 2023-07-21T22:26:17.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab227fc700 1 -- 172.21.15.66:0/1360500017 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fab1c0cce20 con 0x7fab2c0731c0 2023-07-21T22:26:17.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab3179b700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fab180a1340 0x7fab180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 -- start start 2023-07-21T22:26:17.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b0206e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02069350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02069da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b01f6ecb0 con 0x558b0206e1b0 2023-07-21T22:26:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b01fc1a40 con 0x558b0206eac0 2023-07-21T22:26:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.883+0000 7fab473ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b01fbaed0 con 0x558b02069930 2023-07-21T22:26:17.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab3179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b0206e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab31f9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02069da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02069350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33450/0 (socket says 172.21.15.66:33450) 2023-07-21T22:26:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 -- 172.21.15.66:0/2356089143 learned_addr learned my addr 172.21.15.66:0/2356089143 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab3179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b0206e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50628/0 (socket says 172.21.15.66:50628) 2023-07-21T22:26:17.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab3179b700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fab180a1340 0x7fab180a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b2e0 tx=0x7fab28005fd0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 -- 172.21.15.66:0/2356089143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 msgr2=0x558b02069da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02069da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 -- 172.21.15.66:0/2356089143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 msgr2=0x558b0206e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b0206e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 -- 172.21.15.66:0/2356089143 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b01de8de0 con 0x558b0206eac0 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.884+0000 7fab30f9a700 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02069350 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0965b0 tx=0x7fab1c022420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dc335c17f2b6d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab177fe700 1 -- 172.21.15.66:0/2356089143 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0a1f90 con 0x558b0206eac0 2023-07-21T22:26:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab177fe700 1 -- 172.21.15.66:0/2356089143 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c004380 con 0x558b0206eac0 2023-07-21T22:26:17.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab177fe700 1 -- 172.21.15.66:0/2356089143 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0a1b00 con 0x558b0206eac0 2023-07-21T22:26:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 -- 172.21.15.66:0/2356089143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 msgr2=0x558b02069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02069350 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0965b0 tx=0x7fab1c022420 comp rx=0 tx=0).stop 2023-07-21T22:26:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 -- 172.21.15.66:0/2356089143 shutdown_connections 2023-07-21T22:26:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02069da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b0206e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2356089143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02069350 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.885+0000 7fab473ceb80 1 -- 172.21.15.66:0/2356089143 >> 172.21.15.66:0/2356089143 conn(0x558b0205eef0 msgr2=0x558b02064830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.886+0000 7fab473ceb80 1 -- 172.21.15.66:0/2356089143 shutdown_connections 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.886+0000 7fab473ceb80 1 -- 172.21.15.66:0/2356089143 wait complete. 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.886+0000 7fab473ceb80 1 -- start start 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.886+0000 7fab473ceb80 1 Processor -- start 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b02069930 0x558b02072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b020734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206eac0 0x558b02073a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b01f5aac0 con 0x558b0206e1b0 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b01f5a7c0 con 0x558b02069930 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b01e3e920 con 0x558b0206eac0 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab473ceb80 10 client.0 _ll_register_callbacks cb 0x558b01f58760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b020734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab3179b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b02069930 0x558b02072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b020734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50632/0 (socket says 172.21.15.66:50632) 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab30f9a700 1 -- 172.21.15.66:0/3233088325 learned_addr learned my addr 172.21.15.66:0/3233088325 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab31f9c700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206eac0 0x558b02073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab3179b700 1 -- 172.21.15.66:0/3233088325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206eac0 msgr2=0x558b02073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab3179b700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206eac0 0x558b02073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab3179b700 1 -- 172.21.15.66:0/3233088325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 msgr2=0x558b020734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab3179b700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b020734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab3179b700 1 -- 172.21.15.66:0/3233088325 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c1a4c40 con 0x558b02069930 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.887+0000 7fab30f9a700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b020734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.888+0000 7fab3179b700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b02069930 0x558b02072fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b6f0 tx=0x7fab2801a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bb7dcf679395705 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.888+0000 7fab14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.888+0000 7fab14ff9700 1 -- 172.21.15.66:0/3233088325 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2800f870 con 0x558b02069930 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.888+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b01f144f0 con 0x558b02069930 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.888+0000 7fab14ff9700 1 -- 172.21.15.66:0/3233088325 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fab2800f9d0 con 0x558b02069930 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab14ff9700 1 -- 172.21.15.66:0/3233088325 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab2800fb90 con 0x558b02069930 2023-07-21T22:26:17.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 10 client.26582 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558b01dd1c60 con 0x558b02069930 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab14ff9700 1 -- 172.21.15.66:0/3233088325 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b01f144f0 con 0x558b02069930 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab14ff9700 1 -- 172.21.15.66:0/3233088325 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab2802fc90 con 0x558b02069930 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 10 client.26582 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 10 client.26582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 20 client.26582 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b02083470 0x558b02085850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 4 client.26582 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.889+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b01fb3eb0 con 0x558b02083470 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.890+0000 7fab30f9a700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b02083470 0x558b02085850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.890+0000 7fab30f9a700 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b02083470 0x558b02085850 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b1a198830a1362 server_cookie=b532d8cee5a75fe4 in_seq=0 out_seq=0 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.890+0000 7fab14ff9700 10 client.26582 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab14ff9700 1 -- 172.21.15.66:0/3233088325 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b01fb3eb0 con 0x558b02083470 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab14ff9700 10 client.26582 handle_command_reply: tid=0 2023-07-21T22:26:17.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 client.26582 shutdown 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b02069930 msgr2=0x558b02072fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b02069930 0x558b02072fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b6f0 tx=0x7fab2801a050 comp rx=0 tx=0).stop 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 shutdown_connections 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b02069930 0x558b02072fb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b02083470 0x558b02085850 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206eac0 0x558b02073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab14ff9700 0 client.26582 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:17.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3233088325 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b020734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 >> 172.21.15.66:0/3233088325 conn(0x558b0205eef0 msgr2=0x558b020643c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 shutdown_connections 2023-07-21T22:26:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.891+0000 7fab473ceb80 1 -- 172.21.15.66:0/3233088325 wait complete. 2023-07-21T22:26:17.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.892+0000 7fab473ceb80 20 client.26582 trim_cache size 0 max 16384 2023-07-21T22:26:17.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 Processor -- start 2023-07-21T22:26:17.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 -- start start 2023-07-21T22:26:17.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02073770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b02067b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02068080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b02083250 con 0x558b0206e1b0 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b02065a50 con 0x558b0206eac0 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.905+0000 7fab473ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b02071510 con 0x558b02069930 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b02067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b02067b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50648/0 (socket says 172.21.15.66:50648) 2023-07-21T22:26:17.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 -- 172.21.15.66:0/2783196670 learned_addr learned my addr 172.21.15.66:0/2783196670 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab3179b700 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02073770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab31f9c700 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02068080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 -- 172.21.15.66:0/2783196670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 msgr2=0x558b02073770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02073770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 -- 172.21.15.66:0/2783196670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 msgr2=0x558b02068080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02068080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 -- 172.21.15.66:0/2783196670 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b02063230 con 0x558b0206e1b0 2023-07-21T22:26:17.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab3179b700 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:17.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.906+0000 7fab30f9a700 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b02067b40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0965b0 tx=0x7fab1c0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd85770694940ee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab17fff700 1 -- 172.21.15.66:0/2783196670 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0b6070 con 0x558b0206e1b0 2023-07-21T22:26:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab17fff700 1 -- 172.21.15.66:0/2783196670 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c097ee0 con 0x558b0206e1b0 2023-07-21T22:26:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab17fff700 1 -- 172.21.15.66:0/2783196670 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c099030 con 0x558b0206e1b0 2023-07-21T22:26:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 -- 172.21.15.66:0/2783196670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 msgr2=0x558b02067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b02067b40 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0965b0 tx=0x7fab1c0050b0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 -- 172.21.15.66:0/2783196670 shutdown_connections 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02068080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b02069930 0x558b02073770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 --2- 172.21.15.66:0/2783196670 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b0206e1b0 0x558b02067b40 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 -- 172.21.15.66:0/2783196670 >> 172.21.15.66:0/2783196670 conn(0x558b0205eef0 msgr2=0x558b02069d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 -- 172.21.15.66:0/2783196670 shutdown_connections 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.907+0000 7fab473ceb80 1 -- 172.21.15.66:0/2783196670 wait complete. 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 -- start start 2023-07-21T22:26:17.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 Processor -- start 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b02069930 0x558b020784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 0x558b020789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02078f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b020d63e0 con 0x558b02069930 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b020d6550 con 0x558b0206eac0 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.908+0000 7fab473ceb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b020d6710 con 0x558b0206e1b0 2023-07-21T22:26:17.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab473ceb80 10 client.0 _ll_register_callbacks cb 0x7fab28024410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab3179b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b02069930 0x558b020784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 0x558b020789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab31f9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02078f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 0x558b020789e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:34052/0 (socket says 172.21.15.66:34052) 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab31f9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02078f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33472/0 (socket says 172.21.15.66:33472) 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 -- 172.21.15.66:0/3893879336 learned_addr learned my addr 172.21.15.66:0/3893879336 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 -- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 msgr2=0x558b02078f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.960 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02078f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 -- 172.21.15.66:0/3893879336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b02069930 msgr2=0x558b020784a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b02069930 0x558b020784a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 -- 172.21.15.66:0/3893879336 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b01dd1c60 con 0x558b0206e1b0 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab30f9a700 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 0x558b020789e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab1c099ad0 tx=0x7fab1c099bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c327478502f7141d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.909+0000 7fab14ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab14ff9700 1 -- 172.21.15.66:0/3893879336 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0b6050 con 0x558b0206e1b0 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab14ff9700 1 -- 172.21.15.66:0/3893879336 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fab1c099230 con 0x558b0206e1b0 2023-07-21T22:26:17.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab1c004380 con 0x558b0206e1b0 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab14ff9700 1 -- 172.21.15.66:0/3893879336 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab1c099da0 con 0x558b0206e1b0 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 10 client.26563 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x558b01de8de0 con 0x558b0206e1b0 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab14ff9700 1 -- 172.21.15.66:0/3893879336 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c004380 con 0x558b0206e1b0 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab14ff9700 1 -- 172.21.15.66:0/3893879336 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab1c098aa0 con 0x558b0206e1b0 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 10 client.26563 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:17.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 10 client.26563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 20 client.26563 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b020d7990 0x558b020d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 4 client.26563 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:26:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.910+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b01f76050 con 0x558b020d7990 2023-07-21T22:26:17.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab3179b700 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b020d7990 0x558b020d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:17.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab3179b700 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b020d7990 0x558b020d9da0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b43da691a61160a1 server_cookie=a986b4d66f7cc496 in_seq=0 out_seq=0 2023-07-21T22:26:17.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab14ff9700 10 client.26563 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:17.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab14ff9700 1 -- 172.21.15.66:0/3893879336 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b01f76050 con 0x558b020d7990 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab14ff9700 10 client.26563 handle_command_reply: tid=0 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab473ceb80 1 client.26563 shutdown 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 msgr2=0x558b020789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.911+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 0x558b020789e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab1c099ad0 tx=0x7fab1c099bc0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 shutdown_connections 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558b0206eac0 0x558b02078f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x558b020d7990 0x558b020d9da0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x558b02069930 0x558b020784a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 --2- 172.21.15.66:0/3893879336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558b0206e1b0 0x558b020789e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 >> 172.21.15.66:0/3893879336 conn(0x558b0205eef0 msgr2=0x558b02069d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab14ff9700 0 client.26563 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 shutdown_connections 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 1 -- 172.21.15.66:0/3893879336 wait complete. 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab473ceb80 20 client.26563 trim_cache size 0 max 16384 2023-07-21T22:26:17.966 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:26:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fab180a1340 msgr2=0x7fab180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab339ff700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fab180a1340 0x7fab180a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b2e0 tx=0x7fab28005fd0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 msgr2=0x7fab2c198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab339ff700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c198350 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00d980 tx=0x7fab1c00dc90 comp rx=0 tx=0).stop 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 shutdown_connections 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.912+0000 7fab339ff700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fab180a1340 0x7fab180a37f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.913+0000 7fab339ff700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fab2c10c310 0x7fab2c19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.913+0000 7fab339ff700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fab2c0728b0 0x7fab2c197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.913+0000 7fab339ff700 1 --2- 172.21.15.66:0/1360500017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab2c0731c0 0x7fab2c198350 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:17.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.913+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 >> 172.21.15.66:0/1360500017 conn(0x7fab2c0fb5f0 msgr2=0x7fab2c10e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:17.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.913+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 shutdown_connections 2023-07-21T22:26:17.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:17.913+0000 7fab339ff700 1 -- 172.21.15.66:0/1360500017 wait complete. 2023-07-21T22:26:17.975 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:17 smithi066 ceph-mon[126379]: pgmap v147: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2023-07-21T22:26:17.975 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:17 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/527392130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:26:18.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:17 smithi149 ceph-mon[129268]: pgmap v147: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2023-07-21T22:26:18.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:17 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/527392130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:26:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:17 smithi104 ceph-mon[129021]: pgmap v147: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2023-07-21T22:26:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:17 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/527392130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:26:18.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.73 MB/sec execute 105 sec latency 11193.072 ms 2023-07-21T22:26:18.978 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:26:18.978 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:26:19.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 -- 172.21.15.66:0/2482763672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb00ff9d0 msgr2=0x7f8cb00ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 --2- 172.21.15.66:0/2482763672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb00ff9d0 0x7f8cb00ffe40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0009990 tx=0x7f8ca0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 -- 172.21.15.66:0/2482763672 shutdown_connections 2023-07-21T22:26:19.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 --2- 172.21.15.66:0/2482763672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb0100380 0x7f8cb010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 --2- 172.21.15.66:0/2482763672 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cb01059e0 0x7f8cb0105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 --2- 172.21.15.66:0/2482763672 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb00ff9d0 0x7f8cb00ffe40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 -- 172.21.15.66:0/2482763672 >> 172.21.15.66:0/2482763672 conn(0x7f8cb00fb4b0 msgr2=0x7f8cb00fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 -- 172.21.15.66:0/2482763672 shutdown_connections 2023-07-21T22:26:19.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.154+0000 7f8cb659d700 1 -- 172.21.15.66:0/2482763672 wait complete. 2023-07-21T22:26:19.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.155+0000 7f8cb659d700 1 Processor -- start 2023-07-21T22:26:19.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 -- start start 2023-07-21T22:26:19.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb00ff9d0 0x7f8cb019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cb0100380 0x7f8cb019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 0x7f8cb0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0197fb0 con 0x7f8cb01059e0 2023-07-21T22:26:19.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0198120 con 0x7f8cb00ff9d0 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb659d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb01982e0 con 0x7f8cb0100380 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb5d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 0x7f8cb0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb5d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 0x7f8cb0197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50690/0 (socket says 172.21.15.66:50690) 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb559b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb00ff9d0 0x7f8cb019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb5d9c700 1 -- 172.21.15.66:0/4186223819 learned_addr learned my addr 172.21.15.66:0/4186223819 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.156+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cb0100380 0x7f8cb019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb5d9c700 1 -- 172.21.15.66:0/4186223819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cb0100380 msgr2=0x7f8cb019de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb5d9c700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cb0100380 0x7f8cb019de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb5d9c700 1 -- 172.21.15.66:0/4186223819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb00ff9d0 msgr2=0x7f8cb019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb5d9c700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb00ff9d0 0x7f8cb019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb5d9c700 1 -- 172.21.15.66:0/4186223819 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca00095f0 con 0x7f8cb01059e0 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb5d9c700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 0x7f8cb0197940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800ef30 tx=0x7f8ca800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8ca67fc700 1 -- 172.21.15.66:0/4186223819 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8018070 con 0x7f8cb01059e0 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8ca67fc700 1 -- 172.21.15.66:0/4186223819 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ca800f040 con 0x7f8cb01059e0 2023-07-21T22:26:19.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb0198560 con 0x7f8cb01059e0 2023-07-21T22:26:19.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8ca67fc700 1 -- 172.21.15.66:0/4186223819 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca800cd00 con 0x7f8cb01059e0 2023-07-21T22:26:19.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.157+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb01a48c0 con 0x7f8cb01059e0 2023-07-21T22:26:19.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.162+0000 7f8ca67fc700 1 -- 172.21.15.66:0/4186223819 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8ca8003a00 con 0x7f8cb01059e0 2023-07-21T22:26:19.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.163+0000 7f8ccb193b80 1 Processor -- start 2023-07-21T22:26:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.163+0000 7f8ccb193b80 1 -- start start 2023-07-21T22:26:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7e20 0x563ef53b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef52add30 con 0x563ef53a7510 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef5300ac0 con 0x563ef53b2cc0 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef52f9f50 con 0x563ef53a7e20 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53a78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50692/0 (socket says 172.21.15.66:50692) 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb4d9a700 1 -- 172.21.15.66:0/3788204848 learned_addr learned my addr 172.21.15.66:0/3788204848 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ca67fc700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8c9c0a13e0 0x7f8c9c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7e20 0x563ef53b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8ca67fc700 1 -- 172.21.15.66:0/4186223819 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8ca80ccb70 con 0x7f8cb01059e0 2023-07-21T22:26:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb559b700 1 -- 172.21.15.66:0/3788204848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 msgr2=0x563ef53b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.164+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb5d9c700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8c9c0a13e0 0x7f8c9c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb559b700 1 -- 172.21.15.66:0/3788204848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 msgr2=0x563ef53a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb559b700 1 -- 172.21.15.66:0/3788204848 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef5127e60 con 0x563ef53a7e20 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7e20 0x563ef53b2780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009c80 tx=0x7f8cac009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd6bf29dfe5ed1c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8c9b7fe700 1 -- 172.21.15.66:0/3788204848 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac011cb0 con 0x563ef53a7e20 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8cb5d9c700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8c9c0a13e0 0x7f8c9c0a3890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c9f70 tx=0x7f8ca801c000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8c9b7fe700 1 -- 172.21.15.66:0/3788204848 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cac011e10 con 0x563ef53a7e20 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.165+0000 7f8c9b7fe700 1 -- 172.21.15.66:0/3788204848 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac005220 con 0x563ef53a7e20 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3788204848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7e20 msgr2=0x563ef53b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7e20 0x563ef53b2780 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009c80 tx=0x7f8cac009f90 comp rx=0 tx=0).stop 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3788204848 shutdown_connections 2023-07-21T22:26:19.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3788204848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7e20 0x563ef53b2780 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3788204848 >> 172.21.15.66:0/3788204848 conn(0x563ef539df70 msgr2=0x563ef53a38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3788204848 shutdown_connections 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.166+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3788204848 wait complete. 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 -- start start 2023-07-21T22:26:19.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 Processor -- start 2023-07-21T22:26:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7e20 0x563ef53af900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef5299b40 con 0x563ef53a7510 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef5299840 con 0x563ef53a7e20 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.167+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef517d9a0 con 0x563ef53b2cc0 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8ccb193b80 10 client.0 _ll_register_callbacks cb 0x563ef52977e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50706/0 (socket says 172.21.15.66:50706) 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 -- 172.21.15.66:0/206310089 learned_addr learned my addr 172.21.15.66:0/206310089 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb559b700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7e20 0x563ef53af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 -- 172.21.15.66:0/206310089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 msgr2=0x563ef53b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 -- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7e20 msgr2=0x563ef53af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7e20 0x563ef53af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 -- 172.21.15.66:0/206310089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca0020040 con 0x563ef53a7510 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53b26f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000bbf0 tx=0x7f8ca0005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400e757a6a4cab91 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.168+0000 7f8c98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8c98ff9700 1 -- 172.21.15.66:0/206310089 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca001d070 con 0x563ef53a7510 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ef5253570 con 0x563ef53a7510 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8c98ff9700 1 -- 172.21.15.66:0/206310089 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8ca0005490 con 0x563ef53a7510 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8c98ff9700 1 -- 172.21.15.66:0/206310089 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ca0005650 con 0x563ef53a7510 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8ccb193b80 10 client.16542 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f8cac0098e0 con 0x563ef53a7510 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8c98ff9700 1 -- 172.21.15.66:0/206310089 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ef5253570 con 0x563ef53a7510 2023-07-21T22:26:19.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.169+0000 7f8c98ff9700 1 -- 172.21.15.66:0/206310089 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8ca000f4f0 con 0x563ef53a7510 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8ccb193b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8ccb193b80 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8ccb193b80 20 client.16542 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef53c2530 0x563ef53c4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8ccb193b80 4 client.16542 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ef52f2f30 con 0x563ef53c2530 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.170+0000 7f8cb559b700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef53c2530 0x563ef53c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.171+0000 7f8c98ff9700 10 client.16542 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.171+0000 7f8cb559b700 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef53c2530 0x563ef53c4940 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61edfba596921b09 server_cookie=2cfbe65ce7b2df53 in_seq=0 out_seq=0 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.171+0000 7f8c98ff9700 1 -- 172.21.15.66:0/206310089 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563ef52f2f30 con 0x563ef53c2530 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.171+0000 7f8c98ff9700 10 client.16542 handle_command_reply: tid=0 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.171+0000 7f8ccb193b80 1 client.16542 shutdown 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 msgr2=0x563ef53b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53b26f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000bbf0 tx=0x7f8ca0005a90 comp rx=0 tx=0).stop 2023-07-21T22:26:19.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 shutdown_connections 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef53c2530 0x563ef53c4940 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7510 0x563ef53b26f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8c98ff9700 0 client.16542 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/206310089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7e20 0x563ef53af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 >> 172.21.15.66:0/206310089 conn(0x563ef539df70 msgr2=0x563ef53a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 shutdown_connections 2023-07-21T22:26:19.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 1 -- 172.21.15.66:0/206310089 wait complete. 2023-07-21T22:26:19.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.172+0000 7f8ccb193b80 20 client.16542 trim_cache size 0 max 16384 2023-07-21T22:26:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 Processor -- start 2023-07-21T22:26:19.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 -- start start 2023-07-21T22:26:19.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7510 0x563ef53b2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53a3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef53a3e90 con 0x563ef53a7e20 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef53a00c0 con 0x563ef53b2cc0 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef53a8270 con 0x563ef53a7510 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.190+0000 7f8cb559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7510 0x563ef53b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53a3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50716/0 (socket says 172.21.15.66:50716) 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7510 0x563ef53b2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:34100/0 (socket says 172.21.15.66:34100) 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 -- 172.21.15.66:0/3682671954 learned_addr learned my addr 172.21.15.66:0/3682671954 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 -- 172.21.15.66:0/3682671954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7510 msgr2=0x563ef53b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7510 0x563ef53b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 -- 172.21.15.66:0/3682671954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 msgr2=0x563ef53a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 -- 172.21.15.66:0/3682671954 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef5110ce0 con 0x563ef53a7e20 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8cb559b700 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53a3410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0097c0 tx=0x7f8cac004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459af8d46c21f8ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.191+0000 7f8c9bfff700 1 -- 172.21.15.66:0/3682671954 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac004870 con 0x563ef53a7e20 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8c9bfff700 1 -- 172.21.15.66:0/3682671954 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cac011c30 con 0x563ef53a7e20 2023-07-21T22:26:19.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8c9bfff700 1 -- 172.21.15.66:0/3682671954 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac003b10 con 0x563ef53a7e20 2023-07-21T22:26:19.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3682671954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 msgr2=0x563ef53a3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53a3410 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0097c0 tx=0x7f8cac004a90 comp rx=0 tx=0).stop 2023-07-21T22:26:19.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3682671954 shutdown_connections 2023-07-21T22:26:19.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53b2cc0 0x563ef53a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53a7510 0x563ef53b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/3682671954 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53a3410 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3682671954 >> 172.21.15.66:0/3682671954 conn(0x563ef539df70 msgr2=0x563ef53aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3682671954 shutdown_connections 2023-07-21T22:26:19.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.192+0000 7f8ccb193b80 1 -- 172.21.15.66:0/3682671954 wait complete. 2023-07-21T22:26:19.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 -- start start 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 Processor -- start 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7510 0x563ef53b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef53b6d00 con 0x563ef53a7e20 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef53b6e70 con 0x563ef53a7510 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef53b7030 con 0x563ef53b2cc0 2023-07-21T22:26:19.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.193+0000 7f8ccb193b80 10 client.0 _ll_register_callbacks cb 0x7f8ca000b930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7510 0x563ef53b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb4d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7510 0x563ef53b91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33532/0 (socket says 172.21.15.66:33532) 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53b96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50726/0 (socket says 172.21.15.66:50726) 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb4d9a700 1 -- 172.21.15.66:0/62543400 learned_addr learned my addr 172.21.15.66:0/62543400 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 -- 172.21.15.66:0/62543400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 msgr2=0x563ef53b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 -- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7510 msgr2=0x563ef53b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7510 0x563ef53b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 -- 172.21.15.66:0/62543400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac01d040 con 0x563ef53a7e20 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8cb559b700 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53b96f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x563ef53b0b70 tx=0x7f8cac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33db139e188add0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.194+0000 7f8c98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8c98ff9700 1 -- 172.21.15.66:0/62543400 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac005ce0 con 0x563ef53a7e20 2023-07-21T22:26:19.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8c98ff9700 1 -- 172.21.15.66:0/62543400 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8cac004020 con 0x563ef53a7e20 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cac011e10 con 0x563ef53a7e20 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8c98ff9700 1 -- 172.21.15.66:0/62543400 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cac0041e0 con 0x563ef53a7e20 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8ccb193b80 10 client.16548 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x563ef514e2d0 con 0x563ef53a7e20 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.195+0000 7f8c98ff9700 1 -- 172.21.15.66:0/62543400 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cac011e10 con 0x563ef53a7e20 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8c98ff9700 1 -- 172.21.15.66:0/62543400 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8cac026430 con 0x563ef53a7e20 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8ccb193b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:19.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8ccb193b80 10 client.16548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8ccb193b80 20 client.16548 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef5418390 0x563ef541a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8ccb193b80 4 client.16548 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563ef52b50d0 con 0x563ef5418390 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.196+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef5418390 0x563ef541a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.197+0000 7f8cb4d9a700 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef5418390 0x563ef541a770 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dcbb95f7cc02661 server_cookie=af7ce751a47c5341 in_seq=0 out_seq=0 2023-07-21T22:26:19.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.197+0000 7f8c98ff9700 10 client.16548 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8c98ff9700 1 -- 172.21.15.66:0/62543400 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563ef52b50d0 con 0x563ef5418390 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8c98ff9700 10 client.16548 handle_command_reply: tid=0 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 client.16548 shutdown 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 msgr2=0x563ef53b96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53b96f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x563ef53b0b70 tx=0x7f8cac009bb0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 shutdown_connections 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563ef53a7510 0x563ef53b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x563ef5418390 0x563ef541a770 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563ef53b2cc0 0x563ef53b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 --2- 172.21.15.66:0/62543400 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563ef53a7e20 0x563ef53b96f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8c98ff9700 0 client.16548 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 >> 172.21.15.66:0/62543400 conn(0x563ef539df70 msgr2=0x563ef53aabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 shutdown_connections 2023-07-21T22:26:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.198+0000 7f8ccb193b80 1 -- 172.21.15.66:0/62543400 wait complete. 2023-07-21T22:26:19.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8ccb193b80 20 client.16548 trim_cache size 0 max 16384 2023-07-21T22:26:19.253 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:26:19.253 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:26:19.253 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:26:19.253 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:26:19.253 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:26:19.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8c9c0a13e0 msgr2=0x7f8c9c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8c9c0a13e0 0x7f8c9c0a3890 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80c9f70 tx=0x7f8ca801c000 comp rx=0 tx=0).stop 2023-07-21T22:26:19.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 msgr2=0x7f8cb0197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 0x7f8cb0197940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800ef30 tx=0x7f8ca800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 shutdown_connections 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cb00ff9d0 0x7f8cb019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8c9c0a13e0 0x7f8c9c0a3890 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8cb01059e0 0x7f8cb0197940 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 --2- 172.21.15.66:0/4186223819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8cb0100380 0x7f8cb019de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 >> 172.21.15.66:0/4186223819 conn(0x7f8cb00fb4b0 msgr2=0x7f8cb0072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.199+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 shutdown_connections 2023-07-21T22:26:19.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.200+0000 7f8cb659d700 1 -- 172.21.15.66:0/4186223819 wait complete. 2023-07-21T22:26:19.265 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:26:19.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 -- 172.21.15.66:0/615166968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd80ffae0 msgr2=0x7f7cd80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/615166968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd80ffae0 0x7f7cd80fff50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80098f0 tx=0x7f7cc8009c00 comp rx=0 tx=0).stop 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 -- 172.21.15.66:0/615166968 shutdown_connections 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/615166968 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd8100490 0x7f7cd810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/615166968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd8105af0 0x7f7cd8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/615166968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd80ffae0 0x7f7cd80fff50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 -- 172.21.15.66:0/615166968 >> 172.21.15.66:0/615166968 conn(0x7f7cd80fb5e0 msgr2=0x7f7cd80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 -- 172.21.15.66:0/615166968 shutdown_connections 2023-07-21T22:26:19.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.433+0000 7f7cdfbff700 1 -- 172.21.15.66:0/615166968 wait complete. 2023-07-21T22:26:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.434+0000 7f7cdfbff700 1 Processor -- start 2023-07-21T22:26:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.434+0000 7f7cdfbff700 1 -- start start 2023-07-21T22:26:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.434+0000 7f7cdfbff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd80ffae0 0x7f7cd8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.434+0000 7f7cdfbff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd8100490 0x7f7cd81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdfbff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd8105af0 0x7f7cd819c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdfbff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd819ce80 con 0x7f7cd8105af0 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdfbff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd819cff0 con 0x7f7cd8100490 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdfbff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd819d1b0 con 0x7f7cd80ffae0 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd80ffae0 0x7f7cd8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd80ffae0 0x7f7cd8197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:34134/0 (socket says 172.21.15.66:34134) 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd8100490 0x7f7cd81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd99b700 1 -- 172.21.15.66:0/1634037342 learned_addr learned my addr 172.21.15.66:0/1634037342 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cde19c700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd8105af0 0x7f7cd819c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 -- 172.21.15.66:0/1634037342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd80ffae0 msgr2=0x7f7cd8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd80ffae0 0x7f7cd8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 -- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd8105af0 msgr2=0x7f7cd819c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd8105af0 0x7f7cd819c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 -- 172.21.15.66:0/1634037342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd0009d00 con 0x7f7cd8100490 2023-07-21T22:26:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.435+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd8100490 0x7f7cd81983e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009810 tx=0x7f7cc800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.436+0000 7f7cceffd700 1 -- 172.21.15.66:0/1634037342 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc801c070 con 0x7f7cd8100490 2023-07-21T22:26:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.436+0000 7f7cceffd700 1 -- 172.21.15.66:0/1634037342 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cc800bb20 con 0x7f7cd8100490 2023-07-21T22:26:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.436+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc8009520 con 0x7f7cd8100490 2023-07-21T22:26:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.436+0000 7f7cceffd700 1 -- 172.21.15.66:0/1634037342 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80038e0 con 0x7f7cd8100490 2023-07-21T22:26:19.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.436+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd819d6c0 con 0x7f7cd8100490 2023-07-21T22:26:19.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.440+0000 7f7cceffd700 1 -- 172.21.15.66:0/1634037342 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7cc8020030 con 0x7f7cd8100490 2023-07-21T22:26:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.441+0000 7f7cf35dfb80 1 Processor -- start 2023-07-21T22:26:19.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.441+0000 7f7cf35dfb80 1 -- start start 2023-07-21T22:26:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.441+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.441+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a83b3cb0 con 0x5613a84b3ac0 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a8406a40 con 0x5613a84ae930 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a83ffed0 con 0x5613a84b31b0 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cde19c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cde19c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:34144/0 (socket says 172.21.15.66:34144) 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84ae350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50740/0 (socket says 172.21.15.66:50740) 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cde19c700 1 -- 172.21.15.66:0/3659775362 learned_addr learned my addr 172.21.15.66:0/3659775362 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cceffd700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7cc40a12c0 0x7f7cc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 -- 172.21.15.66:0/3659775362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 msgr2=0x5613a84b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 -- 172.21.15.66:0/3659775362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 msgr2=0x5613a84aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cdd99b700 1 -- 172.21.15.66:0/3659775362 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613a822dde0 con 0x5613a84b3ac0 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.442+0000 7f7cceffd700 1 -- 172.21.15.66:0/1634037342 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7cc80cdf50 con 0x7f7cd8100490 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.443+0000 7f7cde19c700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7cc40a12c0 0x7f7cc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.443+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84ae350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009cc0 tx=0x7f7cd4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb3d61a888cd7d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.443+0000 7f7cc3fff700 1 -- 172.21.15.66:0/3659775362 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4011d50 con 0x5613a84b3ac0 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.443+0000 7f7cc3fff700 1 -- 172.21.15.66:0/3659775362 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cd4004510 con 0x5613a84b3ac0 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.443+0000 7f7cc3fff700 1 -- 172.21.15.66:0/3659775362 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4005280 con 0x5613a84b3ac0 2023-07-21T22:26:19.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cde19c700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7cc40a12c0 0x7f7cc40a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009610 tx=0x7f7cd0004ad0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/3659775362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 msgr2=0x5613a84ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84ae350 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009cc0 tx=0x7f7cd4011440 comp rx=0 tx=0).stop 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/3659775362 shutdown_connections 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/3659775362 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84ae350 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/3659775362 >> 172.21.15.66:0/3659775362 conn(0x5613a84a3ef0 msgr2=0x5613a84a9830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/3659775362 shutdown_connections 2023-07-21T22:26:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/3659775362 wait complete. 2023-07-21T22:26:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.444+0000 7f7cf35dfb80 1 -- start start 2023-07-21T22:26:19.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 Processor -- start 2023-07-21T22:26:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a839fac0 con 0x5613a84b3ac0 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a839f7c0 con 0x5613a84ae930 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a8283920 con 0x5613a84b31b0 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cf35dfb80 10 client.0 _ll_register_callbacks cb 0x5613a839d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.445+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cde19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84bac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:33548/0 (socket says 172.21.15.66:33548) 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 -- 172.21.15.66:0/496873035 learned_addr learned my addr 172.21.15.66:0/496873035 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 -- 172.21.15.66:0/496873035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 msgr2=0x5613a84b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 -- 172.21.15.66:0/496873035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 msgr2=0x5613a84b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 -- 172.21.15.66:0/496873035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd40098f0 con 0x5613a84ae930 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84bac00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40046e0 tx=0x7f7cd4005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf92cb407e7393d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.446+0000 7f7cc17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cc17fa700 1 -- 172.21.15.66:0/496873035 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4011b50 con 0x5613a84ae930 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613a83594f0 con 0x5613a84ae930 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cc17fa700 1 -- 172.21.15.66:0/496873035 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7cd4011cb0 con 0x5613a84ae930 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cc17fa700 1 -- 172.21.15.66:0/496873035 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cd4003680 con 0x5613a84ae930 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cf35dfb80 10 client.26618 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:19.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5613a8216c60 con 0x5613a84ae930 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cc17fa700 1 -- 172.21.15.66:0/496873035 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613a83594f0 con 0x5613a84ae930 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cc17fa700 1 -- 172.21.15.66:0/496873035 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cd40050d0 con 0x5613a84ae930 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cf35dfb80 10 client.26618 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.447+0000 7f7cf35dfb80 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cf35dfb80 20 client.26618 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a84c83f0 0x5613a84ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cf35dfb80 4 client.26618 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613a83f8eb0 con 0x5613a84c83f0 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a84c83f0 0x5613a84ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a84c83f0 0x5613a84ca800 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a71b9ab36e4c7b server_cookie=5f058883cc766a39 in_seq=0 out_seq=0 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.448+0000 7f7cc17fa700 10 client.26618 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cc17fa700 1 -- 172.21.15.66:0/496873035 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5613a83f8eb0 con 0x5613a84c83f0 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cc17fa700 10 client.26618 handle_command_reply: tid=0 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 client.26618 shutdown 2023-07-21T22:26:19.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 msgr2=0x5613a84bac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84bac00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40046e0 tx=0x7f7cd4005e50 comp rx=0 tx=0).stop 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 shutdown_connections 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84ae930 0x5613a84bac00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a84c83f0 0x5613a84ca800 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/496873035 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b31b0 0x5613a84b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cc17fa700 0 client.26618 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.449+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 >> 172.21.15.66:0/496873035 conn(0x5613a84a3ef0 msgr2=0x5613a84a9420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.450+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 shutdown_connections 2023-07-21T22:26:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.450+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/496873035 wait complete. 2023-07-21T22:26:19.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.450+0000 7f7cf35dfb80 20 client.26618 trim_cache size 0 max 16384 2023-07-21T22:26:19.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.467+0000 7f7cf35dfb80 1 Processor -- start 2023-07-21T22:26:19.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.467+0000 7f7cf35dfb80 1 -- start start 2023-07-21T22:26:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 0x5613a84ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a84b5f10 con 0x5613a84b3ac0 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a84b8160 con 0x5613a84b31b0 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a84b2cc0 con 0x5613a84ae930 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cde19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 0x5613a84ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 0x5613a84ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:34164/0 (socket says 172.21.15.66:34164) 2023-07-21T22:26:19.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cde19c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50764/0 (socket says 172.21.15.66:50764) 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2411032534 learned_addr learned my addr 172.21.15.66:0/2411032534 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.468+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2411032534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 msgr2=0x5613a84b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84b76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2411032534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 msgr2=0x5613a84b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2411032534 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613a8254250 con 0x5613a84ae930 2023-07-21T22:26:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cde19c700 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 0x5613a84ba730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40096b0 tx=0x7f7cd4005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d61f0297434db26b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cccbff700 1 -- 172.21.15.66:0/2411032534 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4004100 con 0x5613a84ae930 2023-07-21T22:26:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cccbff700 1 -- 172.21.15.66:0/2411032534 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cd4011790 con 0x5613a84ae930 2023-07-21T22:26:19.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.469+0000 7f7cccbff700 1 -- 172.21.15.66:0/2411032534 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40038e0 con 0x5613a84ae930 2023-07-21T22:26:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2411032534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 msgr2=0x5613a84ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 0x5613a84ba730 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40096b0 tx=0x7f7cd4005650 comp rx=0 tx=0).stop 2023-07-21T22:26:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2411032534 shutdown_connections 2023-07-21T22:26:19.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84b3ac0 0x5613a84b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84ae930 0x5613a84ba730 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2411032534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2411032534 >> 172.21.15.66:0/2411032534 conn(0x5613a84a3ef0 msgr2=0x5613a84acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2411032534 shutdown_connections 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2411032534 wait complete. 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.470+0000 7f7cf35dfb80 1 -- start start 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 Processor -- start 2023-07-21T22:26:19.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 0x5613a84bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b3ac0 0x5613a84bd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a84bde40 con 0x5613a84ae930 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a84bdfb0 con 0x5613a84b31b0 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613a84be170 con 0x5613a84b3ac0 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cf35dfb80 10 client.0 _ll_register_callbacks cb 0x7f7cc80d1890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 0x5613a84bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cdd19a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cdd99b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 0x5613a84bf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50766/0 (socket says 172.21.15.66:50766) 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2977572302 learned_addr learned my addr 172.21.15.66:0/2977572302 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.471+0000 7f7cde19c700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b3ac0 0x5613a84bd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2977572302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b3ac0 msgr2=0x5613a84bd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b3ac0 0x5613a84bd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 msgr2=0x5613a84bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd99b700 1 -- 172.21.15.66:0/2977572302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd00046d0 con 0x5613a84ae930 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cdd99b700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 0x5613a84bf220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400a010 tx=0x7f7cd4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd008f3f8b7a49a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cc17fa700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cc17fa700 1 -- 172.21.15.66:0/2977572302 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4004ce0 con 0x5613a84ae930 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cd4004510 con 0x5613a84ae930 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cc17fa700 1 -- 172.21.15.66:0/2977572302 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7cd4000e40 con 0x5613a84ae930 2023-07-21T22:26:19.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cc17fa700 1 -- 172.21.15.66:0/2977572302 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cd4004e40 con 0x5613a84ae930 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cf35dfb80 10 client.16566 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.472+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5613a8216c60 con 0x5613a84ae930 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cc17fa700 1 -- 172.21.15.66:0/2977572302 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7cd4004510 con 0x5613a84ae930 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cc17fa700 1 -- 172.21.15.66:0/2977572302 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7cd40225a0 con 0x5613a84ae930 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cf35dfb80 10 client.16566 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cf35dfb80 10 client.16566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cf35dfb80 20 client.16566 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:19.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a851d0f0 0x5613a851f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cf35dfb80 4 client.16566 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5613a83bb050 con 0x5613a851d0f0 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.473+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a851d0f0 0x5613a851f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cc17fa700 10 client.16566 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cdd19a700 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a851d0f0 0x5613a851f4d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262e821856048d66 server_cookie=23014233c33e2497 in_seq=0 out_seq=0 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cc17fa700 1 -- 172.21.15.66:0/2977572302 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5613a83bb050 con 0x5613a851d0f0 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cc17fa700 10 client.16566 handle_command_reply: tid=0 2023-07-21T22:26:19.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cf35dfb80 1 client.16566 shutdown 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 msgr2=0x5613a84bf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.474+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 0x5613a84bf220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400a010 tx=0x7f7cd4011b50 comp rx=0 tx=0).stop 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 shutdown_connections 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5613a851d0f0 0x5613a851f4d0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5613a84b3ac0 0x5613a84bd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5613a84ae930 0x5613a84bf220 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 --2- 172.21.15.66:0/2977572302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613a84b31b0 0x5613a84bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cc17fa700 0 client.16566 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:19.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 >> 172.21.15.66:0/2977572302 conn(0x5613a84a3ef0 msgr2=0x5613a84ada20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 shutdown_connections 2023-07-21T22:26:19.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 1 -- 172.21.15.66:0/2977572302 wait complete. 2023-07-21T22:26:19.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cf35dfb80 20 client.16566 trim_cache size 0 max 16384 2023-07-21T22:26:19.529 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:26:19.530 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:26:19.531 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:26:19.531 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:26:19.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7cc40a12c0 msgr2=0x7f7cc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7cc40a12c0 0x7f7cc40a3770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0009610 tx=0x7f7cd0004ad0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd8100490 msgr2=0x7f7cd81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.475+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd8100490 0x7f7cd81983e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009810 tx=0x7f7cc800fa00 comp rx=0 tx=0).stop 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 shutdown_connections 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7cc40a12c0 0x7f7cc40a3770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7cd8105af0 0x7f7cd819c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cd80ffae0 0x7f7cd8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 --2- 172.21.15.66:0/1634037342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd8100490 0x7f7cd81983e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:19.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 >> 172.21.15.66:0/1634037342 conn(0x7f7cd80fb5e0 msgr2=0x7f7cd810ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:19.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 shutdown_connections 2023-07-21T22:26:19.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:19.476+0000 7f7cdfbff700 1 -- 172.21.15.66:0/1634037342 wait complete. 2023-07-21T22:26:19.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 138963 4.69 MB/sec execute 106 sec latency 12193.120 ms 2023-07-21T22:26:19.540 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:26:20.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:19 smithi066 ceph-mon[126379]: pgmap v148: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2023-07-21T22:26:20.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:19 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:26:20.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:19 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:26:20.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:19 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:26:20.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:19 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:26:20.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:19 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:26:20.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:19 smithi149 ceph-mon[129268]: pgmap v148: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2023-07-21T22:26:20.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:19 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:26:20.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:19 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:26:20.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:19 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:26:20.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:19 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:26:20.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:19 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:26:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:19 smithi104 ceph-mon[129021]: pgmap v148: 129 pgs: 129 active+clean; 115 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 1.8 MiB/s wr, 89 op/s 2023-07-21T22:26:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:19 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:26:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:19 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:26:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:19 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:26:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:19 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:26:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:19 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:26:20.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 139475 4.66 MB/sec execute 107 sec latency 12868.006 ms 2023-07-21T22:26:21.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 141743 4.73 MB/sec execute 108 sec latency 26.886 ms 2023-07-21T22:26:22.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:21 smithi066 ceph-mon[126379]: pgmap v149: 129 pgs: 129 active+clean; 112 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2023-07-21T22:26:22.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:21 smithi149 ceph-mon[129268]: pgmap v149: 129 pgs: 129 active+clean; 112 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2023-07-21T22:26:22.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:21 smithi104 ceph-mon[129021]: pgmap v149: 129 pgs: 129 active+clean; 112 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 2.4 MiB/s wr, 119 op/s 2023-07-21T22:26:22.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 142733 4.74 MB/sec execute 109 sec latency 106.855 ms 2023-07-21T22:26:23.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 144931 4.80 MB/sec execute 110 sec latency 16.101 ms 2023-07-21T22:26:24.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:23 smithi066 ceph-mon[126379]: pgmap v150: 129 pgs: 129 active+clean; 112 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2023-07-21T22:26:24.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:23 smithi149 ceph-mon[129268]: pgmap v150: 129 pgs: 129 active+clean; 112 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2023-07-21T22:26:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:23 smithi104 ceph-mon[129021]: pgmap v150: 129 pgs: 129 active+clean; 112 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2023-07-21T22:26:24.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 146286 4.82 MB/sec execute 111 sec latency 22.520 ms 2023-07-21T22:26:25.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 148374 4.89 MB/sec execute 112 sec latency 71.146 ms 2023-07-21T22:26:26.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:25 smithi066 ceph-mon[126379]: pgmap v151: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2023-07-21T22:26:26.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:25 smithi149 ceph-mon[129268]: pgmap v151: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2023-07-21T22:26:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:25 smithi104 ceph-mon[129021]: pgmap v151: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 152 op/s 2023-07-21T22:26:26.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 149602 4.90 MB/sec execute 113 sec latency 25.704 ms 2023-07-21T22:26:27.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 150874 4.91 MB/sec execute 114 sec latency 20.438 ms 2023-07-21T22:26:28.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:27 smithi066 ceph-mon[126379]: pgmap v152: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T22:26:28.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:27 smithi149 ceph-mon[129268]: pgmap v152: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T22:26:28.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:27 smithi104 ceph-mon[129021]: pgmap v152: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T22:26:28.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 152876 4.98 MB/sec execute 115 sec latency 29.332 ms 2023-07-21T22:26:29.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 154209 4.99 MB/sec execute 116 sec latency 21.659 ms 2023-07-21T22:26:30.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:29 smithi149 ceph-mon[129268]: pgmap v153: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T22:26:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:29 smithi104 ceph-mon[129021]: pgmap v153: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T22:26:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:29 smithi066 ceph-mon[126379]: pgmap v153: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2023-07-21T22:26:30.532 INFO:tasks.workunit.client.0.smithi066.stdout: 1 156335 5.06 MB/sec execute 117 sec latency 60.071 ms 2023-07-21T22:26:31.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 157528 5.06 MB/sec execute 118 sec latency 34.329 ms 2023-07-21T22:26:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:31 smithi149 ceph-mon[129268]: pgmap v154: 129 pgs: 129 active+clean; 117 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2023-07-21T22:26:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:31 smithi104 ceph-mon[129021]: pgmap v154: 129 pgs: 129 active+clean; 117 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2023-07-21T22:26:32.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:31 smithi066 ceph-mon[126379]: pgmap v154: 129 pgs: 129 active+clean; 117 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2023-07-21T22:26:32.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 159716 5.13 MB/sec execute 119 sec latency 15.677 ms 2023-07-21T22:26:33.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 160818 5.11 MB/sec execute 120 sec latency 25.744 ms 2023-07-21T22:26:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:33 smithi149 ceph-mon[129268]: pgmap v155: 129 pgs: 129 active+clean; 117 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-07-21T22:26:34.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:33 smithi104 ceph-mon[129021]: pgmap v155: 129 pgs: 129 active+clean; 117 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-07-21T22:26:34.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:33 smithi066 ceph-mon[126379]: pgmap v155: 129 pgs: 129 active+clean; 117 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2023-07-21T22:26:34.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 162663 5.16 MB/sec execute 121 sec latency 42.116 ms 2023-07-21T22:26:35.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 163923 5.17 MB/sec execute 122 sec latency 45.959 ms 2023-07-21T22:26:36.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:35 smithi149 ceph-mon[129268]: pgmap v156: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2023-07-21T22:26:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:35 smithi104 ceph-mon[129021]: pgmap v156: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2023-07-21T22:26:36.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:35 smithi066 ceph-mon[126379]: pgmap v156: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2023-07-21T22:26:36.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 166058 5.22 MB/sec execute 123 sec latency 21.238 ms 2023-07-21T22:26:37.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 167399 5.24 MB/sec execute 124 sec latency 22.378 ms 2023-07-21T22:26:38.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:37 smithi149 ceph-mon[129268]: pgmap v157: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2023-07-21T22:26:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:37 smithi104 ceph-mon[129021]: pgmap v157: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2023-07-21T22:26:38.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:37 smithi066 ceph-mon[126379]: pgmap v157: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2023-07-21T22:26:38.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 168575 5.24 MB/sec execute 125 sec latency 27.957 ms 2023-07-21T22:26:39.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 170537 5.31 MB/sec execute 126 sec latency 31.088 ms 2023-07-21T22:26:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:39 smithi149 ceph-mon[129268]: pgmap v158: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2023-07-21T22:26:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:39 smithi104 ceph-mon[129021]: pgmap v158: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2023-07-21T22:26:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:39 smithi066 ceph-mon[126379]: pgmap v158: 129 pgs: 129 active+clean; 123 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2023-07-21T22:26:40.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 171769 5.31 MB/sec execute 127 sec latency 31.325 ms 2023-07-21T22:26:41.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 174036 5.37 MB/sec execute 128 sec latency 28.011 ms 2023-07-21T22:26:42.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:41 smithi149 ceph-mon[129268]: pgmap v159: 129 pgs: 129 active+clean; 128 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 288 op/s 2023-07-21T22:26:42.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:41 smithi104 ceph-mon[129021]: pgmap v159: 129 pgs: 129 active+clean; 128 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 288 op/s 2023-07-21T22:26:42.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:41 smithi066 ceph-mon[126379]: pgmap v159: 129 pgs: 129 active+clean; 128 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 288 op/s 2023-07-21T22:26:42.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 175223 5.37 MB/sec execute 129 sec latency 26.042 ms 2023-07-21T22:26:43.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:42 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:43.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:42 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:43.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:42 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:26:43.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 177182 5.40 MB/sec execute 130 sec latency 26.980 ms 2023-07-21T22:26:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:43 smithi149 ceph-mon[129268]: pgmap v160: 129 pgs: 129 active+clean; 128 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2023-07-21T22:26:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:43 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:43 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:43 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:26:44.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:43 smithi104 ceph-mon[129021]: pgmap v160: 129 pgs: 129 active+clean; 128 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2023-07-21T22:26:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:43 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:43 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:43 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:26:44.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:43 smithi066 ceph-mon[126379]: pgmap v160: 129 pgs: 129 active+clean; 128 MiB data, 973 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2023-07-21T22:26:44.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:43 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:26:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:43 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:26:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:43 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:26:44.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 178173 5.40 MB/sec execute 131 sec latency 29.557 ms 2023-07-21T22:26:45.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 180340 5.45 MB/sec execute 132 sec latency 25.339 ms 2023-07-21T22:26:46.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:45 smithi149 ceph-mon[129268]: pgmap v161: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2023-07-21T22:26:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:45 smithi104 ceph-mon[129021]: pgmap v161: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2023-07-21T22:26:46.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:45 smithi066 ceph-mon[126379]: pgmap v161: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.3 MiB/s wr, 281 op/s 2023-07-21T22:26:46.533 INFO:tasks.workunit.client.0.smithi066.stdout: 1 181634 5.46 MB/sec execute 133 sec latency 24.905 ms 2023-07-21T22:26:47.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 182891 5.46 MB/sec execute 134 sec latency 17.365 ms 2023-07-21T22:26:48.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:47 smithi149 ceph-mon[129268]: pgmap v162: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-07-21T22:26:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:47 smithi104 ceph-mon[129021]: pgmap v162: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-07-21T22:26:48.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:47 smithi066 ceph-mon[126379]: pgmap v162: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-07-21T22:26:48.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 184819 5.52 MB/sec execute 135 sec latency 13.159 ms 2023-07-21T22:26:49.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 186115 5.52 MB/sec execute 136 sec latency 19.002 ms 2023-07-21T22:26:49.541 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:26:49.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 -- 172.21.15.66:0/2813554643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688100480 msgr2=0x7f768810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 --2- 172.21.15.66:0/2813554643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688100480 0x7f768810c910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f768000b0a0 tx=0x7f768000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 -- 172.21.15.66:0/2813554643 shutdown_connections 2023-07-21T22:26:49.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 --2- 172.21.15.66:0/2813554643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7688100480 0x7f768810c910 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 --2- 172.21.15.66:0/2813554643 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688105ae0 0x7f7688105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 --2- 172.21.15.66:0/2813554643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76880ffad0 0x7f76880fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 -- 172.21.15.66:0/2813554643 >> 172.21.15.66:0/2813554643 conn(0x7f76880fb5f0 msgr2=0x7f76880fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:49.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 -- 172.21.15.66:0/2813554643 shutdown_connections 2023-07-21T22:26:49.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.720+0000 7f768f1ff700 1 -- 172.21.15.66:0/2813554643 wait complete. 2023-07-21T22:26:49.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.722+0000 7f768f1ff700 1 Processor -- start 2023-07-21T22:26:49.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 -- start start 2023-07-21T22:26:49.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 0x7f7688072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688100480 0x7f7688072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688105ae0 0x7f76881a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688073580 con 0x7f7688105ae0 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76881a43d0 con 0x7f76880ffad0 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768f1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76881a4510 con 0x7f7688100480 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 0x7f7688072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 0x7f7688072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47488/0 (socket says 172.21.15.66:47488) 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 -- 172.21.15.66:0/580960309 learned_addr learned my addr 172.21.15.66:0/580960309 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:49.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.723+0000 7f7687fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688100480 0x7f7688072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768d79c700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688105ae0 0x7f76881a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 -- 172.21.15.66:0/580960309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688100480 msgr2=0x7f7688072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688100480 0x7f7688072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 -- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688105ae0 msgr2=0x7f76881a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688105ae0 0x7f76881a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 -- 172.21.15.66:0/580960309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7680009d00 con 0x7f76880ffad0 2023-07-21T22:26:49.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f7687fff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688100480 0x7f7688072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768cf9b700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 0x7f7688072810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f767c00e9a0 tx=0x7f767c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:49.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f7685ffb700 1 -- 172.21.15.66:0/580960309 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c009790 con 0x7f76880ffad0 2023-07-21T22:26:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f7685ffb700 1 -- 172.21.15.66:0/580960309 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c010430 con 0x7f76880ffad0 2023-07-21T22:26:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76881a4770 con 0x7f76880ffad0 2023-07-21T22:26:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.724+0000 7f7685ffb700 1 -- 172.21.15.66:0/580960309 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c018680 con 0x7f76880ffad0 2023-07-21T22:26:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.725+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76881a4ca0 con 0x7f76880ffad0 2023-07-21T22:26:49.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.728+0000 7f7685ffb700 1 -- 172.21.15.66:0/580960309 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f767c0105a0 con 0x7f76880ffad0 2023-07-21T22:26:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.729+0000 7f76a2b46b80 1 Processor -- start 2023-07-21T22:26:49.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.729+0000 7f76a2b46b80 1 -- start start 2023-07-21T22:26:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae38350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae38930 0x5648aae38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aad3dcb0 con 0x5648aae38930 2023-07-21T22:26:49.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aad90a40 con 0x5648aae3dac0 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aad89ed0 con 0x5648aae3d1b0 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae38350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f7685ffb700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76780a1340 0x7f76780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768d79c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae38350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47496/0 (socket says 172.21.15.66:47496) 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 -- 172.21.15.66:0/1695388571 learned_addr learned my addr 172.21.15.66:0/1695388571 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:49.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f7687fff700 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae38930 0x5648aae38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f7685ffb700 1 -- 172.21.15.66:0/580960309 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f767c014070 con 0x7f76880ffad0 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 -- 172.21.15.66:0/1695388571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 msgr2=0x5648aae3d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae3d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 -- 172.21.15.66:0/1695388571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae38930 msgr2=0x5648aae38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae38930 0x5648aae38da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768cf9b700 1 -- 172.21.15.66:0/1695388571 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648aabb7de0 con 0x5648aae3dac0 2023-07-21T22:26:49.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.730+0000 7f768d79c700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76780a1340 0x7f76780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.731+0000 7f7687fff700 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae38930 0x5648aae38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:26:49.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.731+0000 7f768cf9b700 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae38350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f767c0ca5f0 tx=0x7f767c099e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2cffa02dd113699 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.731+0000 7f7672ffd700 1 -- 172.21.15.66:0/1695388571 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c099f10 con 0x5648aae3dac0 2023-07-21T22:26:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.731+0000 7f7672ffd700 1 -- 172.21.15.66:0/1695388571 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c09df40 con 0x5648aae3dac0 2023-07-21T22:26:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.731+0000 7f7672ffd700 1 -- 172.21.15.66:0/1695388571 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c099660 con 0x5648aae3dac0 2023-07-21T22:26:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1695388571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 msgr2=0x5648aae38350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f768d79c700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76780a1340 0x7f76780a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7688074920 tx=0x7f768000b3b0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae38350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f767c0ca5f0 tx=0x7f767c099e10 comp rx=0 tx=0).stop 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1695388571 shutdown_connections 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae38930 0x5648aae38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae3d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1695388571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae38350 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1695388571 >> 172.21.15.66:0/1695388571 conn(0x5648aae2def0 msgr2=0x5648aae33830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:49.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1695388571 shutdown_connections 2023-07-21T22:26:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.732+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1695388571 wait complete. 2023-07-21T22:26:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.733+0000 7f76a2b46b80 1 -- start start 2023-07-21T22:26:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.733+0000 7f76a2b46b80 1 Processor -- start 2023-07-21T22:26:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 0x5648aae41ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3dac0 0x5648aae42920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aad29ac0 con 0x5648aae3dac0 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aad297c0 con 0x5648aae38930 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aac0d920 con 0x5648aae3d1b0 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f76a2b46b80 10 client.0 _ll_register_callbacks cb 0x5648aad27760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f7687fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 0x5648aae41ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f7687fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33526/0 (socket says 172.21.15.66:33526) 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f7687fff700 1 -- 172.21.15.66:0/1317186546 learned_addr learned my addr 172.21.15.66:0/1317186546 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 0x5648aae41ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47500/0 (socket says 172.21.15.66:47500) 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.734+0000 7f768d79c700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3dac0 0x5648aae42920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f768cf9b700 1 -- 172.21.15.66:0/1317186546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 msgr2=0x5648aae423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f768cf9b700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f768cf9b700 1 -- 172.21.15.66:0/1317186546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3dac0 msgr2=0x5648aae42920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f768cf9b700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3dac0 0x5648aae42920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f768cf9b700 1 -- 172.21.15.66:0/1317186546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76881a4ca0 con 0x5648aae38930 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f7687fff700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f768cf9b700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 0x5648aae41ea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5648aae451f0 tx=0x7f767c010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7beeeffd180dfab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f7667fff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f7667fff700 1 -- 172.21.15.66:0/1317186546 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a2360 con 0x5648aae38930 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648aace34f0 con 0x5648aae38930 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f7667fff700 1 -- 172.21.15.66:0/1317186546 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f767c09a380 con 0x5648aae38930 2023-07-21T22:26:49.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f7667fff700 1 -- 172.21.15.66:0/1317186546 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f767c0a1ed0 con 0x5648aae38930 2023-07-21T22:26:49.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f76a2b46b80 10 client.26636 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:49.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.735+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5648aaba0c60 con 0x5648aae38930 2023-07-21T22:26:49.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f7667fff700 1 -- 172.21.15.66:0/1317186546 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c0b8030 con 0x5648aae38930 2023-07-21T22:26:49.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f7667fff700 1 -- 172.21.15.66:0/1317186546 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f767c0990b0 con 0x5648aae38930 2023-07-21T22:26:49.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f76a2b46b80 10 client.26636 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:49.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f76a2b46b80 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f76a2b46b80 20 client.26636 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aae52190 0x5648aae545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f76a2b46b80 4 client.26636 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648aad82eb0 con 0x5648aae52190 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.736+0000 7f7687fff700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aae52190 0x5648aae545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.737+0000 7f7667fff700 10 client.26636 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:49.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.737+0000 7f7687fff700 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aae52190 0x5648aae545a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56818a0bcb176acd server_cookie=a287113498db8a94 in_seq=0 out_seq=0 2023-07-21T22:26:49.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f7667fff700 1 -- 172.21.15.66:0/1317186546 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5648aad82eb0 con 0x5648aae52190 2023-07-21T22:26:49.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f7667fff700 10 client.26636 handle_command_reply: tid=0 2023-07-21T22:26:49.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 client.26636 shutdown 2023-07-21T22:26:49.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 msgr2=0x5648aae41ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 0x5648aae41ea0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x5648aae451f0 tx=0x7f767c010a90 comp rx=0 tx=0).stop 2023-07-21T22:26:49.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 shutdown_connections 2023-07-21T22:26:49.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae38930 0x5648aae41ea0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aae52190 0x5648aae545a0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3dac0 0x5648aae42920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/1317186546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae3d1b0 0x5648aae423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f7667fff700 0 client.26636 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 >> 172.21.15.66:0/1317186546 conn(0x5648aae2def0 msgr2=0x5648aae333c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 shutdown_connections 2023-07-21T22:26:49.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 1 -- 172.21.15.66:0/1317186546 wait complete. 2023-07-21T22:26:49.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.739+0000 7f76a2b46b80 20 client.26636 trim_cache size 0 max 16384 2023-07-21T22:26:49.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.758+0000 7f76a2b46b80 1 Processor -- start 2023-07-21T22:26:49.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.758+0000 7f76a2b46b80 1 -- start start 2023-07-21T22:26:49.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae49510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae42750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aae2d9b0 con 0x5648aae3d1b0 2023-07-21T22:26:49.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aae3ce00 con 0x5648aae3dac0 2023-07-21T22:26:49.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aae2f900 con 0x5648aae38930 2023-07-21T22:26:49.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae49510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768d79c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae42750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae49510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33538/0 (socket says 172.21.15.66:33538) 2023-07-21T22:26:49.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 -- 172.21.15.66:0/364287132 learned_addr learned my addr 172.21.15.66:0/364287132 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:49.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f7687fff700 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 -- 172.21.15.66:0/364287132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 msgr2=0x5648aae42750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae42750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 -- 172.21.15.66:0/364287132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 msgr2=0x5648aae380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.759+0000 7f768cf9b700 1 -- 172.21.15.66:0/364287132 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648aae38fd0 con 0x5648aae38930 2023-07-21T22:26:49.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.760+0000 7f768cf9b700 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae49510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c510 tx=0x7f767c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a678e9347e4bfea0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:49.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.760+0000 7f76737fe700 1 -- 172.21.15.66:0/364287132 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a2360 con 0x5648aae38930 2023-07-21T22:26:49.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.760+0000 7f76737fe700 1 -- 172.21.15.66:0/364287132 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f767c0b8040 con 0x5648aae38930 2023-07-21T22:26:49.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.760+0000 7f76737fe700 1 -- 172.21.15.66:0/364287132 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767c0a1ed0 con 0x5648aae38930 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 -- 172.21.15.66:0/364287132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 msgr2=0x5648aae49510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae49510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c510 tx=0x7f767c01f000 comp rx=0 tx=0).stop 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 -- 172.21.15.66:0/364287132 shutdown_connections 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae42750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae49510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/364287132 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 -- 172.21.15.66:0/364287132 >> 172.21.15.66:0/364287132 conn(0x5648aae2def0 msgr2=0x5648aae379e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 -- 172.21.15.66:0/364287132 shutdown_connections 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 -- 172.21.15.66:0/364287132 wait complete. 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 -- start start 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.761+0000 7f76a2b46b80 1 Processor -- start 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae49460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aae479d0 con 0x5648aae3d1b0 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aae47b10 con 0x5648aae3dac0 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648aae47cd0 con 0x5648aae38930 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f76a2b46b80 10 client.0 _ll_register_callbacks cb 0x7f767c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f768d79c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f768d79c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae47330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47516/0 (socket says 172.21.15.66:47516) 2023-07-21T22:26:49.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f768d79c700 1 -- 172.21.15.66:0/2031340649 learned_addr learned my addr 172.21.15.66:0/2031340649 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f7687fff700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae49460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.762+0000 7f768cf9b700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768d79c700 1 -- 172.21.15.66:0/2031340649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 msgr2=0x5648aae48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768d79c700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768d79c700 1 -- 172.21.15.66:0/2031340649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 msgr2=0x5648aae49460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768d79c700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae49460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768d79c700 1 -- 172.21.15.66:0/2031340649 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648aaba0c60 con 0x5648aae3dac0 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768cf9b700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f768d79c700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae47330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f768000b0a0 tx=0x7f76800099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da287884c56255b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f7667fff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.763+0000 7f7667fff700 1 -- 172.21.15.66:0/2031340649 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768000e040 con 0x5648aae3dac0 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f7667fff700 1 -- 172.21.15.66:0/2031340649 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7680020860 con 0x5648aae3dac0 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f767c09df40 con 0x5648aae3dac0 2023-07-21T22:26:49.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f7667fff700 1 -- 172.21.15.66:0/2031340649 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7680007970 con 0x5648aae3dac0 2023-07-21T22:26:49.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 10 client.26642 fetch_fsmap learned FSMap version 33 2023-07-21T22:26:49.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5648aabde250 con 0x5648aae3dac0 2023-07-21T22:26:49.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f7667fff700 1 -- 172.21.15.66:0/2031340649 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7680036020 con 0x5648aae3dac0 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f7667fff700 1 -- 172.21.15.66:0/2031340649 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7680005690 con 0x5648aae3dac0 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 10 client.26642 fetch_fsmap finished waiting for FSMap version 33 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 20 client.26642 populate_metadata read hostname 'smithi066' 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aaea7130 0x5648aaea9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 4 client.26642 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:26:49.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.764+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5648aad45050 con 0x5648aaea7130 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.765+0000 7f768cf9b700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aaea7130 0x5648aaea9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.765+0000 7f768cf9b700 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aaea7130 0x5648aaea9540 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5cb77278b075a5 server_cookie=be41945d8de3cd85 in_seq=0 out_seq=0 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.765+0000 7f7667fff700 10 client.26642 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f7667fff700 1 -- 172.21.15.66:0/2031340649 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5648aad45050 con 0x5648aaea7130 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f7667fff700 10 client.26642 handle_command_reply: tid=0 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 client.26642 shutdown 2023-07-21T22:26:49.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 msgr2=0x5648aae47330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae47330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f768000b0a0 tx=0x7f76800099e0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 shutdown_connections 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5648aae3dac0 0x5648aae47330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5648aaea7130 0x5648aaea9540 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5648aae38930 0x5648aae48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 --2- 172.21.15.66:0/2031340649 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5648aae3d1b0 0x5648aae49460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f7667fff700 0 client.26642 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:26:49.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.766+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 >> 172.21.15.66:0/2031340649 conn(0x5648aae2def0 msgr2=0x5648aae42410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:49.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 shutdown_connections 2023-07-21T22:26:49.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f76a2b46b80 1 -- 172.21.15.66:0/2031340649 wait complete. 2023-07-21T22:26:49.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f76a2b46b80 20 client.26642 trim_cache size 0 max 16384 2023-07-21T22:26:49.820 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:26:49.821 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:26:49.822 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:26:49.822 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:26:49.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76780a1340 msgr2=0x7f76780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f768f1ff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76780a1340 0x7f76780a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7688074920 tx=0x7f768000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 msgr2=0x7f7688072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.767+0000 7f768f1ff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 0x7f7688072810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f767c00e9a0 tx=0x7f767c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 shutdown_connections 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76880ffad0 0x7f7688072810 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76780a1340 0x7f76780a37f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7688105ae0 0x7f76881a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 --2- 172.21.15.66:0/580960309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7688100480 0x7f7688072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:26:49.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 >> 172.21.15.66:0/580960309 conn(0x7f76880fb5f0 msgr2=0x7f7688106930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:26:49.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 shutdown_connections 2023-07-21T22:26:49.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:26:49.768+0000 7f768f1ff700 1 -- 172.21.15.66:0/580960309 wait complete. 2023-07-21T22:26:49.833 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:26:50.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:49 smithi149 ceph-mon[129268]: pgmap v163: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-07-21T22:26:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:49 smithi104 ceph-mon[129021]: pgmap v163: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-07-21T22:26:50.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:49 smithi066 ceph-mon[126379]: pgmap v163: 129 pgs: 129 active+clean; 134 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-07-21T22:26:50.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 188293 5.58 MB/sec execute 137 sec latency 40.824 ms 2023-07-21T22:26:51.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 189476 5.58 MB/sec execute 138 sec latency 32.416 ms 2023-07-21T22:26:52.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:51 smithi149 ceph-mon[129268]: pgmap v164: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2023-07-21T22:26:52.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:51 smithi104 ceph-mon[129021]: pgmap v164: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2023-07-21T22:26:52.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:51 smithi066 ceph-mon[126379]: pgmap v164: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.6 MiB/s wr, 255 op/s 2023-07-21T22:26:52.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 191307 5.60 MB/sec execute 139 sec latency 15.386 ms 2023-07-21T22:26:53.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 192454 5.60 MB/sec execute 140 sec latency 23.651 ms 2023-07-21T22:26:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:53 smithi149 ceph-mon[129268]: pgmap v165: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2023-07-21T22:26:54.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:53 smithi104 ceph-mon[129021]: pgmap v165: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2023-07-21T22:26:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:53 smithi066 ceph-mon[126379]: pgmap v165: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2023-07-21T22:26:54.534 INFO:tasks.workunit.client.0.smithi066.stdout: 1 193908 5.61 MB/sec execute 141 sec latency 19.369 ms 2023-07-21T22:26:55.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 195565 5.65 MB/sec execute 142 sec latency 24.947 ms 2023-07-21T22:26:56.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:55 smithi149 ceph-mon[129268]: pgmap v166: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2023-07-21T22:26:56.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:55 smithi104 ceph-mon[129021]: pgmap v166: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2023-07-21T22:26:56.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:55 smithi066 ceph-mon[126379]: pgmap v166: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2023-07-21T22:26:56.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 196907 5.65 MB/sec execute 143 sec latency 55.820 ms 2023-07-21T22:26:57.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:56 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:26:57.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:56 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:26:57.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:56 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:26:57.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 199020 5.71 MB/sec execute 144 sec latency 27.439 ms 2023-07-21T22:26:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:57 smithi104 ceph-mon[129021]: pgmap v167: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-07-21T22:26:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:57 smithi066 ceph-mon[126379]: pgmap v167: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-07-21T22:26:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:57 smithi149 ceph-mon[129268]: pgmap v167: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-07-21T22:26:58.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 200232 5.70 MB/sec execute 145 sec latency 26.144 ms 2023-07-21T22:26:59.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 202272 5.75 MB/sec execute 146 sec latency 14.375 ms 2023-07-21T22:27:00.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:26:59 smithi104 ceph-mon[129021]: pgmap v168: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-07-21T22:27:00.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:26:59 smithi066 ceph-mon[126379]: pgmap v168: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-07-21T22:27:00.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:26:59 smithi149 ceph-mon[129268]: pgmap v168: 129 pgs: 129 active+clean; 146 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2023-07-21T22:27:00.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 203443 5.73 MB/sec execute 147 sec latency 23.707 ms 2023-07-21T22:27:01.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 205361 5.77 MB/sec execute 148 sec latency 79.756 ms 2023-07-21T22:27:02.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:01 smithi104 ceph-mon[129021]: pgmap v169: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2023-07-21T22:27:02.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:01 smithi066 ceph-mon[126379]: pgmap v169: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2023-07-21T22:27:02.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:01 smithi149 ceph-mon[129268]: pgmap v169: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2023-07-21T22:27:02.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 206556 5.78 MB/sec execute 149 sec latency 16.771 ms 2023-07-21T22:27:03.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 208063 5.78 MB/sec execute 150 sec latency 19.459 ms 2023-07-21T22:27:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:03 smithi104 ceph-mon[129021]: pgmap v170: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T22:27:04.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:03 smithi066 ceph-mon[126379]: pgmap v170: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T22:27:04.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:03 smithi149 ceph-mon[129268]: pgmap v170: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 173 op/s 2023-07-21T22:27:04.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 209740 5.82 MB/sec execute 151 sec latency 11.158 ms 2023-07-21T22:27:05.536 INFO:tasks.workunit.client.0.smithi066.stdout: 1 211370 5.82 MB/sec execute 152 sec latency 33.161 ms 2023-07-21T22:27:06.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:05 smithi104 ceph-mon[129021]: pgmap v171: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2023-07-21T22:27:06.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:05 smithi066 ceph-mon[126379]: pgmap v171: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2023-07-21T22:27:06.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:05 smithi149 ceph-mon[129268]: pgmap v171: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 281 op/s 2023-07-21T22:27:06.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213215 5.87 MB/sec execute 153 sec latency 32.228 ms 2023-07-21T22:27:07.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.83 MB/sec execute 154 sec latency 653.748 ms 2023-07-21T22:27:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:07 smithi104 ceph-mon[129021]: pgmap v172: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-21T22:27:08.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:07 smithi066 ceph-mon[126379]: pgmap v172: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-21T22:27:08.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:07 smithi149 ceph-mon[129268]: pgmap v172: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-21T22:27:08.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.80 MB/sec execute 155 sec latency 1653.858 ms 2023-07-21T22:27:09.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.76 MB/sec execute 156 sec latency 2653.998 ms 2023-07-21T22:27:10.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:09 smithi104 ceph-mon[129021]: pgmap v173: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-21T22:27:10.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:09 smithi066 ceph-mon[126379]: pgmap v173: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-21T22:27:10.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:09 smithi149 ceph-mon[129268]: pgmap v173: 129 pgs: 129 active+clean; 157 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2023-07-21T22:27:10.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.72 MB/sec execute 157 sec latency 3654.132 ms 2023-07-21T22:27:11.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.69 MB/sec execute 158 sec latency 4654.265 ms 2023-07-21T22:27:12.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:11 smithi104 ceph-mon[129021]: pgmap v174: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:27:12.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:11 smithi066 ceph-mon[126379]: pgmap v174: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:27:12.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:11 smithi149 ceph-mon[129268]: pgmap v174: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:27:12.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.65 MB/sec execute 159 sec latency 5654.407 ms 2023-07-21T22:27:13.537 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.61 MB/sec execute 160 sec latency 6654.534 ms 2023-07-21T22:27:14.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:13 smithi104 ceph-mon[129021]: pgmap v175: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 162 op/s 2023-07-21T22:27:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:13 smithi066 ceph-mon[126379]: pgmap v175: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 162 op/s 2023-07-21T22:27:14.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:13 smithi149 ceph-mon[129268]: pgmap v175: 129 pgs: 129 active+clean; 159 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 162 op/s 2023-07-21T22:27:14.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.58 MB/sec execute 161 sec latency 7654.658 ms 2023-07-21T22:27:15.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 213414 5.54 MB/sec execute 162 sec latency 8654.800 ms 2023-07-21T22:27:16.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:15 smithi104 ceph-mon[129021]: pgmap v176: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 185 op/s 2023-07-21T22:27:16.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:15 smithi066 ceph-mon[126379]: pgmap v176: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 185 op/s 2023-07-21T22:27:16.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:15 smithi149 ceph-mon[129268]: pgmap v176: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 185 op/s 2023-07-21T22:27:16.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 214853 5.54 MB/sec execute 163 sec latency 8698.055 ms 2023-07-21T22:27:17.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 216811 5.59 MB/sec execute 164 sec latency 27.906 ms 2023-07-21T22:27:18.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:17 smithi104 ceph-mon[129021]: pgmap v177: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2023-07-21T22:27:18.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:17 smithi066 ceph-mon[126379]: pgmap v177: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2023-07-21T22:27:18.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:17 smithi149 ceph-mon[129268]: pgmap v177: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2023-07-21T22:27:18.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 217981 5.59 MB/sec execute 165 sec latency 130.917 ms 2023-07-21T22:27:19.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 220260 5.64 MB/sec execute 166 sec latency 27.153 ms 2023-07-21T22:27:19.834 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:27:20.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 -- 172.21.15.66:0/1140311505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ffad0 msgr2=0x7f51340fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 --2- 172.21.15.66:0/1140311505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ffad0 0x7f51340fff40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f51280098f0 tx=0x7f5128009c00 comp rx=0 tx=0).stop 2023-07-21T22:27:20.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 -- 172.21.15.66:0/1140311505 shutdown_connections 2023-07-21T22:27:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 --2- 172.21.15.66:0/1140311505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134105ae0 0x7f5134105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 --2- 172.21.15.66:0/1140311505 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5134100480 0x7f513410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 --2- 172.21.15.66:0/1140311505 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f51340ffad0 0x7f51340fff40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 -- 172.21.15.66:0/1140311505 >> 172.21.15.66:0/1140311505 conn(0x7f51340fb5f0 msgr2=0x7f51340fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 -- 172.21.15.66:0/1140311505 shutdown_connections 2023-07-21T22:27:20.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.009+0000 7f513a5ff700 1 -- 172.21.15.66:0/1140311505 wait complete. 2023-07-21T22:27:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 Processor -- start 2023-07-21T22:27:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 -- start start 2023-07-21T22:27:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 0x7f5134072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134100480 0x7f5134072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5134105ae0 0x7f51341a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5134073580 con 0x7f5134105ae0 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341a4320 con 0x7f5134100480 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f513a5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341a4460 con 0x7f51340ffad0 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f5133fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 0x7f5134072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f51337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134100480 0x7f5134072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f5133fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 0x7f5134072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33530/0 (socket says 172.21.15.66:33530) 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f5133fff700 1 -- 172.21.15.66:0/3445844338 learned_addr learned my addr 172.21.15.66:0/3445844338 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.011+0000 7f5138b9c700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5134105ae0 0x7f51341a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51337fe700 1 -- 172.21.15.66:0/3445844338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 msgr2=0x7f5134072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51337fe700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 0x7f5134072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51337fe700 1 -- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5134105ae0 msgr2=0x7f51341a3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51337fe700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5134105ae0 0x7f51341a3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51337fe700 1 -- 172.21.15.66:0/3445844338 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51240095f0 con 0x7f5134100480 2023-07-21T22:27:20.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f5133fff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 0x7f5134072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51337fe700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134100480 0x7f5134072d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5128009810 tx=0x7f512800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51317fa700 1 -- 172.21.15.66:0/3445844338 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512801c070 con 0x7f5134100480 2023-07-21T22:27:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.012+0000 7f51317fa700 1 -- 172.21.15.66:0/3445844338 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512800bb20 con 0x7f5134100480 2023-07-21T22:27:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.013+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5128009520 con 0x7f5134100480 2023-07-21T22:27:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.013+0000 7f51317fa700 1 -- 172.21.15.66:0/3445844338 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280038e0 con 0x7f5134100480 2023-07-21T22:27:20.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.013+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51341a48f0 con 0x7f5134100480 2023-07-21T22:27:20.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.016+0000 7f51317fa700 1 -- 172.21.15.66:0/3445844338 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5128020030 con 0x7f5134100480 2023-07-21T22:27:20.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.017+0000 7f514dfebb80 1 Processor -- start 2023-07-21T22:27:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.017+0000 7f514dfebb80 1 -- start start 2023-07-21T22:27:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f51317fa700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f511c0a12c0 0x7f511c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f51317fa700 1 -- 172.21.15.66:0/3445844338 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f51280cdf50 con 0x7f5134100480 2023-07-21T22:27:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ce1b0 0x562cd91ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91c9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91c9930 0x562cd91c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f514dfebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd90cecb0 con 0x562cd91ce1b0 2023-07-21T22:27:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f514dfebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd9121a40 con 0x562cd91c9930 2023-07-21T22:27:20.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f514dfebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd911aed0 con 0x562cd91ceac0 2023-07-21T22:27:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f5138b9c700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f511c0a12c0 0x7f511c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f51337fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f5133fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ce1b0 0x562cd91ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f51337fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91c9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33546/0 (socket says 172.21.15.66:33546) 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f51337fe700 1 -- 172.21.15.66:0/290301375 learned_addr learned my addr 172.21.15.66:0/290301375 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.018+0000 7f51337fe700 1 -- 172.21.15.66:0/290301375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91c9930 msgr2=0x562cd91c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f5138b9c700 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91c9930 0x562cd91c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f51337fe700 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91c9930 0x562cd91c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f51337fe700 1 -- 172.21.15.66:0/290301375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ce1b0 msgr2=0x562cd91ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f51337fe700 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ce1b0 0x562cd91ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f51337fe700 1 -- 172.21.15.66:0/290301375 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd8f48de0 con 0x562cd91ceac0 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f5133fff700 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ce1b0 0x562cd91ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f5138b9c700 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91c9930 0x562cd91c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f51337fe700 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91c9350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5128097770 tx=0x7f5128009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163fb62908a1fb53 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f5138b9c700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f511c0a12c0 0x7f511c0a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5134074920 tx=0x7f512c009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f511a7fc700 1 -- 172.21.15.66:0/290301375 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280b2070 con 0x562cd91ceac0 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f511a7fc700 1 -- 172.21.15.66:0/290301375 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512809ed30 con 0x562cd91ceac0 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.019+0000 7f511a7fc700 1 -- 172.21.15.66:0/290301375 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128005140 con 0x562cd91ceac0 2023-07-21T22:27:20.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 -- 172.21.15.66:0/290301375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 msgr2=0x562cd91c9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91c9350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5128097770 tx=0x7f5128009c00 comp rx=0 tx=0).stop 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 -- 172.21.15.66:0/290301375 shutdown_connections 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91c9930 0x562cd91c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ce1b0 0x562cd91ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 --2- 172.21.15.66:0/290301375 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91c9350 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 -- 172.21.15.66:0/290301375 >> 172.21.15.66:0/290301375 conn(0x562cd91beef0 msgr2=0x562cd91c4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 -- 172.21.15.66:0/290301375 shutdown_connections 2023-07-21T22:27:20.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.020+0000 7f514dfebb80 1 -- 172.21.15.66:0/290301375 wait complete. 2023-07-21T22:27:20.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 -- start start 2023-07-21T22:27:20.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 Processor -- start 2023-07-21T22:27:20.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd90baac0 con 0x562cd91c9930 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd90ba7c0 con 0x562cd91ce1b0 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.021+0000 7f514dfebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd8f9e920 con 0x562cd91ceac0 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f514dfebb80 10 client.0 _ll_register_callbacks cb 0x562cd90b8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f5133fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f5138b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f5133fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91d5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36408/0 (socket says 172.21.15.66:36408) 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f5133fff700 1 -- 172.21.15.66:0/3128678021 learned_addr learned my addr 172.21.15.66:0/3128678021 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 -- 172.21.15.66:0/3128678021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 msgr2=0x562cd91d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 -- 172.21.15.66:0/3128678021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 msgr2=0x562cd91d5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91d5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 -- 172.21.15.66:0/3128678021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5124011440 con 0x562cd91ce1b0 2023-07-21T22:27:20.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f5133fff700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f5138b9c700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51337fe700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91d2d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5128097770 tx=0x7f51280cb260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36bdc8477059369e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.022+0000 7f51137fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f51137fe700 1 -- 172.21.15.66:0/3128678021 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51280b2070 con 0x562cd91ce1b0 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f51137fe700 1 -- 172.21.15.66:0/3128678021 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5128097410 con 0x562cd91ce1b0 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562cd90744f0 con 0x562cd91ce1b0 2023-07-21T22:27:20.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f51137fe700 1 -- 172.21.15.66:0/3128678021 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51280975d0 con 0x562cd91ce1b0 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 10 client.26651 fetch_fsmap learned FSMap version 34 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f51341a48f0 con 0x562cd91ce1b0 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f51137fe700 1 -- 172.21.15.66:0/3128678021 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562cd90744f0 con 0x562cd91ce1b0 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f51137fe700 1 -- 172.21.15.66:0/3128678021 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5128021c50 con 0x562cd91ce1b0 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 10 client.26651 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 10 client.26651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 20 client.26651 populate_metadata read hostname 'smithi066' 2023-07-21T22:27:20.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd91e2bb0 0x562cd91e4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 4 client.26651 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.023+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562cd9113eb0 con 0x562cd91e2bb0 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.024+0000 7f5133fff700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd91e2bb0 0x562cd91e4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.024+0000 7f5133fff700 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd91e2bb0 0x562cd91e4f90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919890712805f48d server_cookie=6280e4836e32c426 in_seq=0 out_seq=0 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.024+0000 7f51137fe700 10 client.26651 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f51137fe700 1 -- 172.21.15.66:0/3128678021 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562cd9113eb0 con 0x562cd91e2bb0 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f51137fe700 10 client.26651 handle_command_reply: tid=0 2023-07-21T22:27:20.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 client.26651 shutdown 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 msgr2=0x562cd91d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91d2d40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5128097770 tx=0x7f51280cb260 comp rx=0 tx=0).stop 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 shutdown_connections 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd91e2bb0 0x562cd91e4f90 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f51137fe700 0 client.26651 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:20.078 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91d5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3128678021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91d2d40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 >> 172.21.15.66:0/3128678021 conn(0x562cd91beef0 msgr2=0x562cd91c43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.025+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 shutdown_connections 2023-07-21T22:27:20.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.026+0000 7f514dfebb80 1 -- 172.21.15.66:0/3128678021 wait complete. 2023-07-21T22:27:20.079 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.026+0000 7f514dfebb80 20 client.26651 trim_cache size 0 max 16384 2023-07-21T22:27:20.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.043+0000 7f514dfebb80 1 Processor -- start 2023-07-21T22:27:20.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.043+0000 7f514dfebb80 1 -- start start 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 0x562cd91da380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91df480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ceac0 0x562cd91c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f514dfebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd90744f0 con 0x562cd91ceac0 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f514dfebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd91d5e70 con 0x562cd91ce1b0 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f514dfebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd91cddc0 con 0x562cd91c9930 2023-07-21T22:27:20.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f51337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91df480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 0x562cd91da380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5138b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ceac0 0x562cd91c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 0x562cd91da380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33558/0 (socket says 172.21.15.66:33558) 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 -- 172.21.15.66:0/3320376326 learned_addr learned my addr 172.21.15.66:0/3320376326 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f51337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91df480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:57044/0 (socket says 172.21.15.66:57044) 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 -- 172.21.15.66:0/3320376326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 msgr2=0x562cd91df480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91df480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 -- 172.21.15.66:0/3320376326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ceac0 msgr2=0x562cd91c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ceac0 0x562cd91c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.044+0000 7f5133fff700 1 -- 172.21.15.66:0/3320376326 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd8f31c60 con 0x562cd91c9930 2023-07-21T22:27:20.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.045+0000 7f5133fff700 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 0x562cd91da380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5124005b40 tx=0x7f5124004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5080f37341dbd310 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.045+0000 7f511affd700 1 -- 172.21.15.66:0/3320376326 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124017050 con 0x562cd91c9930 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.045+0000 7f511affd700 1 -- 172.21.15.66:0/3320376326 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5124022070 con 0x562cd91c9930 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.045+0000 7f511affd700 1 -- 172.21.15.66:0/3320376326 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124004dd0 con 0x562cd91c9930 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 -- 172.21.15.66:0/3320376326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 msgr2=0x562cd91da380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 0x562cd91da380 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5124005b40 tx=0x7f5124004750 comp rx=0 tx=0).stop 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 -- 172.21.15.66:0/3320376326 shutdown_connections 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91ceac0 0x562cd91c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91c9930 0x562cd91da380 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 --2- 172.21.15.66:0/3320376326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91df480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 -- 172.21.15.66:0/3320376326 >> 172.21.15.66:0/3320376326 conn(0x562cd91beef0 msgr2=0x562cd91d5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 -- 172.21.15.66:0/3320376326 shutdown_connections 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 -- 172.21.15.66:0/3320376326 wait complete. 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 -- start start 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.046+0000 7f514dfebb80 1 Processor -- start 2023-07-21T22:27:20.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd91d6cc0 con 0x562cd91c9930 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd91d6e30 con 0x562cd91ce1b0 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd91d6ff0 con 0x562cd91ceac0 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f514dfebb80 10 client.0 _ll_register_callbacks cb 0x7f5124009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5138b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:36422/0 (socket says 172.21.15.66:36422) 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 -- 172.21.15.66:0/1226531216 learned_addr learned my addr 172.21.15.66:0/1226531216 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f51337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5138b9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:33564/0 (socket says 172.21.15.66:33564) 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 -- 172.21.15.66:0/1226531216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 msgr2=0x562cd91d65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 -- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 msgr2=0x562cd91da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:20.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.047+0000 7f5133fff700 1 -- 172.21.15.66:0/1226531216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5124012c90 con 0x562cd91c9930 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f5133fff700 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91da050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f512400a010 tx=0x7f51240044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1471c7b222142613 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f51137fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f51137fe700 1 -- 172.21.15.66:0/1226531216 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124004c50 con 0x562cd91c9930 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f51137fe700 1 -- 172.21.15.66:0/1226531216 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5124004db0 con 0x562cd91c9930 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f512809ed30 con 0x562cd91c9930 2023-07-21T22:27:20.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f51137fe700 1 -- 172.21.15.66:0/1226531216 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5124017030 con 0x562cd91c9930 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f514dfebb80 10 client.16602 fetch_fsmap learned FSMap version 34 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.048+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x562cd8f6f250 con 0x562cd91c9930 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f51137fe700 1 -- 172.21.15.66:0/1226531216 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f512403a020 con 0x562cd91c9930 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f51137fe700 1 -- 172.21.15.66:0/1226531216 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 34) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5124003680 con 0x562cd91c9930 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f514dfebb80 10 client.16602 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f514dfebb80 10 client.16602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f514dfebb80 20 client.16602 populate_metadata read hostname 'smithi066' 2023-07-21T22:27:20.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f514dfebb80 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd9237d10 0x562cd923a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f514dfebb80 4 client.16602 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:27:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562cd90d6050 con 0x562cd9237d10 2023-07-21T22:27:20.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.049+0000 7f51337fe700 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd9237d10 0x562cd923a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:20.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f51337fe700 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd9237d10 0x562cd923a120 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8871bd86a4def3e1 server_cookie=8ea7ebd4aa3a46cc in_seq=0 out_seq=0 2023-07-21T22:27:20.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f51137fe700 10 client.16602 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:20.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f51137fe700 1 -- 172.21.15.66:0/1226531216 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562cd90d6050 con 0x562cd9237d10 2023-07-21T22:27:20.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f51137fe700 10 client.16602 handle_command_reply: tid=0 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 client.16602 shutdown 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 msgr2=0x562cd91da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91da050 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f512400a010 tx=0x7f51240044b0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 shutdown_connections 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562cd9237d10 0x562cd923a120 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562cd91ceac0 0x562cd91d65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562cd91c9930 0x562cd91da050 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f51137fe700 0 client.16602 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 --2- 172.21.15.66:0/1226531216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562cd91ce1b0 0x562cd91da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 >> 172.21.15.66:0/1226531216 conn(0x562cd91beef0 msgr2=0x562cd91c78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 shutdown_connections 2023-07-21T22:27:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.050+0000 7f514dfebb80 1 -- 172.21.15.66:0/1226531216 wait complete. 2023-07-21T22:27:20.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f514dfebb80 20 client.16602 trim_cache size 0 max 16384 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:27:20.105 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:27:20.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:27:20.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:27:20.106 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:27:20.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f511c0a12c0 msgr2=0x7f511c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f511c0a12c0 0x7f511c0a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5134074920 tx=0x7f512c009920 comp rx=0 tx=0).stop 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134100480 msgr2=0x7f5134072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134100480 0x7f5134072d50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5128009810 tx=0x7f512800fa00 comp rx=0 tx=0).stop 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 shutdown_connections 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f511c0a12c0 0x7f511c0a3770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5134105ae0 0x7f51341a3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51340ffad0 0x7f5134072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 --2- 172.21.15.66:0/3445844338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5134100480 0x7f5134072d50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:20.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 >> 172.21.15.66:0/3445844338 conn(0x7f51340fb5f0 msgr2=0x7f51340fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:20.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 shutdown_connections 2023-07-21T22:27:20.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:20.051+0000 7f513a5ff700 1 -- 172.21.15.66:0/3445844338 wait complete. 2023-07-21T22:27:20.116 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:27:20.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:19 smithi066 ceph-mon[126379]: pgmap v178: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2023-07-21T22:27:20.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:19 smithi104 ceph-mon[129021]: pgmap v178: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2023-07-21T22:27:20.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:19 smithi149 ceph-mon[129268]: pgmap v178: 129 pgs: 129 active+clean; 158 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 1.3 MiB/s wr, 76 op/s 2023-07-21T22:27:20.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 221461 5.64 MB/sec execute 167 sec latency 31.861 ms 2023-07-21T22:27:21.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 222987 5.65 MB/sec execute 168 sec latency 55.521 ms 2023-07-21T22:27:22.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:21 smithi104 ceph-mon[129021]: pgmap v179: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 172 op/s 2023-07-21T22:27:22.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:21 smithi066 ceph-mon[126379]: pgmap v179: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 172 op/s 2023-07-21T22:27:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:21 smithi149 ceph-mon[129268]: pgmap v179: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 172 op/s 2023-07-21T22:27:22.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 224294 5.66 MB/sec execute 169 sec latency 27.917 ms 2023-07-21T22:27:23.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 225441 5.65 MB/sec execute 170 sec latency 35.591 ms 2023-07-21T22:27:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:23 smithi104 ceph-mon[129021]: pgmap v180: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 119 op/s 2023-07-21T22:27:24.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:23 smithi066 ceph-mon[126379]: pgmap v180: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 119 op/s 2023-07-21T22:27:24.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:23 smithi149 ceph-mon[129268]: pgmap v180: 129 pgs: 129 active+clean; 164 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 119 op/s 2023-07-21T22:27:24.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 227443 5.70 MB/sec execute 171 sec latency 15.582 ms 2023-07-21T22:27:25.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 228676 5.70 MB/sec execute 172 sec latency 14.377 ms 2023-07-21T22:27:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:25 smithi104 ceph-mon[129021]: pgmap v181: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2023-07-21T22:27:26.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:25 smithi066 ceph-mon[126379]: pgmap v181: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2023-07-21T22:27:26.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:25 smithi149 ceph-mon[129268]: pgmap v181: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2023-07-21T22:27:26.538 INFO:tasks.workunit.client.0.smithi066.stdout: 1 230673 5.74 MB/sec execute 173 sec latency 53.146 ms 2023-07-21T22:27:27.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 231594 5.72 MB/sec execute 174 sec latency 25.457 ms 2023-07-21T22:27:28.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:28 smithi104 ceph-mon[129021]: pgmap v182: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2023-07-21T22:27:28.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:28 smithi066 ceph-mon[126379]: pgmap v182: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2023-07-21T22:27:28.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:28 smithi149 ceph-mon[129268]: pgmap v182: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2023-07-21T22:27:28.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 233974 5.76 MB/sec execute 175 sec latency 38.996 ms 2023-07-21T22:27:29.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 235050 5.76 MB/sec execute 176 sec latency 21.033 ms 2023-07-21T22:27:30.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:30 smithi104 ceph-mon[129021]: pgmap v183: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2023-07-21T22:27:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:30 smithi066 ceph-mon[126379]: pgmap v183: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2023-07-21T22:27:30.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:30 smithi149 ceph-mon[129268]: pgmap v183: 129 pgs: 129 active+clean; 169 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2023-07-21T22:27:30.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 237486 5.80 MB/sec execute 177 sec latency 15.368 ms 2023-07-21T22:27:31.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 238418 5.80 MB/sec execute 178 sec latency 56.246 ms 2023-07-21T22:27:32.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:32 smithi104 ceph-mon[129021]: pgmap v184: 129 pgs: 129 active+clean; 175 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 272 op/s 2023-07-21T22:27:32.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:32 smithi066 ceph-mon[126379]: pgmap v184: 129 pgs: 129 active+clean; 175 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 272 op/s 2023-07-21T22:27:32.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:32 smithi149 ceph-mon[129268]: pgmap v184: 129 pgs: 129 active+clean; 175 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 272 op/s 2023-07-21T22:27:32.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 240174 5.81 MB/sec execute 179 sec latency 34.099 ms 2023-07-21T22:27:33.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 241779 5.84 MB/sec execute 180 sec latency 10.703 ms 2023-07-21T22:27:34.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:34 smithi104 ceph-mon[129021]: pgmap v185: 129 pgs: 129 active+clean; 175 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2023-07-21T22:27:34.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:34 smithi066 ceph-mon[126379]: pgmap v185: 129 pgs: 129 active+clean; 175 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2023-07-21T22:27:34.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:34 smithi149 ceph-mon[129268]: pgmap v185: 129 pgs: 129 active+clean; 175 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2023-07-21T22:27:34.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 243155 5.84 MB/sec execute 181 sec latency 23.092 ms 2023-07-21T22:27:35.539 INFO:tasks.workunit.client.0.smithi066.stdout: 1 245171 5.88 MB/sec execute 182 sec latency 12.368 ms 2023-07-21T22:27:36.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:36 smithi104 ceph-mon[129021]: pgmap v186: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 268 op/s 2023-07-21T22:27:36.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:36 smithi066 ceph-mon[126379]: pgmap v186: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 268 op/s 2023-07-21T22:27:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:36 smithi149 ceph-mon[129268]: pgmap v186: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 268 op/s 2023-07-21T22:27:36.540 INFO:tasks.workunit.client.0.smithi066.stdout: 1 245385 5.85 MB/sec execute 183 sec latency 703.221 ms 2023-07-21T22:27:37.540 INFO:tasks.workunit.client.0.smithi066.stdout: 1 245385 5.82 MB/sec execute 184 sec latency 1703.299 ms 2023-07-21T22:27:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:38 smithi104 ceph-mon[129021]: pgmap v187: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-21T22:27:38.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:38 smithi066 ceph-mon[126379]: pgmap v187: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-21T22:27:38.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:38 smithi149 ceph-mon[129268]: pgmap v187: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-21T22:27:38.540 INFO:tasks.workunit.client.0.smithi066.stdout: 1 245385 5.79 MB/sec execute 185 sec latency 2703.446 ms 2023-07-21T22:27:39.540 INFO:tasks.workunit.client.0.smithi066.stdout: 1 245385 5.75 MB/sec execute 186 sec latency 3703.611 ms 2023-07-21T22:27:40.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:40 smithi104 ceph-mon[129021]: pgmap v188: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-21T22:27:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:40 smithi066 ceph-mon[126379]: pgmap v188: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-21T22:27:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:40 smithi149 ceph-mon[129268]: pgmap v188: 129 pgs: 129 active+clean; 182 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 184 op/s 2023-07-21T22:27:40.540 INFO:tasks.workunit.client.0.smithi066.stdout: 1 245776 5.73 MB/sec execute 187 sec latency 4388.958 ms 2023-07-21T22:27:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:41 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:27:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:41 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:27:41.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:41 smithi104 ceph-mon[129021]: pgmap v189: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 221 op/s 2023-07-21T22:27:41.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:41 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:27:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:41 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:27:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:41 smithi066 ceph-mon[126379]: pgmap v189: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 221 op/s 2023-07-21T22:27:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:41 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:27:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:41 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:27:41.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:41 smithi149 ceph-mon[129268]: pgmap v189: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 221 op/s 2023-07-21T22:27:41.540 INFO:tasks.workunit.client.0.smithi066.stdout: 1 246478 5.72 MB/sec execute 188 sec latency 656.618 ms 2023-07-21T22:27:42.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 246478 5.69 MB/sec execute 189 sec latency 1656.750 ms 2023-07-21T22:27:43.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 246478 5.66 MB/sec execute 190 sec latency 2656.839 ms 2023-07-21T22:27:44.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:43 smithi066 ceph-mon[126379]: pgmap v190: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-07-21T22:27:44.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:43 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:43 smithi149 ceph-mon[129268]: pgmap v190: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-07-21T22:27:44.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:43 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:43 smithi104 ceph-mon[129021]: pgmap v190: 129 pgs: 129 active+clean; 183 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2023-07-21T22:27:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:43 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:27:44.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 246478 5.63 MB/sec execute 191 sec latency 3656.967 ms 2023-07-21T22:27:45.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:44 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:45.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:44 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:45.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:44 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:27:45.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:44 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:45.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:44 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:45.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:44 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:27:45.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:44 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:27:45.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:44 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:27:45.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:44 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:27:45.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 247382 5.62 MB/sec execute 192 sec latency 4296.112 ms 2023-07-21T22:27:46.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:45 smithi066 ceph-mon[126379]: pgmap v191: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2023-07-21T22:27:46.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:45 smithi149 ceph-mon[129268]: pgmap v191: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2023-07-21T22:27:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:45 smithi104 ceph-mon[129021]: pgmap v191: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2023-07-21T22:27:46.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 248876 5.65 MB/sec execute 193 sec latency 33.585 ms 2023-07-21T22:27:47.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:46 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:27:47.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:46 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:27:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:46 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:27:47.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 250021 5.64 MB/sec execute 194 sec latency 222.101 ms 2023-07-21T22:27:48.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:47 smithi149 ceph-mon[129268]: pgmap v192: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 975 KiB/s wr, 58 op/s 2023-07-21T22:27:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:47 smithi104 ceph-mon[129021]: pgmap v192: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 975 KiB/s wr, 58 op/s 2023-07-21T22:27:48.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:47 smithi066 ceph-mon[126379]: pgmap v192: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 975 KiB/s wr, 58 op/s 2023-07-21T22:27:48.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 250021 5.61 MB/sec execute 195 sec latency 1222.252 ms 2023-07-21T22:27:49.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 250021 5.59 MB/sec execute 196 sec latency 2222.411 ms 2023-07-21T22:27:50.117 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:27:50.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:49 smithi149 ceph-mon[129268]: pgmap v193: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 975 KiB/s wr, 58 op/s 2023-07-21T22:27:50.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 -- 172.21.15.66:0/1559353654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 msgr2=0x7f53f410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 --2- 172.21.15.66:0/1559353654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f410c910 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00b0a0 tx=0x7f53ec00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 -- 172.21.15.66:0/1559353654 shutdown_connections 2023-07-21T22:27:50.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 --2- 172.21.15.66:0/1559353654 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f410c910 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 --2- 172.21.15.66:0/1559353654 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f4105ae0 0x7f53f4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 --2- 172.21.15.66:0/1559353654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f40ffad0 0x7f53f40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 -- 172.21.15.66:0/1559353654 >> 172.21.15.66:0/1559353654 conn(0x7f53f40fb5f0 msgr2=0x7f53f40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 -- 172.21.15.66:0/1559353654 shutdown_connections 2023-07-21T22:27:50.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.298+0000 7f53fabff700 1 -- 172.21.15.66:0/1559353654 wait complete. 2023-07-21T22:27:50.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.299+0000 7f53fabff700 1 Processor -- start 2023-07-21T22:27:50.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.299+0000 7f53fabff700 1 -- start start 2023-07-21T22:27:50.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53fabff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f40ffad0 0x7f53f4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53fabff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53fabff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f4105ae0 0x7f53f419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53fabff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419cf10 con 0x7f53f40ffad0 2023-07-21T22:27:50.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53fabff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419d080 con 0x7f53f4100480 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53fabff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419d240 con 0x7f53f4105ae0 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f40ffad0 0x7f53f4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50094/0 (socket says 172.21.15.66:50094) 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f40ffad0 0x7f53f4197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50050/0 (socket says 172.21.15.66:50050) 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 -- 172.21.15.66:0/3512127305 learned_addr learned my addr 172.21.15.66:0/3512127305 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f919c700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f4105ae0 0x7f53f419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 -- 172.21.15.66:0/3512127305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f4105ae0 msgr2=0x7f53f419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f4105ae0 0x7f53f419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 -- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f40ffad0 msgr2=0x7f53f4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f40ffad0 0x7f53f4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.300+0000 7f53f3fff700 1 -- 172.21.15.66:0/3512127305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e8009520 con 0x7f53f4100480 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.301+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f4198470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9a0 tx=0x7f53e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.301+0000 7f53f1ffb700 1 -- 172.21.15.66:0/3512127305 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0009790 con 0x7f53f4100480 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.301+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec009d00 con 0x7f53f4100480 2023-07-21T22:27:50.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.301+0000 7f53f1ffb700 1 -- 172.21.15.66:0/3512127305 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e0010430 con 0x7f53f4100480 2023-07-21T22:27:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.301+0000 7f53f1ffb700 1 -- 172.21.15.66:0/3512127305 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0018680 con 0x7f53f4100480 2023-07-21T22:27:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.301+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f419d710 con 0x7f53f4100480 2023-07-21T22:27:50.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.304+0000 7f53f1ffb700 1 -- 172.21.15.66:0/3512127305 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f53e00105a0 con 0x7f53f4100480 2023-07-21T22:27:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.306+0000 7f540e57eb80 1 Processor -- start 2023-07-21T22:27:50.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.306+0000 7f540e57eb80 1 -- start start 2023-07-21T22:27:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.306+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.306+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55e5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f540e57eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc54eacb0 con 0x556cc55eaac0 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f540e57eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc553da40 con 0x556cc55ea1b0 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f540e57eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc5536ed0 con 0x556cc55e5930 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50098/0 (socket says 172.21.15.66:50098) 2023-07-21T22:27:50.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 -- 172.21.15.66:0/3087547324 learned_addr learned my addr 172.21.15.66:0/3087547324 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f899b700 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55e5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 -- 172.21.15.66:0/3087547324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 msgr2=0x556cc55e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 -- 172.21.15.66:0/3087547324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 msgr2=0x556cc55e5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55e5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 -- 172.21.15.66:0/3087547324 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cc5364de0 con 0x556cc55ea1b0 2023-07-21T22:27:50.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:27:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f1ffb700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f53e40a12c0 0x7f53e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.307+0000 7f53f919c700 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55ea580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f53ec015000 tx=0x7f53ec009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=635c9f7b1e63b8b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.308+0000 7f53f1ffb700 1 -- 172.21.15.66:0/3512127305 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f53e0014070 con 0x7f53f4100480 2023-07-21T22:27:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.308+0000 7f53f899b700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f53e40a12c0 0x7f53e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.308+0000 7f53deffd700 1 -- 172.21.15.66:0/3087547324 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00e040 con 0x556cc55ea1b0 2023-07-21T22:27:50.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.308+0000 7f53deffd700 1 -- 172.21.15.66:0/3087547324 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53ec00bcb0 con 0x556cc55ea1b0 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.308+0000 7f53deffd700 1 -- 172.21.15.66:0/3087547324 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec01c6a0 con 0x556cc55ea1b0 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 -- 172.21.15.66:0/3087547324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 msgr2=0x556cc55ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f53f899b700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f53e40a12c0 0x7f53e40a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x556cc55e65b0 tx=0x7f53e8009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55ea580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f53ec015000 tx=0x7f53ec009c20 comp rx=0 tx=0).stop 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 -- 172.21.15.66:0/3087547324 shutdown_connections 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55ea580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3087547324 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55e5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 -- 172.21.15.66:0/3087547324 >> 172.21.15.66:0/3087547324 conn(0x556cc55daef0 msgr2=0x556cc55e0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 -- 172.21.15.66:0/3087547324 shutdown_connections 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 -- 172.21.15.66:0/3087547324 wait complete. 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.309+0000 7f540e57eb80 1 -- start start 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 Processor -- start 2023-07-21T22:27:50.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55ea1b0 0x556cc55ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55eaac0 0x556cc55ef950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc54d6ac0 con 0x556cc55ea1b0 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc54d67c0 con 0x556cc55eaac0 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc53ba920 con 0x556cc55e5930 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f540e57eb80 10 client.0 _ll_register_callbacks cb 0x556cc54d4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f53f899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f53f899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58132/0 (socket says 172.21.15.66:58132) 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f53f899b700 1 -- 172.21.15.66:0/3218197681 learned_addr learned my addr 172.21.15.66:0/3218197681 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f53f919c700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55eaac0 0x556cc55ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.310+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55ea1b0 0x556cc55ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53f899b700 1 -- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55eaac0 msgr2=0x556cc55ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53f899b700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55eaac0 0x556cc55ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53f899b700 1 -- 172.21.15.66:0/3218197681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55ea1b0 msgr2=0x556cc55ef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53f899b700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55ea1b0 0x556cc55ef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53f899b700 1 -- 172.21.15.66:0/3218197681 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f410d210 con 0x556cc55e5930 2023-07-21T22:27:50.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53f899b700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55eeed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53e801d9c0 tx=0x7f53e80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3dcbcc2d269d396 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:27:50.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53d3fff700 1 -- 172.21.15.66:0/3218197681 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8022070 con 0x556cc55e5930 2023-07-21T22:27:50.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cc54904f0 con 0x556cc55e5930 2023-07-21T22:27:50.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.311+0000 7f53d3fff700 1 -- 172.21.15.66:0/3218197681 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f53e801dd80 con 0x556cc55e5930 2023-07-21T22:27:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f53d3fff700 1 -- 172.21.15.66:0/3218197681 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e8017070 con 0x556cc55e5930 2023-07-21T22:27:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 10 client.26647 fetch_fsmap learned FSMap version 35 2023-07-21T22:27:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556cc534dc60 con 0x556cc55e5930 2023-07-21T22:27:50.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f53d3fff700 1 -- 172.21.15.66:0/3218197681 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556cc54904f0 con 0x556cc55e5930 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f53d3fff700 1 -- 172.21.15.66:0/3218197681 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f53e801c670 con 0x556cc55e5930 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 10 client.26647 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 10 client.26647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 20 client.26647 populate_metadata read hostname 'smithi066' 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc55ff4b0 0x556cc56018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 4 client.26647 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.312+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cc552feb0 con 0x556cc55ff4b0 2023-07-21T22:27:50.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.313+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc55ff4b0 0x556cc56018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.313+0000 7f53d3fff700 10 client.26647 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.313+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc55ff4b0 0x556cc56018c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685522e53978f875 server_cookie=41feafaf67869fe0 in_seq=0 out_seq=0 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f53d3fff700 1 -- 172.21.15.66:0/3218197681 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556cc552feb0 con 0x556cc55ff4b0 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f53d3fff700 10 client.26647 handle_command_reply: tid=0 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 client.26647 shutdown 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 msgr2=0x556cc55eeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55eeed0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f53e801d9c0 tx=0x7f53e80047c0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 shutdown_connections 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55eaac0 0x556cc55ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc55ff4b0 0x556cc56018c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55eeed0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3218197681 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55ea1b0 0x556cc55ef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f53d3fff700 0 client.26647 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 >> 172.21.15.66:0/3218197681 conn(0x556cc55daef0 msgr2=0x556cc55e0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 shutdown_connections 2023-07-21T22:27:50.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.314+0000 7f540e57eb80 1 -- 172.21.15.66:0/3218197681 wait complete. 2023-07-21T22:27:50.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.315+0000 7f540e57eb80 20 client.26647 trim_cache size 0 max 16384 2023-07-21T22:27:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:49 smithi104 ceph-mon[129021]: pgmap v193: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 975 KiB/s wr, 58 op/s 2023-07-21T22:27:50.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 Processor -- start 2023-07-21T22:27:50.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 -- start start 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 0x556cc55e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55ea1b0 0x556cc55f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc55dd010 con 0x556cc55eaac0 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc55e6400 con 0x556cc55e5930 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f540e57eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc55ff290 con 0x556cc55ea1b0 2023-07-21T22:27:50.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f53f899b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 0x556cc55e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f53f899b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 0x556cc55e5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50116/0 (socket says 172.21.15.66:50116) 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f53f899b700 1 -- 172.21.15.66:0/130651309 learned_addr learned my addr 172.21.15.66:0/130651309 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f53f919c700 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.333+0000 7f53f3fff700 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55ea1b0 0x556cc55f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53f899b700 1 -- 172.21.15.66:0/130651309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55ea1b0 msgr2=0x556cc55f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53f899b700 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55ea1b0 0x556cc55f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53f899b700 1 -- 172.21.15.66:0/130651309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 msgr2=0x556cc55ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53f899b700 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53f899b700 1 -- 172.21.15.66:0/130651309 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cc538b250 con 0x556cc55e5930 2023-07-21T22:27:50.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53f899b700 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 0x556cc55e5120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f53e8012af0 tx=0x7f53e8012b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813e5ff6c72530de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53df7fe700 1 -- 172.21.15.66:0/130651309 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8022070 con 0x556cc55e5930 2023-07-21T22:27:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53df7fe700 1 -- 172.21.15.66:0/130651309 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53e801c860 con 0x556cc55e5930 2023-07-21T22:27:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.334+0000 7f53df7fe700 1 -- 172.21.15.66:0/130651309 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e800fa60 con 0x556cc55e5930 2023-07-21T22:27:50.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 -- 172.21.15.66:0/130651309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 msgr2=0x556cc55e5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 0x556cc55e5120 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f53e8012af0 tx=0x7f53e8012b20 comp rx=0 tx=0).stop 2023-07-21T22:27:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 -- 172.21.15.66:0/130651309 shutdown_connections 2023-07-21T22:27:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55e5930 0x556cc55e5120 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 --2- 172.21.15.66:0/130651309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55ea1b0 0x556cc55f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 -- 172.21.15.66:0/130651309 >> 172.21.15.66:0/130651309 conn(0x556cc55daef0 msgr2=0x556cc55e4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 -- 172.21.15.66:0/130651309 shutdown_connections 2023-07-21T22:27:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.335+0000 7f540e57eb80 1 -- 172.21.15.66:0/130651309 wait complete. 2023-07-21T22:27:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 -- start start 2023-07-21T22:27:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 Processor -- start 2023-07-21T22:27:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc56533d0 con 0x556cc55eaac0 2023-07-21T22:27:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc5653510 con 0x556cc55ea1b0 2023-07-21T22:27:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cc56536d0 con 0x556cc55e5930 2023-07-21T22:27:50.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.336+0000 7f540e57eb80 10 client.0 _ll_register_callbacks cb 0x7f53e801d7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58140/0 (socket says 172.21.15.66:58140) 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 -- 172.21.15.66:0/3455899608 learned_addr learned my addr 172.21.15.66:0/3455899608 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:50128/0 (socket says 172.21.15.66:50128) 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f919c700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 -- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 msgr2=0x556cc55f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 -- 172.21.15.66:0/3455899608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 msgr2=0x556cc55f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 -- 172.21.15.66:0/3455899608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cc534dc60 con 0x556cc55e5930 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53f899b700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55f4c60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f53e8019040 tx=0x7f53e801cbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dba230307b8e314c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:27:50.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.337+0000 7f53d3fff700 1 -- 172.21.15.66:0/3455899608 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8022040 con 0x556cc55e5930 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cc54f2050 con 0x556cc55e5930 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f53d3fff700 1 -- 172.21.15.66:0/3455899608 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f53e801c7b0 con 0x556cc55e5930 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f53d3fff700 1 -- 172.21.15.66:0/3455899608 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e800f840 con 0x556cc55e5930 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 10 client.26653 fetch_fsmap learned FSMap version 35 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556cc53632b0 con 0x556cc55e5930 2023-07-21T22:27:50.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f53d3fff700 1 -- 172.21.15.66:0/3455899608 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556cc54f2050 con 0x556cc55e5930 2023-07-21T22:27:50.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f53d3fff700 1 -- 172.21.15.66:0/3455899608 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 35) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f53e803f070 con 0x556cc55e5930 2023-07-21T22:27:50.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 10 client.26653 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T22:27:50.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:27:50.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 20 client.26653 populate_metadata read hostname 'smithi066' 2023-07-21T22:27:50.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc5654a50 0x556cc5656e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:27:50.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 4 client.26653 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:27:50.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.338+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556cc5385f70 con 0x556cc5654a50 2023-07-21T22:27:50.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.339+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc5654a50 0x556cc5656e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:27:50.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.339+0000 7f53d3fff700 10 client.26653 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:50.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.339+0000 7f53f3fff700 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc5654a50 0x556cc5656e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ae2fd070d345c0 server_cookie=3852900a07937df6 in_seq=0 out_seq=0 2023-07-21T22:27:50.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f53d3fff700 1 -- 172.21.15.66:0/3455899608 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556cc5385f70 con 0x556cc5654a50 2023-07-21T22:27:50.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f53d3fff700 10 client.26653 handle_command_reply: tid=0 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 client.26653 shutdown 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 msgr2=0x556cc55f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55f4c60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f53e8019040 tx=0x7f53e801cbd0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 shutdown_connections 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cc5654a50 0x556cc5656e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cc55eaac0 0x556cc55f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cc55e5930 0x556cc55f4c60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 --2- 172.21.15.66:0/3455899608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cc55ea1b0 0x556cc55f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f53d3fff700 0 client.26653 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:27:50.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.340+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 >> 172.21.15.66:0/3455899608 conn(0x556cc55daef0 msgr2=0x556cc55ed680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 shutdown_connections 2023-07-21T22:27:50.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f540e57eb80 1 -- 172.21.15.66:0/3455899608 wait complete. 2023-07-21T22:27:50.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f540e57eb80 20 client.26653 trim_cache size 0 max 16384 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:27:50.399 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:27:50.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:27:50.400 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:27:50.400 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:27:50.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f53e40a12c0 msgr2=0x7f53e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f53e40a12c0 0x7f53e40a3770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x556cc55e65b0 tx=0x7f53e8009260 comp rx=0 tx=0).stop 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 msgr2=0x7f53f4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f4198470 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f53e000e9a0 tx=0x7f53e000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 shutdown_connections 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f53e40a12c0 0x7f53e40a3770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f53f4105ae0 0x7f53f419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.341+0000 7f53fabff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f53f40ffad0 0x7f53f4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.342+0000 7f53fabff700 1 --2- 172.21.15.66:0/3512127305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53f4100480 0x7f53f4198470 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:27:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.342+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 >> 172.21.15.66:0/3512127305 conn(0x7f53f40fb5f0 msgr2=0x7f53f410aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:27:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.342+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 shutdown_connections 2023-07-21T22:27:50.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:27:50.342+0000 7f53fabff700 1 -- 172.21.15.66:0/3512127305 wait complete. 2023-07-21T22:27:50.405 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:49 smithi066 ceph-mon[126379]: pgmap v193: 129 pgs: 129 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 975 KiB/s wr, 58 op/s 2023-07-21T22:27:50.408 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:27:50.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 251088 5.59 MB/sec execute 197 sec latency 2863.850 ms 2023-07-21T22:27:51.541 INFO:tasks.workunit.client.0.smithi066.stdout: 1 252473 5.60 MB/sec execute 198 sec latency 12.086 ms 2023-07-21T22:27:52.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:51 smithi149 ceph-mon[129268]: pgmap v194: 129 pgs: 129 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2023-07-21T22:27:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:51 smithi104 ceph-mon[129021]: pgmap v194: 129 pgs: 129 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2023-07-21T22:27:52.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:51 smithi066 ceph-mon[126379]: pgmap v194: 129 pgs: 129 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.3 MiB/s wr, 100 op/s 2023-07-21T22:27:52.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 253572 5.60 MB/sec execute 199 sec latency 168.920 ms 2023-07-21T22:27:53.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 253572 5.57 MB/sec execute 200 sec latency 1168.982 ms 2023-07-21T22:27:54.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:53 smithi149 ceph-mon[129268]: pgmap v195: 129 pgs: 129 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 585 KiB/s wr, 64 op/s 2023-07-21T22:27:54.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:53 smithi104 ceph-mon[129021]: pgmap v195: 129 pgs: 129 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 585 KiB/s wr, 64 op/s 2023-07-21T22:27:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:53 smithi066 ceph-mon[126379]: pgmap v195: 129 pgs: 129 active+clean; 187 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 585 KiB/s wr, 64 op/s 2023-07-21T22:27:54.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 253572 5.54 MB/sec execute 201 sec latency 2169.114 ms 2023-07-21T22:27:55.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 254475 5.53 MB/sec execute 202 sec latency 2811.813 ms 2023-07-21T22:27:56.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:55 smithi149 ceph-mon[129268]: pgmap v196: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2023-07-21T22:27:56.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:55 smithi104 ceph-mon[129021]: pgmap v196: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2023-07-21T22:27:56.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:55 smithi066 ceph-mon[126379]: pgmap v196: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2023-07-21T22:27:56.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 255975 5.56 MB/sec execute 203 sec latency 12.561 ms 2023-07-21T22:27:57.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 257113 5.55 MB/sec execute 204 sec latency 209.070 ms 2023-07-21T22:27:58.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:57 smithi149 ceph-mon[129268]: pgmap v197: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-21T22:27:58.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:57 smithi104 ceph-mon[129021]: pgmap v197: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-21T22:27:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:57 smithi066 ceph-mon[126379]: pgmap v197: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-21T22:27:58.542 INFO:tasks.workunit.client.0.smithi066.stdout: 1 257113 5.53 MB/sec execute 205 sec latency 1209.199 ms 2023-07-21T22:27:59.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 257113 5.50 MB/sec execute 206 sec latency 2209.349 ms 2023-07-21T22:28:00.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:27:59 smithi149 ceph-mon[129268]: pgmap v198: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-21T22:28:00.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:27:59 smithi104 ceph-mon[129021]: pgmap v198: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-21T22:28:00.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:27:59 smithi066 ceph-mon[126379]: pgmap v198: 129 pgs: 129 active+clean; 189 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.3 MiB/s wr, 99 op/s 2023-07-21T22:28:00.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 258012 5.49 MB/sec execute 207 sec latency 2858.658 ms 2023-07-21T22:28:01.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 259160 5.51 MB/sec execute 208 sec latency 61.439 ms 2023-07-21T22:28:02.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:01 smithi149 ceph-mon[129268]: pgmap v199: 129 pgs: 129 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2023-07-21T22:28:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:01 smithi104 ceph-mon[129021]: pgmap v199: 129 pgs: 129 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2023-07-21T22:28:02.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:01 smithi066 ceph-mon[126379]: pgmap v199: 129 pgs: 129 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.2 MiB/s wr, 152 op/s 2023-07-21T22:28:02.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 259707 5.49 MB/sec execute 209 sec latency 75.424 ms 2023-07-21T22:28:03.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 260661 5.49 MB/sec execute 210 sec latency 343.337 ms 2023-07-21T22:28:04.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:03 smithi149 ceph-mon[129268]: pgmap v200: 129 pgs: 129 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.8 MiB/s wr, 110 op/s 2023-07-21T22:28:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:03 smithi104 ceph-mon[129021]: pgmap v200: 129 pgs: 129 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.8 MiB/s wr, 110 op/s 2023-07-21T22:28:04.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:03 smithi066 ceph-mon[126379]: pgmap v200: 129 pgs: 129 active+clean; 191 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.8 MiB/s wr, 110 op/s 2023-07-21T22:28:04.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 260661 5.46 MB/sec execute 211 sec latency 1343.431 ms 2023-07-21T22:28:05.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 261527 5.45 MB/sec execute 212 sec latency 1983.072 ms 2023-07-21T22:28:06.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:05 smithi104 ceph-mon[129021]: pgmap v201: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-21T22:28:06.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:05 smithi066 ceph-mon[126379]: pgmap v201: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-21T22:28:06.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:05 smithi149 ceph-mon[129268]: pgmap v201: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2023-07-21T22:28:06.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.47 MB/sec execute 213 sec latency 16.900 ms 2023-07-21T22:28:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:06 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:28:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:06 smithi104 ceph-mon[129021]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T22:28:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:06 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:28:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:06 smithi066 ceph-mon[126379]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T22:28:07.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:06 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:28:07.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:06 smithi149 ceph-mon[129268]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T22:28:07.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.45 MB/sec execute 214 sec latency 1002.254 ms 2023-07-21T22:28:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:07 smithi104 ceph-mon[129021]: pgmap v203: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2023-07-21T22:28:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:07 smithi066 ceph-mon[126379]: pgmap v203: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2023-07-21T22:28:08.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:07 smithi149 ceph-mon[129268]: pgmap v203: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2023-07-21T22:28:08.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.42 MB/sec execute 215 sec latency 2002.399 ms 2023-07-21T22:28:09.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.40 MB/sec execute 216 sec latency 3002.497 ms 2023-07-21T22:28:10.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:09 smithi104 ceph-mon[129021]: pgmap v204: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2023-07-21T22:28:10.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:09 smithi066 ceph-mon[126379]: pgmap v204: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2023-07-21T22:28:10.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:09 smithi149 ceph-mon[129268]: pgmap v204: 129 pgs: 129 active+clean; 195 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 2.2 MiB/s wr, 171 op/s 2023-07-21T22:28:10.543 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.37 MB/sec execute 217 sec latency 4002.597 ms 2023-07-21T22:28:11.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.35 MB/sec execute 218 sec latency 5002.724 ms 2023-07-21T22:28:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:11 smithi104 ceph-mon[129021]: pgmap v205: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-21T22:28:12.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:11 smithi066 ceph-mon[126379]: pgmap v205: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-21T22:28:12.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:11 smithi149 ceph-mon[129268]: pgmap v205: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-21T22:28:12.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.32 MB/sec execute 219 sec latency 6002.863 ms 2023-07-21T22:28:13.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.30 MB/sec execute 220 sec latency 7003.019 ms 2023-07-21T22:28:14.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:13 smithi066 ceph-mon[126379]: pgmap v206: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-21T22:28:14.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:13 smithi066 ceph-mon[126379]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T22:28:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:13 smithi149 ceph-mon[129268]: pgmap v206: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-21T22:28:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:13 smithi149 ceph-mon[129268]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T22:28:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:13 smithi104 ceph-mon[129021]: pgmap v206: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 2.1 MiB/s wr, 151 op/s 2023-07-21T22:28:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:13 smithi104 ceph-mon[129021]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T22:28:14.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.28 MB/sec execute 221 sec latency 8003.160 ms 2023-07-21T22:28:15.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.25 MB/sec execute 222 sec latency 9003.232 ms 2023-07-21T22:28:16.196 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:15 smithi066 ceph-mon[126379]: pgmap v208: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 997 KiB/s wr, 65 op/s 2023-07-21T22:28:16.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:15 smithi149 ceph-mon[129268]: pgmap v208: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 997 KiB/s wr, 65 op/s 2023-07-21T22:28:16.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:15 smithi104 ceph-mon[129021]: pgmap v208: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 997 KiB/s wr, 65 op/s 2023-07-21T22:28:16.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.23 MB/sec execute 223 sec latency 10003.344 ms 2023-07-21T22:28:17.544 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.21 MB/sec execute 224 sec latency 11003.468 ms 2023-07-21T22:28:18.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:17 smithi066 ceph-mon[126379]: pgmap v209: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 886 KiB/s wr, 57 op/s 2023-07-21T22:28:18.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:17 smithi149 ceph-mon[129268]: pgmap v209: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 886 KiB/s wr, 57 op/s 2023-07-21T22:28:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:17 smithi104 ceph-mon[129021]: pgmap v209: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 886 KiB/s wr, 57 op/s 2023-07-21T22:28:18.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.18 MB/sec execute 225 sec latency 12003.616 ms 2023-07-21T22:28:19.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.16 MB/sec execute 226 sec latency 13003.738 ms 2023-07-21T22:28:20.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:19 smithi066 ceph-mon[126379]: pgmap v210: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 886 KiB/s wr, 57 op/s 2023-07-21T22:28:20.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:19 smithi149 ceph-mon[129268]: pgmap v210: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 886 KiB/s wr, 57 op/s 2023-07-21T22:28:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:19 smithi104 ceph-mon[129021]: pgmap v210: 129 pgs: 129 active+clean; 196 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 886 KiB/s wr, 57 op/s 2023-07-21T22:28:20.408 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:28:20.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263038 5.14 MB/sec execute 227 sec latency 14003.800 ms 2023-07-21T22:28:20.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 -- 172.21.15.66:0/3302584436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908105ae0 msgr2=0x7f1908105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 --2- 172.21.15.66:0/3302584436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908105ae0 0x7f1908105eb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1904009f90 tx=0x7f190400b330 comp rx=0 tx=0).stop 2023-07-21T22:28:20.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 -- 172.21.15.66:0/3302584436 shutdown_connections 2023-07-21T22:28:20.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 --2- 172.21.15.66:0/3302584436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908100480 0x7f190810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 --2- 172.21.15.66:0/3302584436 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908105ae0 0x7f1908105eb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 --2- 172.21.15.66:0/3302584436 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f19080fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 -- 172.21.15.66:0/3302584436 >> 172.21.15.66:0/3302584436 conn(0x7f19080fb5f0 msgr2=0x7f19080fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 -- 172.21.15.66:0/3302584436 shutdown_connections 2023-07-21T22:28:20.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.592+0000 7f19101ff700 1 -- 172.21.15.66:0/3302584436 wait complete. 2023-07-21T22:28:20.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.593+0000 7f19101ff700 1 Processor -- start 2023-07-21T22:28:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.593+0000 7f19101ff700 1 -- start start 2023-07-21T22:28:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f19101ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f1908197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f19101ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908100480 0x7f1908198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f19101ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908105ae0 0x7f190819c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f19101ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190819cf20 con 0x7f1908100480 2023-07-21T22:28:20.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f19101ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190819d090 con 0x7f1908105ae0 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f19101ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190819d250 con 0x7f19080ffad0 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f1908197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f1908197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:42362/0 (socket says 172.21.15.66:42362) 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 -- 172.21.15.66:0/609947360 learned_addr learned my addr 172.21.15.66:0/609947360 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190d79a700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908100480 0x7f1908198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190e79c700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908105ae0 0x7f190819c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 -- 172.21.15.66:0/609947360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908105ae0 msgr2=0x7f190819c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908105ae0 0x7f190819c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 -- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908100480 msgr2=0x7f1908198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908100480 0x7f1908198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.594+0000 7f190df9b700 1 -- 172.21.15.66:0/609947360 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1904009c40 con 0x7f19080ffad0 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.595+0000 7f190df9b700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f1908197f40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f190400b6c0 tx=0x7f1904004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.595+0000 7f18feffd700 1 -- 172.21.15.66:0/609947360 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190400e040 con 0x7f19080ffad0 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.595+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190819d4d0 con 0x7f19080ffad0 2023-07-21T22:28:20.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.595+0000 7f18feffd700 1 -- 172.21.15.66:0/609947360 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f190400bbf0 con 0x7f19080ffad0 2023-07-21T22:28:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.595+0000 7f18feffd700 1 -- 172.21.15.66:0/609947360 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040052a0 con 0x7f19080ffad0 2023-07-21T22:28:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.595+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081a4d70 con 0x7f19080ffad0 2023-07-21T22:28:20.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.599+0000 7f18feffd700 1 -- 172.21.15.66:0/609947360 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f190401d470 con 0x7f19080ffad0 2023-07-21T22:28:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.600+0000 7f1923ae1b80 1 Processor -- start 2023-07-21T22:28:20.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.600+0000 7f1923ae1b80 1 -- start start 2023-07-21T22:28:20.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 0x557f7334b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f73346350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73346da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7324bcb0 con 0x557f73346930 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7329ea40 con 0x557f7334b1b0 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f73297ed0 con 0x557f7334bac0 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 0x557f7334b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 0x557f7334b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:60794/0 (socket says 172.21.15.66:60794) 2023-07-21T22:28:20.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 -- 172.21.15.66:0/3355254678 learned_addr learned my addr 172.21.15.66:0/3355254678 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:20.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190d79a700 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73346da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190df9b700 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f73346350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f18feffd700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18f40a12c0 0x7f18f40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 -- 172.21.15.66:0/3355254678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 msgr2=0x557f73346350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f73346350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 -- 172.21.15.66:0/3355254678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 msgr2=0x557f73346da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73346da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.601+0000 7f190e79c700 1 -- 172.21.15.66:0/3355254678 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f730c5de0 con 0x557f7334b1b0 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.602+0000 7f18feffd700 1 -- 172.21.15.66:0/609947360 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19040ce540 con 0x7f19080ffad0 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.602+0000 7f190e79c700 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 0x557f7334b580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1900009c70 tx=0x7f1900009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=539c593eacde23ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.602+0000 7f190e79c700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18f40a12c0 0x7f18f40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.602+0000 7f18f3fff700 1 -- 172.21.15.66:0/3355254678 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900021070 con 0x557f7334b1b0 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.602+0000 7f18f3fff700 1 -- 172.21.15.66:0/3355254678 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1900004460 con 0x557f7334b1b0 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.602+0000 7f18f3fff700 1 -- 172.21.15.66:0/3355254678 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190001cbf0 con 0x557f7334b1b0 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3355254678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 msgr2=0x557f7334b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 0x557f7334b580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1900009c70 tx=0x7f1900009f80 comp rx=0 tx=0).stop 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f190e79c700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18f40a12c0 0x7f18f40a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1900011fd0 tx=0x7f1900021000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3355254678 shutdown_connections 2023-07-21T22:28:20.652 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7334b1b0 0x557f7334b580 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73346da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3355254678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f73346350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3355254678 >> 172.21.15.66:0/3355254678 conn(0x557f7333bef0 msgr2=0x557f73341830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3355254678 shutdown_connections 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.603+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3355254678 wait complete. 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 -- start start 2023-07-21T22:28:20.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 Processor -- start 2023-07-21T22:28:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 0x557f73352be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f7334b1b0 0x557f7334fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f733505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f73237ac0 con 0x557f7334b1b0 2023-07-21T22:28:20.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f732377c0 con 0x557f73346930 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.604+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f7311b920 con 0x557f7334bac0 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f1923ae1b80 10 client.0 _ll_register_callbacks cb 0x557f73235760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 0x557f73352be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190d79a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f7334b1b0 0x557f7334fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 0x557f73352be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:60806/0 (socket says 172.21.15.66:60806) 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 -- 172.21.15.66:0/3757594955 learned_addr learned my addr 172.21.15.66:0/3757594955 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:20.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190e79c700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f733505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 -- 172.21.15.66:0/3757594955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 msgr2=0x557f733505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f733505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 -- 172.21.15.66:0/3757594955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f7334b1b0 msgr2=0x557f7334fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f7334b1b0 0x557f7334fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 -- 172.21.15.66:0/3757594955 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1900009920 con 0x557f73346930 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190d79a700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f7334b1b0 0x557f7334fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f190df9b700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 0x557f73352be0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f190409afa0 tx=0x7f190401db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8fe5ec38434c86 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.605+0000 7f18f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f18f17fa700 1 -- 172.21.15.66:0/3757594955 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190409b070 con 0x557f73346930 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f18f17fa700 1 -- 172.21.15.66:0/3757594955 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f190401de40 con 0x557f73346930 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f731f14f0 con 0x557f73346930 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f18f17fa700 1 -- 172.21.15.66:0/3757594955 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f190409ade0 con 0x557f73346930 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f1923ae1b80 10 client.26687 fetch_fsmap learned FSMap version 36 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557f730aec60 con 0x557f73346930 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f18f17fa700 1 -- 172.21.15.66:0/3757594955 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f731f14f0 con 0x557f73346930 2023-07-21T22:28:20.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f18f17fa700 1 -- 172.21.15.66:0/3757594955 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f19040ce160 con 0x557f73346930 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f1923ae1b80 10 client.26687 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f1923ae1b80 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.606+0000 7f1923ae1b80 20 client.26687 populate_metadata read hostname 'smithi066' 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.607+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f73360290 0x557f733626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.607+0000 7f1923ae1b80 4 client.26687 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.607+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f73290eb0 con 0x557f73360290 2023-07-21T22:28:20.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.607+0000 7f190d79a700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f73360290 0x557f733626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.607+0000 7f190d79a700 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f73360290 0x557f733626a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d7c9baf29d6546 server_cookie=b92f3b6e33138496 in_seq=0 out_seq=0 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.607+0000 7f18f17fa700 10 client.26687 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f18f17fa700 1 -- 172.21.15.66:0/3757594955 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f73290eb0 con 0x557f73360290 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f18f17fa700 10 client.26687 handle_command_reply: tid=0 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 client.26687 shutdown 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 msgr2=0x557f73352be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 0x557f73352be0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f190409afa0 tx=0x7f190401db40 comp rx=0 tx=0).stop 2023-07-21T22:28:20.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 shutdown_connections 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f73346930 0x557f73352be0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f73360290 0x557f733626a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334bac0 0x557f733505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3757594955 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f7334b1b0 0x557f7334fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 >> 172.21.15.66:0/3757594955 conn(0x557f7333bef0 msgr2=0x557f73341420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f18f17fa700 0 client.26687 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 shutdown_connections 2023-07-21T22:28:20.662 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.608+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3757594955 wait complete. 2023-07-21T22:28:20.663 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.609+0000 7f1923ae1b80 20 client.26687 trim_cache size 0 max 16384 2023-07-21T22:28:20.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 Processor -- start 2023-07-21T22:28:20.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 -- start start 2023-07-21T22:28:20.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f7334fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73357570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f733459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f733474f0 con 0x557f73346930 2023-07-21T22:28:20.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f73351320 con 0x557f7333bef0 2023-07-21T22:28:20.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.626+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f73342e50 con 0x557f7334b1b0 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f7334fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f7334fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:60822/0 (socket says 172.21.15.66:60822) 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 -- 172.21.15.66:0/3353341304 learned_addr learned my addr 172.21.15.66:0/3353341304 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190d79a700 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73357570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190e79c700 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f733459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 -- 172.21.15.66:0/3353341304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 msgr2=0x557f733459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f733459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 -- 172.21.15.66:0/3353341304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 msgr2=0x557f73357570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73357570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 -- 172.21.15.66:0/3353341304 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f730ec250 con 0x557f7333bef0 2023-07-21T22:28:20.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.627+0000 7f190df9b700 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f7334fe30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1904004b50 tx=0x7f19040ce780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5971f9ff2e15f6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f18fcff9700 1 -- 172.21.15.66:0/3353341304 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040cbd10 con 0x557f7333bef0 2023-07-21T22:28:20.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f18fcff9700 1 -- 172.21.15.66:0/3353341304 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19040a3660 con 0x557f7333bef0 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f18fcff9700 1 -- 172.21.15.66:0/3353341304 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190409a9b0 con 0x557f7333bef0 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3353341304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 msgr2=0x557f7334fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f7334fe30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1904004b50 tx=0x7f19040ce780 comp rx=0 tx=0).stop 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3353341304 shutdown_connections 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f7334fe30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f733459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/3353341304 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73357570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3353341304 >> 172.21.15.66:0/3353341304 conn(0x557f73347060 msgr2=0x557f7334f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3353341304 shutdown_connections 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.628+0000 7f1923ae1b80 1 -- 172.21.15.66:0/3353341304 wait complete. 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 -- start start 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 Processor -- start 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f73357580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73351700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f73351c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f73352180 con 0x557f73346930 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f733522f0 con 0x557f7333bef0 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.629+0000 7f1923ae1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f733524b0 con 0x557f7334b1b0 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f1923ae1b80 10 client.0 _ll_register_callbacks cb 0x557f73235760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f73357580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f73357580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:60830/0 (socket says 172.21.15.66:60830) 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 -- 172.21.15.66:0/846087712 learned_addr learned my addr 172.21.15.66:0/846087712 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:20.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190d79a700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73351700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190e79c700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f73351c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 -- 172.21.15.66:0/846087712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 msgr2=0x557f73351c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f73351c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 -- 172.21.15.66:0/846087712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 msgr2=0x557f73351700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73351700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 -- 172.21.15.66:0/846087712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f730aec60 con 0x557f7333bef0 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f190df9b700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f73357580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f190409b230 tx=0x7f19040a3480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c424de9d90964bb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:20.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f18f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f18f17fa700 1 -- 172.21.15.66:0/846087712 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904009ac0 con 0x557f7333bef0 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1900004460 con 0x557f7333bef0 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.630+0000 7f18f17fa700 1 -- 172.21.15.66:0/846087712 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f190401de40 con 0x557f7333bef0 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f18f17fa700 1 -- 172.21.15.66:0/846087712 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f190409ade0 con 0x557f7333bef0 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 10 client.26693 fetch_fsmap learned FSMap version 36 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x557f730c42b0 con 0x557f7333bef0 2023-07-21T22:28:20.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f18f17fa700 1 -- 172.21.15.66:0/846087712 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19040c9020 con 0x557f7333bef0 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f18f17fa700 1 -- 172.21.15.66:0/846087712 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f190409b500 con 0x557f7333bef0 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 10 client.26693 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 20 client.26693 populate_metadata read hostname 'smithi066' 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f7334bac0 0x557f733b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 4 client.26693 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f73253050 con 0x557f7334bac0 2023-07-21T22:28:20.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.631+0000 7f190d79a700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f7334bac0 0x557f733b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:20.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f18f17fa700 10 client.26693 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:20.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f190d79a700 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f7334bac0 0x557f733b70c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6730a811ba3d4a0 server_cookie=fb0e8ec58548293f in_seq=0 out_seq=0 2023-07-21T22:28:20.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f18f17fa700 1 -- 172.21.15.66:0/846087712 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557f73253050 con 0x557f7334bac0 2023-07-21T22:28:20.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f18f17fa700 10 client.26693 handle_command_reply: tid=0 2023-07-21T22:28:20.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 client.26693 shutdown 2023-07-21T22:28:20.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 msgr2=0x557f73357580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f73357580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f190409b230 tx=0x7f19040a3480 comp rx=0 tx=0).stop 2023-07-21T22:28:20.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 shutdown_connections 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557f7333bef0 0x557f73357580 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557f7334bac0 0x557f733b70c0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557f7334b1b0 0x557f73351c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 --2- 172.21.15.66:0/846087712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557f73346930 0x557f73351700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 >> 172.21.15.66:0/846087712 conn(0x557f73347060 msgr2=0x557f733459b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f18f17fa700 0 client.26693 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 shutdown_connections 2023-07-21T22:28:20.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.632+0000 7f1923ae1b80 1 -- 172.21.15.66:0/846087712 wait complete. 2023-07-21T22:28:20.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f1923ae1b80 20 client.26693 trim_cache size 0 max 16384 2023-07-21T22:28:20.688 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:28:20.688 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:28:20.688 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:28:20.688 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:28:20.689 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:28:20.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:28:20.689 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:28:20.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:28:20.689 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:28:20.689 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:28:20.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18f40a12c0 msgr2=0x7f18f40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18f40a12c0 0x7f18f40a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1900011fd0 tx=0x7f1900021000 comp rx=0 tx=0).stop 2023-07-21T22:28:20.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 msgr2=0x7f1908197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:20.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f1908197f40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f190400b6c0 tx=0x7f1904004a30 comp rx=0 tx=0).stop 2023-07-21T22:28:20.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 shutdown_connections 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1908105ae0 0x7f190819c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18f40a12c0 0x7f18f40a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19080ffad0 0x7f1908197f40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 --2- 172.21.15.66:0/609947360 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1908100480 0x7f1908198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 >> 172.21.15.66:0/609947360 conn(0x7f19080fb5f0 msgr2=0x7f19080fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 shutdown_connections 2023-07-21T22:28:20.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:20.633+0000 7f19101ff700 1 -- 172.21.15.66:0/609947360 wait complete. 2023-07-21T22:28:20.698 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:28:21.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263581 5.12 MB/sec execute 228 sec latency 14333.569 ms 2023-07-21T22:28:22.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:21 smithi066 ceph-mon[126379]: pgmap v211: 129 pgs: 129 active+clean; 201 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 686 KiB/s wr, 29 op/s 2023-07-21T22:28:22.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:21 smithi149 ceph-mon[129268]: pgmap v211: 129 pgs: 129 active+clean; 201 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 686 KiB/s wr, 29 op/s 2023-07-21T22:28:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:21 smithi104 ceph-mon[129021]: pgmap v211: 129 pgs: 129 active+clean; 201 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 686 KiB/s wr, 29 op/s 2023-07-21T22:28:22.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263581 5.10 MB/sec execute 229 sec latency 1172.119 ms 2023-07-21T22:28:23.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263581 5.08 MB/sec execute 230 sec latency 2172.217 ms 2023-07-21T22:28:24.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:23 smithi066 ceph-mon[126379]: pgmap v212: 129 pgs: 129 active+clean; 201 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 686 KiB/s wr, 29 op/s 2023-07-21T22:28:24.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:23 smithi149 ceph-mon[129268]: pgmap v212: 129 pgs: 129 active+clean; 201 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 686 KiB/s wr, 29 op/s 2023-07-21T22:28:24.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:23 smithi104 ceph-mon[129021]: pgmap v212: 129 pgs: 129 active+clean; 201 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 686 KiB/s wr, 29 op/s 2023-07-21T22:28:24.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 263581 5.06 MB/sec execute 231 sec latency 3172.366 ms 2023-07-21T22:28:25.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 264171 5.05 MB/sec execute 232 sec latency 3813.960 ms 2023-07-21T22:28:26.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:25 smithi149 ceph-mon[129268]: pgmap v213: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 952 KiB/s wr, 49 op/s 2023-07-21T22:28:26.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:25 smithi104 ceph-mon[129021]: pgmap v213: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 952 KiB/s wr, 49 op/s 2023-07-21T22:28:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:25 smithi066 ceph-mon[126379]: pgmap v213: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 952 KiB/s wr, 49 op/s 2023-07-21T22:28:26.545 INFO:tasks.workunit.client.0.smithi066.stdout: 1 264204 5.03 MB/sec execute 233 sec latency 972.492 ms 2023-07-21T22:28:27.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 264204 5.01 MB/sec execute 234 sec latency 1972.582 ms 2023-07-21T22:28:28.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:27 smithi149 ceph-mon[129268]: pgmap v214: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 901 KiB/s wr, 37 op/s 2023-07-21T22:28:28.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:27 smithi104 ceph-mon[129021]: pgmap v214: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 901 KiB/s wr, 37 op/s 2023-07-21T22:28:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:27 smithi066 ceph-mon[126379]: pgmap v214: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 901 KiB/s wr, 37 op/s 2023-07-21T22:28:28.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 264204 4.98 MB/sec execute 235 sec latency 2972.670 ms 2023-07-21T22:28:29.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 264204 4.96 MB/sec execute 236 sec latency 3972.785 ms 2023-07-21T22:28:30.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:29 smithi149 ceph-mon[129268]: pgmap v215: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 901 KiB/s wr, 37 op/s 2023-07-21T22:28:30.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:29 smithi104 ceph-mon[129021]: pgmap v215: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 901 KiB/s wr, 37 op/s 2023-07-21T22:28:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:29 smithi066 ceph-mon[126379]: pgmap v215: 129 pgs: 129 active+clean; 202 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 901 KiB/s wr, 37 op/s 2023-07-21T22:28:30.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 265114 4.95 MB/sec execute 237 sec latency 4613.359 ms 2023-07-21T22:28:31.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 266590 4.98 MB/sec execute 238 sec latency 14.117 ms 2023-07-21T22:28:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:31 smithi149 ceph-mon[129268]: pgmap v216: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 57 op/s 2023-07-21T22:28:32.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:31 smithi104 ceph-mon[129021]: pgmap v216: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 57 op/s 2023-07-21T22:28:32.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:31 smithi066 ceph-mon[126379]: pgmap v216: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.2 MiB/s wr, 57 op/s 2023-07-21T22:28:32.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 267746 4.98 MB/sec execute 239 sec latency 19.517 ms 2023-07-21T22:28:33.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 267752 4.96 MB/sec execute 240 sec latency 969.917 ms 2023-07-21T22:28:34.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:33 smithi149 ceph-mon[129268]: pgmap v217: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 681 KiB/s wr, 44 op/s 2023-07-21T22:28:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:33 smithi104 ceph-mon[129021]: pgmap v217: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 681 KiB/s wr, 44 op/s 2023-07-21T22:28:34.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:33 smithi066 ceph-mon[126379]: pgmap v217: 129 pgs: 129 active+clean; 203 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 681 KiB/s wr, 44 op/s 2023-07-21T22:28:34.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 267752 4.94 MB/sec execute 241 sec latency 1970.080 ms 2023-07-21T22:28:35.546 INFO:tasks.workunit.client.0.smithi066.stdout: 1 268668 4.93 MB/sec execute 242 sec latency 2605.399 ms 2023-07-21T22:28:36.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:35 smithi149 ceph-mon[129268]: pgmap v218: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2023-07-21T22:28:36.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:35 smithi104 ceph-mon[129021]: pgmap v218: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2023-07-21T22:28:36.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:35 smithi066 ceph-mon[126379]: pgmap v218: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2023-07-21T22:28:36.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 270208 4.96 MB/sec execute 243 sec latency 14.707 ms 2023-07-21T22:28:37.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 271294 4.96 MB/sec execute 244 sec latency 117.385 ms 2023-07-21T22:28:38.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:37 smithi149 ceph-mon[129268]: pgmap v219: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 749 KiB/s wr, 65 op/s 2023-07-21T22:28:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:37 smithi104 ceph-mon[129021]: pgmap v219: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 749 KiB/s wr, 65 op/s 2023-07-21T22:28:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:37 smithi066 ceph-mon[126379]: pgmap v219: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 749 KiB/s wr, 65 op/s 2023-07-21T22:28:38.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 271294 4.94 MB/sec execute 245 sec latency 1117.518 ms 2023-07-21T22:28:39.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 271294 4.92 MB/sec execute 246 sec latency 2117.619 ms 2023-07-21T22:28:40.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:39 smithi149 ceph-mon[129268]: pgmap v220: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 749 KiB/s wr, 65 op/s 2023-07-21T22:28:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:39 smithi104 ceph-mon[129021]: pgmap v220: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 749 KiB/s wr, 65 op/s 2023-07-21T22:28:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:39 smithi066 ceph-mon[126379]: pgmap v220: 129 pgs: 129 active+clean; 203 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 749 KiB/s wr, 65 op/s 2023-07-21T22:28:40.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 272204 4.91 MB/sec execute 247 sec latency 2758.084 ms 2023-07-21T22:28:41.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 273703 4.93 MB/sec execute 248 sec latency 19.742 ms 2023-07-21T22:28:42.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:41 smithi149 ceph-mon[129268]: pgmap v221: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2023-07-21T22:28:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:41 smithi104 ceph-mon[129021]: pgmap v221: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2023-07-21T22:28:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:41 smithi066 ceph-mon[126379]: pgmap v221: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 1.3 MiB/s wr, 108 op/s 2023-07-21T22:28:42.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 274843 4.93 MB/sec execute 249 sec latency 217.452 ms 2023-07-21T22:28:43.547 INFO:tasks.workunit.client.0.smithi066.stdout: 1 274843 4.91 MB/sec execute 250 sec latency 1217.582 ms 2023-07-21T22:28:44.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:43 smithi149 ceph-mon[129268]: pgmap v222: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1004 KiB/s wr, 88 op/s 2023-07-21T22:28:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:43 smithi104 ceph-mon[129021]: pgmap v222: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1004 KiB/s wr, 88 op/s 2023-07-21T22:28:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:43 smithi066 ceph-mon[126379]: pgmap v222: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 1004 KiB/s wr, 88 op/s 2023-07-21T22:28:44.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 274843 4.89 MB/sec execute 251 sec latency 2217.671 ms 2023-07-21T22:28:45.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:44 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:45.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:44 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:45.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:44 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:28:45.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 275759 4.89 MB/sec execute 252 sec latency 2850.686 ms 2023-07-21T22:28:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:45 smithi104 ceph-mon[129021]: pgmap v223: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-07-21T22:28:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:45 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:45 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:45 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:28:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:45 smithi066 ceph-mon[126379]: pgmap v223: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-07-21T22:28:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:45 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:45 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:46.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:45 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:28:46.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:45 smithi149 ceph-mon[129268]: pgmap v223: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-07-21T22:28:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:45 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:28:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:45 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:28:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:45 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:28:46.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 277180 4.91 MB/sec execute 253 sec latency 30.804 ms 2023-07-21T22:28:47.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 278388 4.91 MB/sec execute 254 sec latency 148.997 ms 2023-07-21T22:28:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:47 smithi104 ceph-mon[129021]: pgmap v224: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 891 KiB/s wr, 85 op/s 2023-07-21T22:28:48.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:47 smithi066 ceph-mon[126379]: pgmap v224: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 891 KiB/s wr, 85 op/s 2023-07-21T22:28:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:47 smithi149 ceph-mon[129268]: pgmap v224: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 891 KiB/s wr, 85 op/s 2023-07-21T22:28:48.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 280566 4.95 MB/sec execute 255 sec latency 14.912 ms 2023-07-21T22:28:49.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 281738 4.95 MB/sec execute 256 sec latency 15.693 ms 2023-07-21T22:28:50.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:49 smithi104 ceph-mon[129021]: pgmap v225: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 891 KiB/s wr, 85 op/s 2023-07-21T22:28:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:49 smithi066 ceph-mon[126379]: pgmap v225: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 891 KiB/s wr, 85 op/s 2023-07-21T22:28:50.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:49 smithi149 ceph-mon[129268]: pgmap v225: 129 pgs: 129 active+clean; 209 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 891 KiB/s wr, 85 op/s 2023-07-21T22:28:50.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 281934 4.93 MB/sec execute 257 sec latency 752.955 ms 2023-07-21T22:28:50.699 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:28:50.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 -- 172.21.15.66:0/2474970588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 msgr2=0x7f08cc10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 --2- 172.21.15.66:0/2474970588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc10c910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b0a0 tx=0x7f08c400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 -- 172.21.15.66:0/2474970588 shutdown_connections 2023-07-21T22:28:50.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 --2- 172.21.15.66:0/2474970588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc10c910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 --2- 172.21.15.66:0/2474970588 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 --2- 172.21.15.66:0/2474970588 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08cc0ffad0 0x7f08cc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 -- 172.21.15.66:0/2474970588 >> 172.21.15.66:0/2474970588 conn(0x7f08cc0fb5f0 msgr2=0x7f08cc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:50.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 -- 172.21.15.66:0/2474970588 shutdown_connections 2023-07-21T22:28:50.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.877+0000 7f08d45ff700 1 -- 172.21.15.66:0/2474970588 wait complete. 2023-07-21T22:28:50.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 Processor -- start 2023-07-21T22:28:50.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 -- start start 2023-07-21T22:28:50.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08cc0ffad0 0x7f08cc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1a40d0 con 0x7f08cc105ae0 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1a4240 con 0x7f08cc100480 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.879+0000 7f08d45ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1a4400 con 0x7f08cc0ffad0 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d2b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47096/0 (socket says 172.21.15.66:47096) 2023-07-21T22:28:50.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d2b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:55460/0 (socket says 172.21.15.66:55460) 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 -- 172.21.15.66:0/504424603 learned_addr learned my addr 172.21.15.66:0/504424603 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d239b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08cc0ffad0 0x7f08cc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 -- 172.21.15.66:0/504424603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08cc0ffad0 msgr2=0x7f08cc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08cc0ffad0 0x7f08cc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 -- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 msgr2=0x7f08cc073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 -- 172.21.15.66:0/504424603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009520 con 0x7f08cc100480 2023-07-21T22:28:50.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:50.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.880+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc072d50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e9a0 tx=0x7f08bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:50.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.881+0000 7f08c37fe700 1 -- 172.21.15.66:0/504424603 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc009790 con 0x7f08cc100480 2023-07-21T22:28:50.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.881+0000 7f08c37fe700 1 -- 172.21.15.66:0/504424603 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08bc010430 con 0x7f08cc100480 2023-07-21T22:28:50.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.881+0000 7f08c37fe700 1 -- 172.21.15.66:0/504424603 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc018680 con 0x7f08cc100480 2023-07-21T22:28:50.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.881+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c4009d00 con 0x7f08cc100480 2023-07-21T22:28:50.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.881+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc1a4a10 con 0x7f08cc100480 2023-07-21T22:28:50.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.884+0000 7f08c37fe700 1 -- 172.21.15.66:0/504424603 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f08bc0105a0 con 0x7f08cc100480 2023-07-21T22:28:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.885+0000 7f08e7ea3b80 1 Processor -- start 2023-07-21T22:28:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.885+0000 7f08e7ea3b80 1 -- start start 2023-07-21T22:28:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.885+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac39580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 0x561f9ac34350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac34930 0x561f9ac34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ab39cb0 con 0x561f9ac391b0 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ab8ca40 con 0x561f9ac39ac0 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ab85ed0 con 0x561f9ac34930 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 0x561f9ac34350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d2b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac39580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 0x561f9ac34350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47098/0 (socket says 172.21.15.66:47098) 2023-07-21T22:28:50.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 -- 172.21.15.66:0/4217325500 learned_addr learned my addr 172.21.15.66:0/4217325500 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d1b9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac34930 0x561f9ac34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 -- 172.21.15.66:0/4217325500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac34930 msgr2=0x561f9ac34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac34930 0x561f9ac34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 -- 172.21.15.66:0/4217325500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 msgr2=0x561f9ac39580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac39580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d239b700 1 -- 172.21.15.66:0/4217325500 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f9a9b3de0 con 0x561f9ac39ac0 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac39580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:28:50.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.886+0000 7f08c37fe700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f08b80a12c0 0x7f08b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08d239b700 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 0x561f9ac34350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009e90 tx=0x7f08c8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e04a177ebf971e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08c37fe700 1 -- 172.21.15.66:0/504424603 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f08bc014070 con 0x7f08cc100480 2023-07-21T22:28:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f08b80a12c0 0x7f08b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08c0bfe700 1 -- 172.21.15.66:0/4217325500 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800e670 con 0x561f9ac39ac0 2023-07-21T22:28:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08c0bfe700 1 -- 172.21.15.66:0/4217325500 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c800ec90 con 0x561f9ac39ac0 2023-07-21T22:28:50.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08c0bfe700 1 -- 172.21.15.66:0/4217325500 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8019d00 con 0x561f9ac39ac0 2023-07-21T22:28:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f08b80a12c0 0x7f08b80a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b780 tx=0x7f08c4006230 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.887+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/4217325500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 msgr2=0x561f9ac34350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 0x561f9ac34350 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009e90 tx=0x7f08c8011720 comp rx=0 tx=0).stop 2023-07-21T22:28:50.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/4217325500 shutdown_connections 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac34930 0x561f9ac34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac39580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/4217325500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac39ac0 0x561f9ac34350 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/4217325500 >> 172.21.15.66:0/4217325500 conn(0x561f9ac29ef0 msgr2=0x561f9ac2f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/4217325500 shutdown_connections 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/4217325500 wait complete. 2023-07-21T22:28:50.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 -- start start 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.888+0000 7f08e7ea3b80 1 Processor -- start 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac34930 0x561f9ac40c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 0x561f9ac3de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ab25ac0 con 0x561f9ac34930 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ab257c0 con 0x561f9ac391b0 2023-07-21T22:28:50.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9aa09920 con 0x561f9ac39ac0 2023-07-21T22:28:50.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08e7ea3b80 10 client.0 _ll_register_callbacks cb 0x561f9ab23760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:50.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08d1b9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 0x561f9ac3de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08d239b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac34930 0x561f9ac40c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08d1b9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 0x561f9ac3de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47112/0 (socket says 172.21.15.66:47112) 2023-07-21T22:28:50.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08d1b9a700 1 -- 172.21.15.66:0/2065908562 learned_addr learned my addr 172.21.15.66:0/2065908562 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:50.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.889+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac3e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08d1b9a700 1 -- 172.21.15.66:0/2065908562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 msgr2=0x561f9ac3e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac3e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08d1b9a700 1 -- 172.21.15.66:0/2065908562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac34930 msgr2=0x561f9ac40c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac34930 0x561f9ac40c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08d1b9a700 1 -- 172.21.15.66:0/2065908562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc10d210 con 0x561f9ac391b0 2023-07-21T22:28:50.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 0x561f9ac3de80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0a1ed0 tx=0x7f08bc0cab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd397c719d4f1e3e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:50.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:28:50.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2065908562 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc099af0 con 0x561f9ac391b0 2023-07-21T22:28:50.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2065908562 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f08bc099fb0 con 0x561f9ac391b0 2023-07-21T22:28:50.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f9aadf4f0 con 0x561f9ac391b0 2023-07-21T22:28:50.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2065908562 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08bc099910 con 0x561f9ac391b0 2023-07-21T22:28:50.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08e7ea3b80 10 client.26705 fetch_fsmap learned FSMap version 36 2023-07-21T22:28:50.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7f08c8009af0 con 0x561f9ac391b0 2023-07-21T22:28:50.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.890+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2065908562 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561f9aadf4f0 con 0x561f9ac391b0 2023-07-21T22:28:50.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2065908562 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f08bc0994a0 con 0x561f9ac391b0 2023-07-21T22:28:50.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08e7ea3b80 10 client.26705 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:28:50.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08e7ea3b80 10 client.26705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:28:50.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08e7ea3b80 20 client.26705 populate_metadata read hostname 'smithi066' 2023-07-21T22:28:50.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9ac4e550 0x561f9ac50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08e7ea3b80 4 client.26705 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:28:50.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f9ab7eeb0 con 0x561f9ac4e550 2023-07-21T22:28:50.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.891+0000 7f08d239b700 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9ac4e550 0x561f9ac50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08b5ffb700 10 client.26705 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:50.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08d239b700 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9ac4e550 0x561f9ac50960 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1ef1be34a97fdd server_cookie=48179aff6b24dce7 in_seq=0 out_seq=0 2023-07-21T22:28:50.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2065908562 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561f9ab7eeb0 con 0x561f9ac4e550 2023-07-21T22:28:50.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08b5ffb700 10 client.26705 handle_command_reply: tid=0 2023-07-21T22:28:50.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08e7ea3b80 1 client.26705 shutdown 2023-07-21T22:28:50.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 msgr2=0x561f9ac3de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.892+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 0x561f9ac3de80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0a1ed0 tx=0x7f08bc0cab90 comp rx=0 tx=0).stop 2023-07-21T22:28:50.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 shutdown_connections 2023-07-21T22:28:50.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9ac4e550 0x561f9ac50960 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac34930 0x561f9ac40c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2065908562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac391b0 0x561f9ac3de80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 >> 172.21.15.66:0/2065908562 conn(0x561f9ac29ef0 msgr2=0x561f9ac2f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:50.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08b5ffb700 0 client.26705 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:50.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 shutdown_connections 2023-07-21T22:28:50.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2065908562 wait complete. 2023-07-21T22:28:50.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.893+0000 7f08e7ea3b80 20 client.26705 trim_cache size 0 max 16384 2023-07-21T22:28:50.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.910+0000 7f08e7ea3b80 1 Processor -- start 2023-07-21T22:28:50.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.910+0000 7f08e7ea3b80 1 -- start start 2023-07-21T22:28:50.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.910+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.910+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac407a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac40ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ac3ee30 con 0x561f9ac391b0 2023-07-21T22:28:50.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ac30320 con 0x561f9ac34930 2023-07-21T22:28:50.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ac4bc30 con 0x561f9ac39ac0 2023-07-21T22:28:50.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47124/0 (socket says 172.21.15.66:47124) 2023-07-21T22:28:50.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 -- 172.21.15.66:0/652468317 learned_addr learned my addr 172.21.15.66:0/652468317 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:50.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac40ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 -- 172.21.15.66:0/652468317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 msgr2=0x561f9ac40ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac40ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 -- 172.21.15.66:0/652468317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 msgr2=0x561f9ac407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 -- 172.21.15.66:0/652468317 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f9a99cc60 con 0x561f9ac34930 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.911+0000 7f08d239b700 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08c801c040 tx=0x7f08c8005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55f8caafa9a405a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08c13ff700 1 -- 172.21.15.66:0/652468317 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80198b0 con 0x561f9ac34930 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08c13ff700 1 -- 172.21.15.66:0/652468317 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c800e470 con 0x561f9ac34930 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08c13ff700 1 -- 172.21.15.66:0/652468317 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8005280 con 0x561f9ac34930 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/652468317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 msgr2=0x561f9ac45570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f08c801c040 tx=0x7f08c8005d20 comp rx=0 tx=0).stop 2023-07-21T22:28:50.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/652468317 shutdown_connections 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45570 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac39ac0 0x561f9ac40ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/652468317 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac391b0 0x561f9ac407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/652468317 >> 172.21.15.66:0/652468317 conn(0x561f9ac29ef0 msgr2=0x561f9ac329c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/652468317 shutdown_connections 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.912+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/652468317 wait complete. 2023-07-21T22:28:50.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.913+0000 7f08e7ea3b80 1 -- start start 2023-07-21T22:28:50.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.913+0000 7f08e7ea3b80 1 Processor -- start 2023-07-21T22:28:50.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.913+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac391b0 0x561f9ac3c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08e7ea3b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac39ac0 0x561f9ac3cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ac3d0b0 con 0x561f9ac39ac0 2023-07-21T22:28:50.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ac3d220 con 0x561f9ac34930 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08e7ea3b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f9ac3d3e0 con 0x561f9ac391b0 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08e7ea3b80 10 client.0 _ll_register_callbacks cb 0x561f9ab23760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47140/0 (socket says 172.21.15.66:47140) 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 -- 172.21.15.66:0/2910278744 learned_addr learned my addr 172.21.15.66:0/2910278744 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac391b0 0x561f9ac3c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d2b9c700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac39ac0 0x561f9ac3cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 -- 172.21.15.66:0/2910278744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac391b0 msgr2=0x561f9ac3c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac391b0 0x561f9ac3c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 -- 172.21.15.66:0/2910278744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac39ac0 msgr2=0x561f9ac3cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac39ac0 0x561f9ac3cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 -- 172.21.15.66:0/2910278744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f9a9b3de0 con 0x561f9ac34930 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08d239b700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08c8011be0 tx=0x7f08c8009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11f8f60217da76aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.914+0000 7f08b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:28:50.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2910278744 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800e9f0 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08c800ec90 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2910278744 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f08c80198b0 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2910278744 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c8019a90 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08e7ea3b80 10 client.26711 fetch_fsmap learned FSMap version 36 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561f9a9da250 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2910278744 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c800ec90 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2910278744 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f08c8031070 con 0x561f9ac34930 2023-07-21T22:28:50.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08e7ea3b80 10 client.26711 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08e7ea3b80 10 client.26711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.915+0000 7f08e7ea3b80 20 client.26711 populate_metadata read hostname 'smithi066' 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9aca3530 0x561f9aca5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08e7ea3b80 4 client.26711 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f9ab41050 con 0x561f9aca3530 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9aca3530 0x561f9aca5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:28:50.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08d1b9a700 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9aca3530 0x561f9aca5910 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cbd5f29967113cf server_cookie=ccb30d9a4bbb1ef9 in_seq=0 out_seq=0 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08b5ffb700 10 client.26711 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08b5ffb700 1 -- 172.21.15.66:0/2910278744 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561f9ab41050 con 0x561f9aca3530 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08b5ffb700 10 client.26711 handle_command_reply: tid=0 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.916+0000 7f08e7ea3b80 1 client.26711 shutdown 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 msgr2=0x561f9ac45580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45580 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08c8011be0 tx=0x7f08c8009480 comp rx=0 tx=0).stop 2023-07-21T22:28:50.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 shutdown_connections 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561f9ac34930 0x561f9ac45580 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x561f9aca3530 0x561f9aca5910 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x561f9ac39ac0 0x561f9ac3cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 --2- 172.21.15.66:0/2910278744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561f9ac391b0 0x561f9ac3c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08b5ffb700 0 client.26711 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 >> 172.21.15.66:0/2910278744 conn(0x561f9ac29ef0 msgr2=0x561f9ac33940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 shutdown_connections 2023-07-21T22:28:50.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 1 -- 172.21.15.66:0/2910278744 wait complete. 2023-07-21T22:28:50.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08e7ea3b80 20 client.26711 trim_cache size 0 max 16384 2023-07-21T22:28:50.979 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:28:50.979 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:28:50.979 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:28:50.979 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:28:50.980 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:28:50.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:28:50.980 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:28:50.980 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:28:50.980 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:28:50.980 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:28:50.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f08b80a12c0 msgr2=0x7f08b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08d45ff700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f08b80a12c0 0x7f08b80a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b780 tx=0x7f08c4006230 comp rx=0 tx=0).stop 2023-07-21T22:28:50.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 msgr2=0x7f08cc072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:28:50.981 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08d45ff700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc072d50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e9a0 tx=0x7f08bc00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 shutdown_connections 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.917+0000 7f08d45ff700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f08b80a12c0 0x7f08b80a3770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.918+0000 7f08d45ff700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f08cc105ae0 0x7f08cc073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.918+0000 7f08d45ff700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08cc0ffad0 0x7f08cc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.918+0000 7f08d45ff700 1 --2- 172.21.15.66:0/504424603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08cc100480 0x7f08cc072d50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.918+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 >> 172.21.15.66:0/504424603 conn(0x7f08cc0fb5f0 msgr2=0x7f08cc0fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:28:50.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.918+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 shutdown_connections 2023-07-21T22:28:50.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:28:50.918+0000 7f08d45ff700 1 -- 172.21.15.66:0/504424603 wait complete. 2023-07-21T22:28:50.983 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:28:51.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 281934 4.91 MB/sec execute 258 sec latency 1753.113 ms 2023-07-21T22:28:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:51 smithi104 ceph-mon[129021]: pgmap v226: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2023-07-21T22:28:52.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:51 smithi066 ceph-mon[126379]: pgmap v226: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2023-07-21T22:28:52.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:51 smithi149 ceph-mon[129268]: pgmap v226: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2023-07-21T22:28:52.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 281937 4.89 MB/sec execute 259 sec latency 2748.286 ms 2023-07-21T22:28:53.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 284098 4.92 MB/sec execute 260 sec latency 17.323 ms 2023-07-21T22:28:54.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:53 smithi104 ceph-mon[129021]: pgmap v227: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1023 KiB/s wr, 112 op/s 2023-07-21T22:28:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:53 smithi066 ceph-mon[126379]: pgmap v227: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1023 KiB/s wr, 112 op/s 2023-07-21T22:28:54.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:53 smithi149 ceph-mon[129268]: pgmap v227: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1023 KiB/s wr, 112 op/s 2023-07-21T22:28:54.548 INFO:tasks.workunit.client.0.smithi066.stdout: 1 285314 4.93 MB/sec execute 261 sec latency 20.528 ms 2023-07-21T22:28:55.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 285476 4.91 MB/sec execute 262 sec latency 894.113 ms 2023-07-21T22:28:56.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:55 smithi104 ceph-mon[129021]: pgmap v228: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2023-07-21T22:28:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:55 smithi066 ceph-mon[126379]: pgmap v228: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2023-07-21T22:28:56.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:55 smithi149 ceph-mon[129268]: pgmap v228: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2023-07-21T22:28:56.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 285476 4.89 MB/sec execute 263 sec latency 1894.251 ms 2023-07-21T22:28:57.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 285479 4.87 MB/sec execute 264 sec latency 2883.978 ms 2023-07-21T22:28:58.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:57 smithi104 ceph-mon[129021]: pgmap v229: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-07-21T22:28:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:57 smithi066 ceph-mon[126379]: pgmap v229: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-07-21T22:28:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:57 smithi149 ceph-mon[129268]: pgmap v229: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-07-21T22:28:58.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 287617 4.90 MB/sec execute 265 sec latency 22.617 ms 2023-07-21T22:28:59.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 288785 4.91 MB/sec execute 266 sec latency 20.194 ms 2023-07-21T22:29:00.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:28:59 smithi104 ceph-mon[129021]: pgmap v230: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-07-21T22:29:00.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:28:59 smithi066 ceph-mon[126379]: pgmap v230: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-07-21T22:29:00.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:28:59 smithi149 ceph-mon[129268]: pgmap v230: 129 pgs: 129 active+clean; 219 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 112 op/s 2023-07-21T22:29:00.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 289934 4.90 MB/sec execute 267 sec latency 447.310 ms 2023-07-21T22:29:01.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 290971 4.92 MB/sec execute 268 sec latency 581.134 ms 2023-07-21T22:29:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:01 smithi104 ceph-mon[129021]: pgmap v231: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2023-07-21T22:29:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:01 smithi066 ceph-mon[126379]: pgmap v231: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2023-07-21T22:29:02.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:01 smithi149 ceph-mon[129268]: pgmap v231: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2023-07-21T22:29:02.549 INFO:tasks.workunit.client.0.smithi066.stdout: 1 290985 4.90 MB/sec execute 269 sec latency 1572.824 ms 2023-07-21T22:29:03.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 291139 4.88 MB/sec execute 270 sec latency 64.039 ms 2023-07-21T22:29:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:03 smithi104 ceph-mon[129021]: pgmap v232: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 720 KiB/s wr, 84 op/s 2023-07-21T22:29:04.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:03 smithi066 ceph-mon[126379]: pgmap v232: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 720 KiB/s wr, 84 op/s 2023-07-21T22:29:04.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:03 smithi149 ceph-mon[129268]: pgmap v232: 129 pgs: 129 active+clean; 223 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 720 KiB/s wr, 84 op/s 2023-07-21T22:29:04.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 292300 4.89 MB/sec execute 271 sec latency 27.969 ms 2023-07-21T22:29:05.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 292567 4.87 MB/sec execute 272 sec latency 777.050 ms 2023-07-21T22:29:06.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:05 smithi104 ceph-mon[129021]: pgmap v233: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 167 op/s 2023-07-21T22:29:06.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:05 smithi066 ceph-mon[126379]: pgmap v233: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 167 op/s 2023-07-21T22:29:06.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:05 smithi149 ceph-mon[129268]: pgmap v233: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 167 op/s 2023-07-21T22:29:06.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 292567 4.85 MB/sec execute 273 sec latency 1777.194 ms 2023-07-21T22:29:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:06 smithi104 ceph-mon[129021]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T22:29:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:06 smithi066 ceph-mon[126379]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T22:29:07.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:06 smithi149 ceph-mon[129268]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T22:29:07.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 292570 4.83 MB/sec execute 274 sec latency 2761.064 ms 2023-07-21T22:29:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:07 smithi104 ceph-mon[129021]: pgmap v235: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2023-07-21T22:29:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:07 smithi066 ceph-mon[126379]: pgmap v235: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2023-07-21T22:29:08.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:07 smithi149 ceph-mon[129268]: pgmap v235: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2023-07-21T22:29:08.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 294707 4.86 MB/sec execute 275 sec latency 27.475 ms 2023-07-21T22:29:09.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 295850 4.87 MB/sec execute 276 sec latency 56.622 ms 2023-07-21T22:29:10.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:09 smithi104 ceph-mon[129021]: pgmap v236: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2023-07-21T22:29:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:09 smithi066 ceph-mon[126379]: pgmap v236: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2023-07-21T22:29:10.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:09 smithi149 ceph-mon[129268]: pgmap v236: 129 pgs: 129 active+clean; 226 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2023-07-21T22:29:10.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 296117 4.85 MB/sec execute 277 sec latency 795.108 ms 2023-07-21T22:29:11.550 INFO:tasks.workunit.client.0.smithi066.stdout: 1 296117 4.83 MB/sec execute 278 sec latency 1795.181 ms 2023-07-21T22:29:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:11 smithi104 ceph-mon[129021]: pgmap v237: 129 pgs: 129 active+clean; 230 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:29:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:11 smithi066 ceph-mon[126379]: pgmap v237: 129 pgs: 129 active+clean; 230 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:29:12.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:11 smithi149 ceph-mon[129268]: pgmap v237: 129 pgs: 129 active+clean; 230 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:29:12.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 296120 4.81 MB/sec execute 279 sec latency 2779.898 ms 2023-07-21T22:29:13.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 298277 4.85 MB/sec execute 280 sec latency 20.498 ms 2023-07-21T22:29:14.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:13 smithi149 ceph-mon[129268]: pgmap v238: 129 pgs: 129 active+clean; 230 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:29:14.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:13 smithi149 ceph-mon[129268]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T22:29:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:13 smithi104 ceph-mon[129021]: pgmap v238: 129 pgs: 129 active+clean; 230 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:29:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:13 smithi104 ceph-mon[129021]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T22:29:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:13 smithi066 ceph-mon[126379]: pgmap v238: 129 pgs: 129 active+clean; 230 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:29:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:13 smithi066 ceph-mon[126379]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T22:29:14.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 299398 4.85 MB/sec execute 281 sec latency 42.224 ms 2023-07-21T22:29:15.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 300570 4.84 MB/sec execute 282 sec latency 441.423 ms 2023-07-21T22:29:16.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:15 smithi149 ceph-mon[129268]: pgmap v240: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-21T22:29:16.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:15 smithi104 ceph-mon[129021]: pgmap v240: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-21T22:29:16.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:15 smithi066 ceph-mon[126379]: pgmap v240: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 131 op/s 2023-07-21T22:29:16.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 302101 4.86 MB/sec execute 283 sec latency 29.071 ms 2023-07-21T22:29:17.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 303217 4.86 MB/sec execute 284 sec latency 47.366 ms 2023-07-21T22:29:18.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:17 smithi149 ceph-mon[129268]: pgmap v241: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2023-07-21T22:29:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:17 smithi104 ceph-mon[129021]: pgmap v241: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2023-07-21T22:29:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:17 smithi066 ceph-mon[126379]: pgmap v241: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2023-07-21T22:29:18.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 305268 4.90 MB/sec execute 285 sec latency 19.314 ms 2023-07-21T22:29:19.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 306286 4.89 MB/sec execute 286 sec latency 26.041 ms 2023-07-21T22:29:20.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:19 smithi149 ceph-mon[129268]: pgmap v242: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2023-07-21T22:29:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:19 smithi104 ceph-mon[129021]: pgmap v242: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2023-07-21T22:29:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:19 smithi066 ceph-mon[126379]: pgmap v242: 129 pgs: 129 active+clean; 233 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 1.6 MiB/s wr, 116 op/s 2023-07-21T22:29:20.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 306757 4.88 MB/sec execute 287 sec latency 628.970 ms 2023-07-21T22:29:20.985 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:29:21.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 -- 172.21.15.66:0/1426652177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210100480 msgr2=0x7f521010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 --2- 172.21.15.66:0/1426652177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210100480 0x7f521010c910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f520800b0a0 tx=0x7f520800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 -- 172.21.15.66:0/1426652177 shutdown_connections 2023-07-21T22:29:21.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 --2- 172.21.15.66:0/1426652177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210100480 0x7f521010c910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 --2- 172.21.15.66:0/1426652177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5210105ae0 0x7f5210105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 --2- 172.21.15.66:0/1426652177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f52100ffad0 0x7f52100fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 -- 172.21.15.66:0/1426652177 >> 172.21.15.66:0/1426652177 conn(0x7f52100fb5f0 msgr2=0x7f52100fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:21.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 -- 172.21.15.66:0/1426652177 shutdown_connections 2023-07-21T22:29:21.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.161+0000 7f52159ff700 1 -- 172.21.15.66:0/1426652177 wait complete. 2023-07-21T22:29:21.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.162+0000 7f52159ff700 1 Processor -- start 2023-07-21T22:29:21.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.162+0000 7f52159ff700 1 -- start start 2023-07-21T22:29:21.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f52159ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 0x7f5210197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f52159ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5210100480 0x7f5210198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f52159ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210105ae0 0x7f521019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f52159ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521019cf00 con 0x7f52100ffad0 2023-07-21T22:29:21.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f52159ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521019d070 con 0x7f5210105ae0 2023-07-21T22:29:21.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f52159ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521019d230 con 0x7f5210100480 2023-07-21T22:29:21.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 0x7f5210197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 0x7f5210197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44304/0 (socket says 172.21.15.66:44304) 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 -- 172.21.15.66:0/1134409837 learned_addr learned my addr 172.21.15.66:0/1134409837 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520f7fe700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210105ae0 0x7f521019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f5207fff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5210100480 0x7f5210198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 -- 172.21.15.66:0/1134409837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5210100480 msgr2=0x7f5210198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5210100480 0x7f5210198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 -- 172.21.15.66:0/1134409837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210105ae0 msgr2=0x7f521019c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210105ae0 0x7f521019c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520effd700 1 -- 172.21.15.66:0/1134409837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208009d00 con 0x7f52100ffad0 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.163+0000 7f520f7fe700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210105ae0 0x7f521019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.164+0000 7f520effd700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 0x7f5210197f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f520000d950 tx=0x7f520000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.164+0000 7f52149fd700 1 -- 172.21.15.66:0/1134409837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000f890 con 0x7f52100ffad0 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.164+0000 7f52149fd700 1 -- 172.21.15.66:0/1134409837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5200010460 con 0x7f52100ffad0 2023-07-21T22:29:21.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.164+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521019d510 con 0x7f52100ffad0 2023-07-21T22:29:21.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.164+0000 7f52149fd700 1 -- 172.21.15.66:0/1134409837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000e610 con 0x7f52100ffad0 2023-07-21T22:29:21.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.164+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52101a4d50 con 0x7f52100ffad0 2023-07-21T22:29:21.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.167+0000 7f52149fd700 1 -- 172.21.15.66:0/1134409837 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f52000108d0 con 0x7f52100ffad0 2023-07-21T22:29:21.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 Processor -- start 2023-07-21T22:29:21.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 -- start start 2023-07-21T22:29:21.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa32350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 0x556afaa32da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afa937cb0 con 0x556afaa37ac0 2023-07-21T22:29:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afa98aa40 con 0x556afaa32930 2023-07-21T22:29:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.169+0000 7f522924fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afa983ed0 con 0x556afaa371b0 2023-07-21T22:29:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 0x556afaa32da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 0x556afaa32da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56698/0 (socket says 172.21.15.66:56698) 2023-07-21T22:29:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f520effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa32350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 -- 172.21.15.66:0/3977911118 learned_addr learned my addr 172.21.15.66:0/3977911118 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f520f7fe700 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f52149fd700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51fc0a12c0 0x7f51fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 -- 172.21.15.66:0/3977911118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 msgr2=0x556afaa37580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa37580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 -- 172.21.15.66:0/3977911118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 msgr2=0x556afaa32350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa32350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f5207fff700 1 -- 172.21.15.66:0/3977911118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556afa7b1de0 con 0x556afaa32930 2023-07-21T22:29:21.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f52149fd700 1 -- 172.21.15.66:0/1134409837 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f52000cce60 con 0x7f52100ffad0 2023-07-21T22:29:21.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.170+0000 7f520f7fe700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51fc0a12c0 0x7f51fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f5207fff700 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 0x556afaa32da0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009d20 tx=0x7f51f8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7002cfce51287855 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:21.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f5205ffb700 1 -- 172.21.15.66:0/3977911118 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8019630 con 0x556afaa32930 2023-07-21T22:29:21.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f5205ffb700 1 -- 172.21.15.66:0/3977911118 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f8019c50 con 0x556afaa32930 2023-07-21T22:29:21.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f5205ffb700 1 -- 172.21.15.66:0/3977911118 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800ebe0 con 0x556afaa32930 2023-07-21T22:29:21.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f520f7fe700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51fc0a12c0 0x7f51fc0a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f520800b780 tx=0x7f52080099e0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:21.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f522924fb80 1 -- 172.21.15.66:0/3977911118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 msgr2=0x556afaa32da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.171+0000 7f522924fb80 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 0x556afaa32da0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009d20 tx=0x7f51f8011720 comp rx=0 tx=0).stop 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 -- 172.21.15.66:0/3977911118 shutdown_connections 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa32930 0x556afaa32da0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa37580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 --2- 172.21.15.66:0/3977911118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa32350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 -- 172.21.15.66:0/3977911118 >> 172.21.15.66:0/3977911118 conn(0x556afaa27ef0 msgr2=0x556afaa2d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 -- 172.21.15.66:0/3977911118 shutdown_connections 2023-07-21T22:29:21.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 -- 172.21.15.66:0/3977911118 wait complete. 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.172+0000 7f522924fb80 1 -- start start 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 Processor -- start 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa3ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa3be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa3c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afa923ac0 con 0x556afaa37ac0 2023-07-21T22:29:21.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afa9237c0 con 0x556afaa371b0 2023-07-21T22:29:21.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afa807920 con 0x556afaa32930 2023-07-21T22:29:21.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f522924fb80 10 client.0 _ll_register_callbacks cb 0x556afa921760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:21.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f5207fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa3be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f5207fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa3be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56712/0 (socket says 172.21.15.66:56712) 2023-07-21T22:29:21.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f5207fff700 1 -- 172.21.15.66:0/1911066496 learned_addr learned my addr 172.21.15.66:0/1911066496 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:21.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f520effd700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa3ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.173+0000 7f520f7fe700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa3c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f5207fff700 1 -- 172.21.15.66:0/1911066496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 msgr2=0x556afaa3ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f5207fff700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa3ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f5207fff700 1 -- 172.21.15.66:0/1911066496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 msgr2=0x556afaa3c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f5207fff700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa3c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f5207fff700 1 -- 172.21.15.66:0/1911066496 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f8009980 con 0x556afaa371b0 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f520f7fe700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:21.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f5207fff700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa3be00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51f800e780 tx=0x7f51f800eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa96f24bada2c126 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:21.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f51eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:29:21.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f51eeffd700 1 -- 172.21.15.66:0/1911066496 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8004220 con 0x556afaa371b0 2023-07-21T22:29:21.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556afa8dd4f0 con 0x556afaa371b0 2023-07-21T22:29:21.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.174+0000 7f51eeffd700 1 -- 172.21.15.66:0/1911066496 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f51f8005ce0 con 0x556afaa371b0 2023-07-21T22:29:21.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f51eeffd700 1 -- 172.21.15.66:0/1911066496 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f8004380 con 0x556afaa371b0 2023-07-21T22:29:21.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 10 client.26723 fetch_fsmap learned FSMap version 36 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556afa79ac60 con 0x556afaa371b0 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f51eeffd700 1 -- 172.21.15.66:0/1911066496 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556afa8dd4f0 con 0x556afaa371b0 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f51eeffd700 1 -- 172.21.15.66:0/1911066496 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f51f8029030 con 0x556afaa371b0 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 10 client.26723 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 10 client.26723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 20 client.26723 populate_metadata read hostname 'smithi066' 2023-07-21T22:29:21.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaa4c320 0x556afaa4e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 4 client.26723 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:21.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556afa97ceb0 con 0x556afaa4c320 2023-07-21T22:29:21.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.175+0000 7f520effd700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaa4c320 0x556afaa4e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.176+0000 7f51eeffd700 10 client.26723 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:21.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.176+0000 7f520effd700 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaa4c320 0x556afaa4e730 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18ead6ba5fc3dc9f server_cookie=105df839512f7f90 in_seq=0 out_seq=0 2023-07-21T22:29:21.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f51eeffd700 1 -- 172.21.15.66:0/1911066496 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556afa97ceb0 con 0x556afaa4c320 2023-07-21T22:29:21.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f51eeffd700 10 client.26723 handle_command_reply: tid=0 2023-07-21T22:29:21.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 client.26723 shutdown 2023-07-21T22:29:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 msgr2=0x556afaa3be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa3be00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51f800e780 tx=0x7f51f800eee0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 shutdown_connections 2023-07-21T22:29:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaa4c320 0x556afaa4e730 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa3c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa3ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 --2- 172.21.15.66:0/1911066496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa3be00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f51eeffd700 0 client.26723 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:21.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 >> 172.21.15.66:0/1911066496 conn(0x556afaa27ef0 msgr2=0x556afaa2d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:21.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 shutdown_connections 2023-07-21T22:29:21.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 1 -- 172.21.15.66:0/1911066496 wait complete. 2023-07-21T22:29:21.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.177+0000 7f522924fb80 20 client.26723 trim_cache size 0 max 16384 2023-07-21T22:29:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 Processor -- start 2023-07-21T22:29:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 -- start start 2023-07-21T22:29:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 0x556afaa48850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa37ac0 0x556afaa31980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afaa2a050 con 0x556afaa32930 2023-07-21T22:29:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afaa2edf0 con 0x556afaa37ac0 2023-07-21T22:29:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f522924fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afaa33650 con 0x556afaa371b0 2023-07-21T22:29:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f520effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 0x556afaa48850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f520effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 0x556afaa48850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:44318/0 (socket says 172.21.15.66:44318) 2023-07-21T22:29:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f520effd700 1 -- 172.21.15.66:0/385252580 learned_addr learned my addr 172.21.15.66:0/385252580 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:21.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f520f7fe700 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa37ac0 0x556afaa31980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.195+0000 7f5207fff700 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f520effd700 1 -- 172.21.15.66:0/385252580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 msgr2=0x556afaa3e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f520effd700 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa3e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f520effd700 1 -- 172.21.15.66:0/385252580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa37ac0 msgr2=0x556afaa31980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f520effd700 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa37ac0 0x556afaa31980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f520effd700 1 -- 172.21.15.66:0/385252580 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556afa7d8250 con 0x556afaa32930 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f520effd700 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 0x556afaa48850 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5200099ac0 tx=0x7f52000cf2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b429b4a46b812b0b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f52067fc700 1 -- 172.21.15.66:0/385252580 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200099490 con 0x556afaa32930 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f52067fc700 1 -- 172.21.15.66:0/385252580 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5200099b30 con 0x556afaa32930 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.196+0000 7f52067fc700 1 -- 172.21.15.66:0/385252580 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200099020 con 0x556afaa32930 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 -- 172.21.15.66:0/385252580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 msgr2=0x556afaa48850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 0x556afaa48850 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5200099ac0 tx=0x7f52000cf2e0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 -- 172.21.15.66:0/385252580 shutdown_connections 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa37ac0 0x556afaa31980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa32930 0x556afaa48850 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 --2- 172.21.15.66:0/385252580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa371b0 0x556afaa3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 -- 172.21.15.66:0/385252580 >> 172.21.15.66:0/385252580 conn(0x556afaa27ef0 msgr2=0x556afaa53050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 -- 172.21.15.66:0/385252580 shutdown_connections 2023-07-21T22:29:21.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 -- 172.21.15.66:0/385252580 wait complete. 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 -- start start 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.197+0000 7f522924fb80 1 Processor -- start 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa43570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afaa425b0 con 0x556afaa37ac0 2023-07-21T22:29:21.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afaa42720 con 0x556afaa371b0 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556afaaa01b0 con 0x556afaa32930 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f522924fb80 10 client.0 _ll_register_callbacks cb 0x7f5200095e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f5207fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f520f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f5207fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:56718/0 (socket says 172.21.15.66:56718) 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f5207fff700 1 -- 172.21.15.66:0/700417194 learned_addr learned my addr 172.21.15.66:0/700417194 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:21.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.198+0000 7f520effd700 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa43570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f5207fff700 1 -- 172.21.15.66:0/700417194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 msgr2=0x556afaa43570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f5207fff700 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa43570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f5207fff700 1 -- 172.21.15.66:0/700417194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 msgr2=0x556afaa41ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f5207fff700 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa41ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f5207fff700 1 -- 172.21.15.66:0/700417194 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52000b9040 con 0x556afaa371b0 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f5207fff700 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa419b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005950 tx=0x7f51f80055d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361b49d51aac115e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:21.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f51eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:29:21.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f51eeffd700 1 -- 172.21.15.66:0/700417194 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8005690 con 0x556afaa371b0 2023-07-21T22:29:21.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51f8019c50 con 0x556afaa371b0 2023-07-21T22:29:21.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f51eeffd700 1 -- 172.21.15.66:0/700417194 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f51f802a3b0 con 0x556afaa371b0 2023-07-21T22:29:21.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f51eeffd700 1 -- 172.21.15.66:0/700417194 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51f802a590 con 0x556afaa371b0 2023-07-21T22:29:21.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f522924fb80 10 client.26729 fetch_fsmap learned FSMap version 36 2023-07-21T22:29:21.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.199+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x556afa7b02b0 con 0x556afaa371b0 2023-07-21T22:29:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f51eeffd700 1 -- 172.21.15.66:0/700417194 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51f8019c50 con 0x556afaa371b0 2023-07-21T22:29:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f51eeffd700 1 -- 172.21.15.66:0/700417194 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f51f8011790 con 0x556afaa371b0 2023-07-21T22:29:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f522924fb80 10 client.26729 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T22:29:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f522924fb80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:21.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f522924fb80 20 client.26729 populate_metadata read hostname 'smithi066' 2023-07-21T22:29:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f522924fb80 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaaa12c0 0x556afaaa36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f522924fb80 4 client.26729 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:29:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556afa93f050 con 0x556afaaa12c0 2023-07-21T22:29:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f520effd700 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaaa12c0 0x556afaaa36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f51eeffd700 10 client.26729 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:21.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.200+0000 7f520effd700 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaaa12c0 0x556afaaa36a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d2660d790ac070 server_cookie=404257c60618f9bf in_seq=0 out_seq=0 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f51eeffd700 1 -- 172.21.15.66:0/700417194 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x556afa93f050 con 0x556afaaa12c0 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f51eeffd700 10 client.26729 handle_command_reply: tid=0 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 client.26729 shutdown 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 msgr2=0x556afaa419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa419b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005950 tx=0x7f51f80055d0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 shutdown_connections 2023-07-21T22:29:21.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556afaaa12c0 0x556afaaa36a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556afaa37ac0 0x556afaa41ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556afaa32930 0x556afaa43570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 --2- 172.21.15.66:0/700417194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556afaa371b0 0x556afaa419b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f51eeffd700 0 client.26729 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 >> 172.21.15.66:0/700417194 conn(0x556afaa27ef0 msgr2=0x556afaa30a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 shutdown_connections 2023-07-21T22:29:21.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 1 -- 172.21.15.66:0/700417194 wait complete. 2023-07-21T22:29:21.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.201+0000 7f522924fb80 20 client.26729 trim_cache size 0 max 16384 2023-07-21T22:29:21.257 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:29:21.257 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:29:21.257 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:29:21.257 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:29:21.258 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:29:21.258 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:29:21.258 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:29:21.258 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:29:21.258 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:29:21.258 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:29:21.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51fc0a12c0 msgr2=0x7f51fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51fc0a12c0 0x7f51fc0a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f520800b780 tx=0x7f52080099e0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 msgr2=0x7f5210197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:21.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 0x7f5210197f20 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f520000d950 tx=0x7f520000dc60 comp rx=0 tx=0).stop 2023-07-21T22:29:21.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 shutdown_connections 2023-07-21T22:29:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5210105ae0 0x7f521019c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f51fc0a12c0 0x7f51fc0a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f52100ffad0 0x7f5210197f20 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 --2- 172.21.15.66:0/1134409837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5210100480 0x7f5210198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 >> 172.21.15.66:0/1134409837 conn(0x7f52100fb5f0 msgr2=0x7f521010ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:21.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 shutdown_connections 2023-07-21T22:29:21.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:21.202+0000 7f52159ff700 1 -- 172.21.15.66:0/1134409837 wait complete. 2023-07-21T22:29:21.265 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:29:21.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 306757 4.86 MB/sec execute 288 sec latency 1629.101 ms 2023-07-21T22:29:22.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:21 smithi149 ceph-mon[129268]: pgmap v243: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-21T22:29:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:21 smithi104 ceph-mon[129021]: pgmap v243: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-21T22:29:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:21 smithi066 ceph-mon[126379]: pgmap v243: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-21T22:29:22.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 306760 4.85 MB/sec execute 289 sec latency 2614.157 ms 2023-07-21T22:29:23.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 308937 4.88 MB/sec execute 290 sec latency 17.329 ms 2023-07-21T22:29:24.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:23 smithi149 ceph-mon[129268]: pgmap v244: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-21T22:29:24.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:23 smithi104 ceph-mon[129021]: pgmap v244: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-21T22:29:24.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:23 smithi066 ceph-mon[126379]: pgmap v244: 129 pgs: 129 active+clean; 247 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 157 op/s 2023-07-21T22:29:24.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 310132 4.88 MB/sec execute 291 sec latency 38.914 ms 2023-07-21T22:29:25.551 INFO:tasks.workunit.client.0.smithi066.stdout: 1 310305 4.86 MB/sec execute 292 sec latency 873.357 ms 2023-07-21T22:29:26.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:25 smithi149 ceph-mon[129268]: pgmap v245: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-07-21T22:29:26.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:25 smithi104 ceph-mon[129021]: pgmap v245: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-07-21T22:29:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:25 smithi066 ceph-mon[126379]: pgmap v245: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2023-07-21T22:29:26.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 310305 4.85 MB/sec execute 293 sec latency 1873.499 ms 2023-07-21T22:29:27.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 310308 4.83 MB/sec execute 294 sec latency 2869.635 ms 2023-07-21T22:29:28.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:27 smithi149 ceph-mon[129268]: pgmap v246: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-21T22:29:28.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:27 smithi104 ceph-mon[129021]: pgmap v246: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-21T22:29:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:27 smithi066 ceph-mon[126379]: pgmap v246: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-21T22:29:28.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 312477 4.86 MB/sec execute 295 sec latency 15.723 ms 2023-07-21T22:29:29.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:28 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:29:29.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:28 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:29:29.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:28 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:29:29.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 313674 4.86 MB/sec execute 296 sec latency 38.511 ms 2023-07-21T22:29:30.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:29 smithi149 ceph-mon[129268]: pgmap v247: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-21T22:29:30.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:29 smithi104 ceph-mon[129021]: pgmap v247: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-21T22:29:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:29 smithi066 ceph-mon[126379]: pgmap v247: 129 pgs: 129 active+clean; 248 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 134 op/s 2023-07-21T22:29:30.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 315859 4.89 MB/sec execute 297 sec latency 28.350 ms 2023-07-21T22:29:31.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 317089 4.89 MB/sec execute 298 sec latency 15.037 ms 2023-07-21T22:29:32.240 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:31 smithi149 ceph-mon[129268]: pgmap v248: 129 pgs: 129 active+clean; 252 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-21T22:29:32.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:31 smithi104 ceph-mon[129021]: pgmap v248: 129 pgs: 129 active+clean; 252 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-21T22:29:32.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:31 smithi066 ceph-mon[126379]: pgmap v248: 129 pgs: 129 active+clean; 252 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2023-07-21T22:29:32.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 319380 4.92 MB/sec execute 299 sec latency 15.300 ms 2023-07-21T22:29:33.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 320305 4.91 MB/sec execute 300 sec latency 25.152 ms 2023-07-21T22:29:34.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:33 smithi149 ceph-mon[129268]: pgmap v249: 129 pgs: 129 active+clean; 252 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2023-07-21T22:29:34.242 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:33 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:29:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:33 smithi104 ceph-mon[129021]: pgmap v249: 129 pgs: 129 active+clean; 252 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2023-07-21T22:29:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:33 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:29:34.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:33 smithi066 ceph-mon[126379]: pgmap v249: 129 pgs: 129 active+clean; 252 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 1.4 MiB/s wr, 100 op/s 2023-07-21T22:29:34.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:33 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:29:34.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 321854 4.92 MB/sec execute 301 sec latency 20.066 ms 2023-07-21T22:29:35.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 323275 4.94 MB/sec execute 302 sec latency 23.743 ms 2023-07-21T22:29:36.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:35 smithi104 ceph-mon[129021]: pgmap v250: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2023-07-21T22:29:36.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:35 smithi066 ceph-mon[126379]: pgmap v250: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2023-07-21T22:29:36.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:35 smithi149 ceph-mon[129268]: pgmap v250: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 181 op/s 2023-07-21T22:29:36.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 324886 4.94 MB/sec execute 303 sec latency 17.079 ms 2023-07-21T22:29:37.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 326724 4.97 MB/sec execute 304 sec latency 14.368 ms 2023-07-21T22:29:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:37 smithi104 ceph-mon[129021]: pgmap v251: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2023-07-21T22:29:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:37 smithi066 ceph-mon[126379]: pgmap v251: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2023-07-21T22:29:38.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:37 smithi149 ceph-mon[129268]: pgmap v251: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2023-07-21T22:29:38.552 INFO:tasks.workunit.client.0.smithi066.stdout: 1 327845 4.97 MB/sec execute 305 sec latency 19.519 ms 2023-07-21T22:29:39.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 329532 4.98 MB/sec execute 306 sec latency 35.761 ms 2023-07-21T22:29:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:39 smithi104 ceph-mon[129021]: pgmap v252: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2023-07-21T22:29:40.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:39 smithi066 ceph-mon[126379]: pgmap v252: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2023-07-21T22:29:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:39 smithi149 ceph-mon[129268]: pgmap v252: 129 pgs: 129 active+clean; 258 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 130 op/s 2023-07-21T22:29:40.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 330618 4.98 MB/sec execute 307 sec latency 23.814 ms 2023-07-21T22:29:41.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 332645 5.00 MB/sec execute 308 sec latency 18.131 ms 2023-07-21T22:29:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:41 smithi104 ceph-mon[129021]: pgmap v253: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2023-07-21T22:29:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:41 smithi066 ceph-mon[126379]: pgmap v253: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2023-07-21T22:29:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:41 smithi149 ceph-mon[129268]: pgmap v253: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2023-07-21T22:29:42.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 334002 5.01 MB/sec execute 309 sec latency 15.248 ms 2023-07-21T22:29:43.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 335845 5.02 MB/sec execute 310 sec latency 16.487 ms 2023-07-21T22:29:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:43 smithi104 ceph-mon[129021]: pgmap v254: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2023-07-21T22:29:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:43 smithi066 ceph-mon[126379]: pgmap v254: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2023-07-21T22:29:44.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:43 smithi149 ceph-mon[129268]: pgmap v254: 129 pgs: 129 active+clean; 267 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2023-07-21T22:29:44.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 337398 5.04 MB/sec execute 311 sec latency 50.683 ms 2023-07-21T22:29:45.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 338754 5.04 MB/sec execute 312 sec latency 20.416 ms 2023-07-21T22:29:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:45 smithi104 ceph-mon[129021]: pgmap v255: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2023-07-21T22:29:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:45 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:45 smithi066 ceph-mon[126379]: pgmap v255: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2023-07-21T22:29:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:45 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:46.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:45 smithi149 ceph-mon[129268]: pgmap v255: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 248 op/s 2023-07-21T22:29:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:45 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:29:46.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 340786 5.07 MB/sec execute 313 sec latency 21.344 ms 2023-07-21T22:29:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:46 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:46 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:46 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:29:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:46 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:46 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:46 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:29:47.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:46 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:29:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:46 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:29:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:46 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:29:47.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 341673 5.06 MB/sec execute 314 sec latency 143.429 ms 2023-07-21T22:29:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:47 smithi104 ceph-mon[129021]: pgmap v256: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2023-07-21T22:29:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:47 smithi066 ceph-mon[126379]: pgmap v256: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2023-07-21T22:29:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:47 smithi149 ceph-mon[129268]: pgmap v256: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2023-07-21T22:29:48.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 344009 5.09 MB/sec execute 315 sec latency 24.105 ms 2023-07-21T22:29:49.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 345011 5.08 MB/sec execute 316 sec latency 13.275 ms 2023-07-21T22:29:50.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:49 smithi104 ceph-mon[129021]: pgmap v257: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2023-07-21T22:29:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:49 smithi066 ceph-mon[126379]: pgmap v257: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2023-07-21T22:29:50.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:49 smithi149 ceph-mon[129268]: pgmap v257: 129 pgs: 129 active+clean; 276 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2023-07-21T22:29:50.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 346851 5.10 MB/sec execute 317 sec latency 34.976 ms 2023-07-21T22:29:51.266 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:29:51.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 -- 172.21.15.66:0/2450381149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac101f70 msgr2=0x7f88ac102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 --2- 172.21.15.66:0/2450381149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac101f70 0x7f88ac102340 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 -- 172.21.15.66:0/2450381149 shutdown_connections 2023-07-21T22:29:51.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 --2- 172.21.15.66:0/2450381149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac101f70 0x7f88ac102340 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 --2- 172.21.15.66:0/2450381149 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88ac106fb0 0x7f88ac10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 --2- 172.21.15.66:0/2450381149 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac102910 0x7f88ac1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 -- 172.21.15.66:0/2450381149 >> 172.21.15.66:0/2450381149 conn(0x7f88ac0fd7e0 msgr2=0x7f88ac0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:51.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 -- 172.21.15.66:0/2450381149 shutdown_connections 2023-07-21T22:29:51.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.452+0000 7f88b2dff700 1 -- 172.21.15.66:0/2450381149 wait complete. 2023-07-21T22:29:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 Processor -- start 2023-07-21T22:29:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 -- start start 2023-07-21T22:29:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88ac101f70 0x7f88ac197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac102910 0x7f88ac198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 0x7f88ac19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19cdd0 con 0x7f88ac101f70 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19cf40 con 0x7f88ac102910 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.454+0000 7f88b2dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac19d100 con 0x7f88ac106fb0 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 0x7f88ac19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac102910 0x7f88ac198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 0x7f88ac19c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58014/0 (socket says 172.21.15.66:58014) 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 -- 172.21.15.66:0/74576743 learned_addr learned my addr 172.21.15.66:0/74576743 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:51.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88ac101f70 0x7f88ac197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 -- 172.21.15.66:0/74576743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac102910 msgr2=0x7f88ac198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac102910 0x7f88ac198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 -- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88ac101f70 msgr2=0x7f88ac197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88ac101f70 0x7f88ac197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 -- 172.21.15.66:0/74576743 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a00095f0 con 0x7f88ac106fb0 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88abfff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac102910 0x7f88ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88b139c700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 0x7f88ac19c790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f88a400ef30 tx=0x7f88a400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:51.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.455+0000 7f88a9ffb700 1 -- 172.21.15.66:0/74576743 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4018070 con 0x7f88ac106fb0 2023-07-21T22:29:51.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.456+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac19d380 con 0x7f88ac106fb0 2023-07-21T22:29:51.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.456+0000 7f88a9ffb700 1 -- 172.21.15.66:0/74576743 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88a400f040 con 0x7f88ac106fb0 2023-07-21T22:29:51.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.456+0000 7f88a9ffb700 1 -- 172.21.15.66:0/74576743 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400cd00 con 0x7f88ac106fb0 2023-07-21T22:29:51.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.456+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac1a4c20 con 0x7f88ac106fb0 2023-07-21T22:29:51.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.461+0000 7f88a9ffb700 1 -- 172.21.15.66:0/74576743 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f88a4003a00 con 0x7f88ac106fb0 2023-07-21T22:29:51.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.462+0000 7f88c67a9b80 1 Processor -- start 2023-07-21T22:29:51.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.462+0000 7f88c67a9b80 1 -- start start 2023-07-21T22:29:51.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4de9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4de4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4ce9cb0 con 0x557cf4de91b0 2023-07-21T22:29:51.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4d3ca40 con 0x557cf4de9ac0 2023-07-21T22:29:51.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4d35ed0 con 0x557cf4de4930 2023-07-21T22:29:51.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4de4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4de4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48396/0 (socket says 172.21.15.66:48396) 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88b0b9b700 1 -- 172.21.15.66:0/590155006 learned_addr learned my addr 172.21.15.66:0/590155006 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88a9ffb700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f889c0a1340 0x7f889c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88abfff700 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4de9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88a9ffb700 1 -- 172.21.15.66:0/74576743 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88a40ccad0 con 0x7f88ac106fb0 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88abfff700 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.463+0000 7f88b139c700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f889c0a1340 0x7f889c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b0b9b700 1 -- 172.21.15.66:0/590155006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 msgr2=0x557cf4de4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b0b9b700 1 -- 172.21.15.66:0/590155006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 msgr2=0x557cf4de9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4de9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b0b9b700 1 -- 172.21.15.66:0/590155006 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cf4b63de0 con 0x557cf4de9ac0 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4de4350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a0005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=852f082b90c8f33b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f88b139c700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f889c0a1340 0x7f889c0a37f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88a40c9f30 tx=0x7f88a401c000 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:51.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f8896ffd700 1 -- 172.21.15.66:0/590155006 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d070 con 0x557cf4de9ac0 2023-07-21T22:29:51.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f8896ffd700 1 -- 172.21.15.66:0/590155006 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88a001e040 con 0x557cf4de9ac0 2023-07-21T22:29:51.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.464+0000 7f8896ffd700 1 -- 172.21.15.66:0/590155006 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0004920 con 0x557cf4de9ac0 2023-07-21T22:29:51.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 -- 172.21.15.66:0/590155006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 msgr2=0x557cf4de4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4de4350 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009990 tx=0x7f88a0005f00 comp rx=0 tx=0).stop 2023-07-21T22:29:51.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 -- 172.21.15.66:0/590155006 shutdown_connections 2023-07-21T22:29:51.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4de9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/590155006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4de4350 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 -- 172.21.15.66:0/590155006 >> 172.21.15.66:0/590155006 conn(0x557cf4dd9ef0 msgr2=0x557cf4ddf830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 -- 172.21.15.66:0/590155006 shutdown_connections 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.465+0000 7f88c67a9b80 1 -- 172.21.15.66:0/590155006 wait complete. 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.466+0000 7f88c67a9b80 1 -- start start 2023-07-21T22:29:51.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.466+0000 7f88c67a9b80 1 Processor -- start 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de4930 0x557cf4df0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 0x557cf4dedec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4dee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4cd5ac0 con 0x557cf4de9ac0 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4cd57c0 con 0x557cf4de4930 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4bb9920 con 0x557cf4de91b0 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88c67a9b80 10 client.0 _ll_register_callbacks cb 0x557cf4cd3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:51.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 0x557cf4dedec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de4930 0x557cf4df0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 0x557cf4dedec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58032/0 (socket says 172.21.15.66:58032) 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88b0b9b700 1 -- 172.21.15.66:0/711123197 learned_addr learned my addr 172.21.15.66:0/711123197 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.467+0000 7f88abfff700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4dee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88b0b9b700 1 -- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de4930 msgr2=0x557cf4df0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de4930 0x557cf4df0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88b0b9b700 1 -- 172.21.15.66:0/711123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 msgr2=0x557cf4dee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4dee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88b0b9b700 1 -- 172.21.15.66:0/711123197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac10f3c0 con 0x557cf4de91b0 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88abfff700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de4930 0x557cf4df0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 0x557cf4dedec0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b6f0 tx=0x7f88a0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bf9d658cee8939c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f888bfff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f888bfff700 1 -- 172.21.15.66:0/711123197 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d040 con 0x557cf4de91b0 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cf4c8f4f0 con 0x557cf4de91b0 2023-07-21T22:29:51.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.468+0000 7f888bfff700 1 -- 172.21.15.66:0/711123197 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88a000f870 con 0x557cf4de91b0 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f888bfff700 1 -- 172.21.15.66:0/711123197 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88a000fa30 con 0x557cf4de91b0 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 10 client.26719 fetch_fsmap learned FSMap version 38 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557cf4b4cc60 con 0x557cf4de91b0 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f888bfff700 1 -- 172.21.15.66:0/711123197 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557cf4c8f4f0 con 0x557cf4de91b0 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f888bfff700 1 -- 172.21.15.66:0/711123197 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f88a002ec70 con 0x557cf4de91b0 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 10 client.26719 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:29:51.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 10 client.26719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:51.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 20 client.26719 populate_metadata read hostname 'smithi066' 2023-07-21T22:29:51.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4dfe010 0x557cf4e003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 4 client.26719 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:29:51.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cf4d2eeb0 con 0x557cf4dfe010 2023-07-21T22:29:51.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.469+0000 7f88abfff700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4dfe010 0x557cf4e003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.470+0000 7f88abfff700 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4dfe010 0x557cf4e003f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a7fda1975c63e8 server_cookie=fbbcbff65ace751c in_seq=0 out_seq=0 2023-07-21T22:29:51.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.470+0000 7f888bfff700 10 client.26719 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f888bfff700 1 -- 172.21.15.66:0/711123197 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557cf4d2eeb0 con 0x557cf4dfe010 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f888bfff700 10 client.26719 handle_command_reply: tid=0 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 client.26719 shutdown 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 msgr2=0x557cf4dedec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 0x557cf4dedec0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f88a000b6f0 tx=0x7f88a0000bc0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 shutdown_connections 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de4930 0x557cf4df0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4dfe010 0x557cf4e003f0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4dee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/711123197 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de91b0 0x557cf4dedec0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f888bfff700 0 client.26719 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 >> 172.21.15.66:0/711123197 conn(0x557cf4dd9ef0 msgr2=0x557cf4ddf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 shutdown_connections 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 1 -- 172.21.15.66:0/711123197 wait complete. 2023-07-21T22:29:51.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.471+0000 7f88c67a9b80 20 client.26719 trim_cache size 0 max 16384 2023-07-21T22:29:51.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 Processor -- start 2023-07-21T22:29:51.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 -- start start 2023-07-21T22:29:51.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 0x557cf4de5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4df0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4de3230 con 0x557cf4de9ac0 2023-07-21T22:29:51.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4de33a0 con 0x557cf4de91b0 2023-07-21T22:29:51.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.491+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4de2340 con 0x557cf4de4930 2023-07-21T22:29:51.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 0x557cf4de5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 0x557cf4de5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:48416/0 (socket says 172.21.15.66:48416) 2023-07-21T22:29:51.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 -- 172.21.15.66:0/2857215510 learned_addr learned my addr 172.21.15.66:0/2857215510 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:51.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88abfff700 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4df0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88abfff700 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 -- 172.21.15.66:0/2857215510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 msgr2=0x557cf4de4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 -- 172.21.15.66:0/2857215510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 msgr2=0x557cf4df0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4df0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 -- 172.21.15.66:0/2857215510 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cf4b8a250 con 0x557cf4de91b0 2023-07-21T22:29:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88abfff700 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4df0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:29:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 0x557cf4de5240 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005850 tx=0x7f88a0005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87a4023222c234e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:51.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.492+0000 7f88977fe700 1 -- 172.21.15.66:0/2857215510 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a001d040 con 0x557cf4de91b0 2023-07-21T22:29:51.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88977fe700 1 -- 172.21.15.66:0/2857215510 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88a000b2b0 con 0x557cf4de91b0 2023-07-21T22:29:51.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88977fe700 1 -- 172.21.15.66:0/2857215510 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0017aa0 con 0x557cf4de91b0 2023-07-21T22:29:51.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 -- 172.21.15.66:0/2857215510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 msgr2=0x557cf4de5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 0x557cf4de5240 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005850 tx=0x7f88a0005bc0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 -- 172.21.15.66:0/2857215510 shutdown_connections 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de9ac0 0x557cf4df0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4de4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/2857215510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de91b0 0x557cf4de5240 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 -- 172.21.15.66:0/2857215510 >> 172.21.15.66:0/2857215510 conn(0x557cf4dd9ef0 msgr2=0x557cf4de2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.493+0000 7f88c67a9b80 1 -- 172.21.15.66:0/2857215510 shutdown_connections 2023-07-21T22:29:51.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.494+0000 7f88c67a9b80 1 -- 172.21.15.66:0/2857215510 wait complete. 2023-07-21T22:29:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.494+0000 7f88c67a9b80 1 -- start start 2023-07-21T22:29:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.494+0000 7f88c67a9b80 1 Processor -- start 2023-07-21T22:29:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4df2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4df3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4df3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4e50fe0 con 0x557cf4de91b0 2023-07-21T22:29:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4e51150 con 0x557cf4de9ac0 2023-07-21T22:29:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cf4e51310 con 0x557cf4de4930 2023-07-21T22:29:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88c67a9b80 10 client.0 _ll_register_callbacks cb 0x7f8898009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:29:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4df3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4df3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:50742/0 (socket says 172.21.15.66:50742) 2023-07-21T22:29:51.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 -- 172.21.15.66:0/1460538747 learned_addr learned my addr 172.21.15.66:0/1460538747 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:29:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88abfff700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4df3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88abfff700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4df2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 -- 172.21.15.66:0/1460538747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 msgr2=0x557cf4df2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4df2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 -- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 msgr2=0x557cf4df3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:29:51.549 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4df3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88b0b9b700 1 -- 172.21.15.66:0/1460538747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cf4b4cc60 con 0x557cf4de91b0 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.495+0000 7f88abfff700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4df3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f88b0b9b700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4df3310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f88a00098e0 tx=0x7f88a00176b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587df2680774ab6e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f888bfff700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f888bfff700 1 -- 172.21.15.66:0/1460538747 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a002d580 con 0x557cf4de91b0 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88a001e040 con 0x557cf4de91b0 2023-07-21T22:29:51.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f888bfff700 1 -- 172.21.15.66:0/1460538747 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88a002d6e0 con 0x557cf4de91b0 2023-07-21T22:29:51.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f888bfff700 1 -- 172.21.15.66:0/1460538747 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88a002df00 con 0x557cf4de91b0 2023-07-21T22:29:51.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f88c67a9b80 10 client.16692 fetch_fsmap learned FSMap version 38 2023-07-21T22:29:51.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.496+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557cf4b622b0 con 0x557cf4de91b0 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f888bfff700 1 -- 172.21.15.66:0/1460538747 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88a001e040 con 0x557cf4de91b0 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f888bfff700 1 -- 172.21.15.66:0/1460538747 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f88a0047070 con 0x557cf4de91b0 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88c67a9b80 10 client.16692 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88c67a9b80 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88c67a9b80 20 client.16692 populate_metadata read hostname 'smithi066' 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4e525c0 0x557cf4e549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:29:51.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88c67a9b80 4 client.16692 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557cf4cf1050 con 0x557cf4e525c0 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.497+0000 7f88abfff700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4e525c0 0x557cf4e549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88abfff700 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4e525c0 0x557cf4e549d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68954cbe216f1c71 server_cookie=af56d8601fd66ccb in_seq=0 out_seq=0 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f888bfff700 10 client.16692 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f888bfff700 1 -- 172.21.15.66:0/1460538747 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557cf4cf1050 con 0x557cf4e525c0 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f888bfff700 10 client.16692 handle_command_reply: tid=0 2023-07-21T22:29:51.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 client.16692 shutdown 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 msgr2=0x557cf4df3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4df3310 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f88a00098e0 tx=0x7f88a00176b0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 shutdown_connections 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cf4de9ac0 0x557cf4df3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557cf4e525c0 0x557cf4e549d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557cf4de4930 0x557cf4df2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 --2- 172.21.15.66:0/1460538747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557cf4de91b0 0x557cf4df3310 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f888bfff700 0 client.16692 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:29:51.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 >> 172.21.15.66:0/1460538747 conn(0x557cf4dd9ef0 msgr2=0x557cf4de4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:51.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.498+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 shutdown_connections 2023-07-21T22:29:51.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88c67a9b80 1 -- 172.21.15.66:0/1460538747 wait complete. 2023-07-21T22:29:51.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88c67a9b80 20 client.16692 trim_cache size 0 max 16384 2023-07-21T22:29:51.557 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:29:51.557 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:29:51.557 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:29:51.557 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:29:51.558 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:29:51.558 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:29:51.558 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:29:51.558 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:29:51.558 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:29:51.558 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:29:51.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f889c0a1340 msgr2=0x7f889c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f889c0a1340 0x7f889c0a37f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88a40c9f30 tx=0x7f88a401c000 comp rx=0 tx=0).stop 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 msgr2=0x7f88ac19c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 0x7f88ac19c790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f88a400ef30 tx=0x7f88a400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 shutdown_connections 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f889c0a1340 0x7f889c0a37f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88ac106fb0 0x7f88ac19c790 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88ac101f70 0x7f88ac197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 --2- 172.21.15.66:0/74576743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88ac102910 0x7f88ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:29:51.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 >> 172.21.15.66:0/74576743 conn(0x7f88ac0fd7e0 msgr2=0x7f88ac0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:29:51.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.499+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 shutdown_connections 2023-07-21T22:29:51.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:29:51.500+0000 7f88b2dff700 1 -- 172.21.15.66:0/74576743 wait complete. 2023-07-21T22:29:51.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 348043 5.11 MB/sec execute 318 sec latency 202.305 ms 2023-07-21T22:29:51.564 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:29:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:51 smithi104 ceph-mon[129021]: pgmap v258: 129 pgs: 129 active+clean; 287 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 258 op/s 2023-07-21T22:29:52.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:51 smithi066 ceph-mon[126379]: pgmap v258: 129 pgs: 129 active+clean; 287 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 258 op/s 2023-07-21T22:29:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:51 smithi149 ceph-mon[129268]: pgmap v258: 129 pgs: 129 active+clean; 287 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.4 MiB/s wr, 258 op/s 2023-07-21T22:29:52.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 349219 5.11 MB/sec execute 319 sec latency 44.724 ms 2023-07-21T22:29:53.553 INFO:tasks.workunit.client.0.smithi066.stdout: 1 351399 5.14 MB/sec execute 320 sec latency 15.656 ms 2023-07-21T22:29:54.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:53 smithi104 ceph-mon[129021]: pgmap v259: 129 pgs: 129 active+clean; 287 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2023-07-21T22:29:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:53 smithi066 ceph-mon[126379]: pgmap v259: 129 pgs: 129 active+clean; 287 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2023-07-21T22:29:54.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:53 smithi149 ceph-mon[129268]: pgmap v259: 129 pgs: 129 active+clean; 287 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2023-07-21T22:29:54.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 352558 5.14 MB/sec execute 321 sec latency 20.745 ms 2023-07-21T22:29:55.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 354496 5.15 MB/sec execute 322 sec latency 37.774 ms 2023-07-21T22:29:56.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:55 smithi104 ceph-mon[129021]: pgmap v260: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2023-07-21T22:29:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:55 smithi066 ceph-mon[126379]: pgmap v260: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2023-07-21T22:29:56.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:55 smithi149 ceph-mon[129268]: pgmap v260: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 260 op/s 2023-07-21T22:29:56.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 355438 5.15 MB/sec execute 323 sec latency 36.440 ms 2023-07-21T22:29:57.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 356392 5.15 MB/sec execute 324 sec latency 132.009 ms 2023-07-21T22:29:58.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:57 smithi104 ceph-mon[129021]: pgmap v261: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2023-07-21T22:29:58.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:57 smithi066 ceph-mon[126379]: pgmap v261: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2023-07-21T22:29:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:57 smithi149 ceph-mon[129268]: pgmap v261: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2023-07-21T22:29:58.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 358318 5.17 MB/sec execute 325 sec latency 57.936 ms 2023-07-21T22:29:59.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 359322 5.17 MB/sec execute 326 sec latency 19.654 ms 2023-07-21T22:30:00.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:29:59 smithi104 ceph-mon[129021]: pgmap v262: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2023-07-21T22:30:00.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:29:59 smithi066 ceph-mon[126379]: pgmap v262: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2023-07-21T22:30:00.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:29:59 smithi149 ceph-mon[129268]: pgmap v262: 129 pgs: 129 active+clean; 291 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2023-07-21T22:30:00.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 361422 5.18 MB/sec execute 327 sec latency 31.733 ms 2023-07-21T22:30:01.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:00 smithi104 ceph-mon[129021]: overall HEALTH_OK 2023-07-21T22:30:01.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:00 smithi066 ceph-mon[126379]: overall HEALTH_OK 2023-07-21T22:30:01.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:00 smithi149 ceph-mon[129268]: overall HEALTH_OK 2023-07-21T22:30:01.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 361988 5.19 MB/sec execute 328 sec latency 89.076 ms 2023-07-21T22:30:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:01 smithi104 ceph-mon[129021]: pgmap v263: 129 pgs: 129 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2023-07-21T22:30:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:01 smithi104 ceph-mon[129021]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T22:30:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:01 smithi066 ceph-mon[126379]: pgmap v263: 129 pgs: 129 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2023-07-21T22:30:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:01 smithi066 ceph-mon[126379]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T22:30:02.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:01 smithi149 ceph-mon[129268]: pgmap v263: 129 pgs: 129 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 249 op/s 2023-07-21T22:30:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:01 smithi149 ceph-mon[129268]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T22:30:02.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 362694 5.17 MB/sec execute 329 sec latency 58.366 ms 2023-07-21T22:30:03.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 363932 5.17 MB/sec execute 330 sec latency 37.166 ms 2023-07-21T22:30:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:03 smithi104 ceph-mon[129021]: pgmap v265: 129 pgs: 129 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2023-07-21T22:30:04.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:03 smithi066 ceph-mon[126379]: pgmap v265: 129 pgs: 129 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2023-07-21T22:30:04.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:03 smithi149 ceph-mon[129268]: pgmap v265: 129 pgs: 129 active+clean; 295 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 190 op/s 2023-07-21T22:30:04.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 365712 5.20 MB/sec execute 331 sec latency 18.315 ms 2023-07-21T22:30:05.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 366899 5.20 MB/sec execute 332 sec latency 48.631 ms 2023-07-21T22:30:06.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:05 smithi104 ceph-mon[129021]: pgmap v266: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-21T22:30:06.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:05 smithi066 ceph-mon[126379]: pgmap v266: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-21T22:30:06.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:05 smithi149 ceph-mon[129268]: pgmap v266: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-21T22:30:06.554 INFO:tasks.workunit.client.0.smithi066.stdout: 1 369094 5.22 MB/sec execute 333 sec latency 20.832 ms 2023-07-21T22:30:07.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 370134 5.21 MB/sec execute 334 sec latency 20.819 ms 2023-07-21T22:30:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:07 smithi104 ceph-mon[129021]: pgmap v267: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-21T22:30:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:07 smithi104 ceph-mon[129021]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T22:30:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:07 smithi066 ceph-mon[126379]: pgmap v267: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-21T22:30:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:07 smithi066 ceph-mon[126379]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T22:30:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:07 smithi149 ceph-mon[129268]: pgmap v267: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 213 op/s 2023-07-21T22:30:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:07 smithi149 ceph-mon[129268]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T22:30:08.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 371540 5.22 MB/sec execute 335 sec latency 22.708 ms 2023-07-21T22:30:09.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 372962 5.23 MB/sec execute 336 sec latency 28.397 ms 2023-07-21T22:30:10.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:09 smithi104 ceph-mon[129021]: pgmap v269: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2023-07-21T22:30:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:09 smithi066 ceph-mon[126379]: pgmap v269: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2023-07-21T22:30:10.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:09 smithi149 ceph-mon[129268]: pgmap v269: 129 pgs: 129 active+clean; 298 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 155 op/s 2023-07-21T22:30:10.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 374305 5.23 MB/sec execute 337 sec latency 54.014 ms 2023-07-21T22:30:11.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 376286 5.26 MB/sec execute 338 sec latency 19.338 ms 2023-07-21T22:30:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:11 smithi104 ceph-mon[129021]: pgmap v270: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2023-07-21T22:30:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:11 smithi066 ceph-mon[126379]: pgmap v270: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2023-07-21T22:30:12.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:11 smithi149 ceph-mon[129268]: pgmap v270: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2023-07-21T22:30:12.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 377427 5.26 MB/sec execute 339 sec latency 16.716 ms 2023-07-21T22:30:13.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 378942 5.27 MB/sec execute 340 sec latency 18.095 ms 2023-07-21T22:30:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:14 smithi104 ceph-mon[129021]: pgmap v271: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2023-07-21T22:30:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:14 smithi066 ceph-mon[126379]: pgmap v271: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2023-07-21T22:30:14.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:14 smithi149 ceph-mon[129268]: pgmap v271: 129 pgs: 129 active+clean; 306 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2023-07-21T22:30:14.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 380182 5.27 MB/sec execute 341 sec latency 26.556 ms 2023-07-21T22:30:15.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 381705 5.27 MB/sec execute 342 sec latency 72.176 ms 2023-07-21T22:30:16.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:16 smithi104 ceph-mon[129021]: pgmap v272: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:30:16.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:16 smithi066 ceph-mon[126379]: pgmap v272: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:30:16.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:16 smithi149 ceph-mon[129268]: pgmap v272: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:30:16.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382911 5.28 MB/sec execute 343 sec latency 449.507 ms 2023-07-21T22:30:17.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382924 5.27 MB/sec execute 344 sec latency 1430.368 ms 2023-07-21T22:30:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:18 smithi104 ceph-mon[129021]: pgmap v273: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:30:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:18 smithi066 ceph-mon[126379]: pgmap v273: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:30:18.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:18 smithi149 ceph-mon[129268]: pgmap v273: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:30:18.555 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382924 5.25 MB/sec execute 345 sec latency 1001.107 ms 2023-07-21T22:30:19.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382924 5.24 MB/sec execute 346 sec latency 2001.190 ms 2023-07-21T22:30:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:20 smithi104 ceph-mon[129021]: pgmap v274: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2023-07-21T22:30:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:20 smithi066 ceph-mon[126379]: pgmap v274: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2023-07-21T22:30:20.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:20 smithi149 ceph-mon[129268]: pgmap v274: 129 pgs: 129 active+clean; 314 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2023-07-21T22:30:20.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382924 5.22 MB/sec execute 347 sec latency 3001.326 ms 2023-07-21T22:30:21.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382924 5.21 MB/sec execute 348 sec latency 4001.463 ms 2023-07-21T22:30:21.565 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:30:21.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 -- 172.21.15.66:0/1844285645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7120100480 msgr2=0x7f712010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 --2- 172.21.15.66:0/1844285645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7120100480 0x7f712010c910 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f711800b0a0 tx=0x7f711800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 -- 172.21.15.66:0/1844285645 shutdown_connections 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 --2- 172.21.15.66:0/1844285645 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7120100480 0x7f712010c910 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 --2- 172.21.15.66:0/1844285645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120105ae0 0x7f7120105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 --2- 172.21.15.66:0/1844285645 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f71200ffad0 0x7f71200fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 -- 172.21.15.66:0/1844285645 >> 172.21.15.66:0/1844285645 conn(0x7f71200fb5f0 msgr2=0x7f71200fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 -- 172.21.15.66:0/1844285645 shutdown_connections 2023-07-21T22:30:21.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.743+0000 7f71285ff700 1 -- 172.21.15.66:0/1844285645 wait complete. 2023-07-21T22:30:21.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.744+0000 7f71285ff700 1 Processor -- start 2023-07-21T22:30:21.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.744+0000 7f71285ff700 1 -- start start 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.744+0000 7f71285ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 0x7f7120197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.744+0000 7f71285ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120100480 0x7f71201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f71285ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7120105ae0 0x7f712019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f71285ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712019ce80 con 0x7f7120105ae0 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f71285ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712019cff0 con 0x7f71200ffad0 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f71285ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712019d1b0 con 0x7f7120100480 2023-07-21T22:30:21.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 0x7f7120197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 0x7f7120197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47552/0 (socket says 172.21.15.66:47552) 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 -- 172.21.15.66:0/1503853552 learned_addr learned my addr 172.21.15.66:0/1503853552 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f7126b9c700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7120105ae0 0x7f712019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f7125b9a700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120100480 0x7f71201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 -- 172.21.15.66:0/1503853552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120100480 msgr2=0x7f71201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120100480 0x7f71201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 -- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7120105ae0 msgr2=0x7f712019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7120105ae0 0x7f712019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f712639b700 1 -- 172.21.15.66:0/1503853552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009d00 con 0x7f71200ffad0 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.745+0000 7f7125b9a700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120100480 0x7f71201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.746+0000 7f712639b700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 0x7f7120197ea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e9a0 tx=0x7f711c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.746+0000 7f71177fe700 1 -- 172.21.15.66:0/1503853552 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c009790 con 0x7f71200ffad0 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.746+0000 7f71177fe700 1 -- 172.21.15.66:0/1503853552 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f711c00cbd0 con 0x7f71200ffad0 2023-07-21T22:30:21.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.746+0000 7f71177fe700 1 -- 172.21.15.66:0/1503853552 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c018680 con 0x7f71200ffad0 2023-07-21T22:30:21.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.746+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712019d490 con 0x7f71200ffad0 2023-07-21T22:30:21.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.746+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71201a4cd0 con 0x7f71200ffad0 2023-07-21T22:30:21.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.751+0000 7f71177fe700 1 -- 172.21.15.66:0/1503853552 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f711c00cd40 con 0x7f71200ffad0 2023-07-21T22:30:21.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.752+0000 7f713bfd9b80 1 Processor -- start 2023-07-21T22:30:21.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.752+0000 7f713bfd9b80 1 -- start start 2023-07-21T22:30:21.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.752+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 0x5641bf3da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.752+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3daac0 0x5641bf3d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.752+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.752+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf2dacb0 con 0x5641bf3daac0 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf32da40 con 0x5641bf3da1b0 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf326ed0 con 0x5641bf3d5930 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 0x5641bf3da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 0x5641bf3da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47556/0 (socket says 172.21.15.66:47556) 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 -- 172.21.15.66:0/3578919956 learned_addr learned my addr 172.21.15.66:0/3578919956 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f712639b700 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3daac0 0x5641bf3d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7125b9a700 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f71177fe700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f710c0a1340 0x7f710c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 -- 172.21.15.66:0/3578919956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 msgr2=0x5641bf3d5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3d5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 -- 172.21.15.66:0/3578919956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3daac0 msgr2=0x5641bf3d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3daac0 0x5641bf3d5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f7126b9c700 1 -- 172.21.15.66:0/3578919956 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641bf154de0 con 0x5641bf3da1b0 2023-07-21T22:30:21.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.753+0000 7f71177fe700 1 -- 172.21.15.66:0/1503853552 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f711c014070 con 0x7f71200ffad0 2023-07-21T22:30:21.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.754+0000 7f7126b9c700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f710c0a1340 0x7f710c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.754+0000 7f7126b9c700 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 0x5641bf3da580 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7118009c70 tx=0x7f71180092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f6db8f8db5f96ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:21.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.754+0000 7f7114bfe700 1 -- 172.21.15.66:0/3578919956 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800e040 con 0x5641bf3da1b0 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.754+0000 7f7114bfe700 1 -- 172.21.15.66:0/3578919956 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71180074e0 con 0x5641bf3da1b0 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.754+0000 7f7114bfe700 1 -- 172.21.15.66:0/3578919956 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711801c640 con 0x5641bf3da1b0 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3578919956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 msgr2=0x5641bf3da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f7126b9c700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f710c0a1340 0x7f710c0a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7118027000 tx=0x7f7118028910 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 0x5641bf3da580 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7118009c70 tx=0x7f71180092e0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3578919956 shutdown_connections 2023-07-21T22:30:21.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3da1b0 0x5641bf3da580 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3578919956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3daac0 0x5641bf3d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3578919956 >> 172.21.15.66:0/3578919956 conn(0x5641bf3caef0 msgr2=0x5641bf3d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3578919956 shutdown_connections 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3578919956 wait complete. 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.755+0000 7f713bfd9b80 1 -- start start 2023-07-21T22:30:21.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 Processor -- start 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 0x5641bf3e1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3dedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3daac0 0x5641bf3df530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf2c6ac0 con 0x5641bf3da1b0 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf2c67c0 con 0x5641bf3d5930 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf1aa920 con 0x5641bf3daac0 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f713bfd9b80 10 client.0 _ll_register_callbacks cb 0x5641bf2c4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:21.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f712639b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 0x5641bf3e1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f712639b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 0x5641bf3e1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47570/0 (socket says 172.21.15.66:47570) 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.756+0000 7f712639b700 1 -- 172.21.15.66:0/803006383 learned_addr learned my addr 172.21.15.66:0/803006383 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7125b9a700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3dedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7126b9c700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3daac0 0x5641bf3df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f712639b700 1 -- 172.21.15.66:0/803006383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3daac0 msgr2=0x5641bf3df530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f712639b700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3daac0 0x5641bf3df530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f712639b700 1 -- 172.21.15.66:0/803006383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 msgr2=0x5641bf3dedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f712639b700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3dedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f712639b700 1 -- 172.21.15.66:0/803006383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712010d210 con 0x5641bf3d5930 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7126b9c700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3daac0 0x5641bf3df530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7125b9a700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3dedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f712639b700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 0x5641bf3e1be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e970 tx=0x7f711c096360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fe79133185be7e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7109ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7109ffb700 1 -- 172.21.15.66:0/803006383 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c09a3d0 con 0x5641bf3d5930 2023-07-21T22:30:21.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641bf2804f0 con 0x5641bf3d5930 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7109ffb700 1 -- 172.21.15.66:0/803006383 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f711c0cf2e0 con 0x5641bf3d5930 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f7109ffb700 1 -- 172.21.15.66:0/803006383 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f711c0998d0 con 0x5641bf3d5930 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f713bfd9b80 10 client.26759 fetch_fsmap learned FSMap version 38 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.757+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5641bf13dc60 con 0x5641bf3d5930 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f7109ffb700 1 -- 172.21.15.66:0/803006383 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f711c0b1030 con 0x5641bf3d5930 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f7109ffb700 1 -- 172.21.15.66:0/803006383 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f711c0a1f30 con 0x5641bf3d5930 2023-07-21T22:30:21.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f713bfd9b80 10 client.26759 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:30:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f713bfd9b80 10 client.26759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f713bfd9b80 20 client.26759 populate_metadata read hostname 'smithi066' 2023-07-21T22:30:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf3ef4c0 0x5641bf3f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f713bfd9b80 4 client.26759 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641bf31feb0 con 0x5641bf3ef4c0 2023-07-21T22:30:21.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.758+0000 7f7125b9a700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf3ef4c0 0x5641bf3f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.759+0000 7f7109ffb700 10 client.26759 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:21.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.759+0000 7f7125b9a700 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf3ef4c0 0x5641bf3f18d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20d7605fb4c8746 server_cookie=77579c595456c108 in_seq=0 out_seq=0 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.759+0000 7f7109ffb700 1 -- 172.21.15.66:0/803006383 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5641bf31feb0 con 0x5641bf3ef4c0 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f7109ffb700 10 client.26759 handle_command_reply: tid=0 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 client.26759 shutdown 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 msgr2=0x5641bf3e1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 0x5641bf3e1be0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e970 tx=0x7f711c096360 comp rx=0 tx=0).stop 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 shutdown_connections 2023-07-21T22:30:21.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3d5930 0x5641bf3e1be0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf3ef4c0 0x5641bf3f18d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3daac0 0x5641bf3df530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/803006383 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3dedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f7109ffb700 0 client.26759 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 >> 172.21.15.66:0/803006383 conn(0x5641bf3caef0 msgr2=0x5641bf3d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 shutdown_connections 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 1 -- 172.21.15.66:0/803006383 wait complete. 2023-07-21T22:30:21.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.760+0000 7f713bfd9b80 20 client.26759 trim_cache size 0 max 16384 2023-07-21T22:30:21.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.778+0000 7f713bfd9b80 1 Processor -- start 2023-07-21T22:30:21.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.778+0000 7f713bfd9b80 1 -- start start 2023-07-21T22:30:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.778+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.778+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3de690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3debd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf3cd010 con 0x5641bf3da1b0 2023-07-21T22:30:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf3e2360 con 0x5641bf3daac0 2023-07-21T22:30:21.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf3ecbc0 con 0x5641bf3d5930 2023-07-21T22:30:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7125b9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3de690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f712639b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3debd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47584/0 (socket says 172.21.15.66:47584) 2023-07-21T22:30:21.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 -- 172.21.15.66:0/3784899233 learned_addr learned my addr 172.21.15.66:0/3784899233 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 -- 172.21.15.66:0/3784899233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 msgr2=0x5641bf3e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 -- 172.21.15.66:0/3784899233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 msgr2=0x5641bf3de690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3de690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.779+0000 7f7126b9c700 1 -- 172.21.15.66:0/3784899233 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641bf3cf230 con 0x5641bf3daac0 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f7126b9c700 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3debd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7118006250 tx=0x7f711802e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57bf1f4e2a81dc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f71153ff700 1 -- 172.21.15.66:0/3784899233 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800e040 con 0x5641bf3daac0 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f71153ff700 1 -- 172.21.15.66:0/3784899233 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7118028470 con 0x5641bf3daac0 2023-07-21T22:30:21.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f71153ff700 1 -- 172.21.15.66:0/3784899233 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180052a0 con 0x5641bf3daac0 2023-07-21T22:30:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3784899233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 msgr2=0x5641bf3debd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3debd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7118006250 tx=0x7f711802e000 comp rx=0 tx=0).stop 2023-07-21T22:30:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3784899233 shutdown_connections 2023-07-21T22:30:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3debd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3d5930 0x5641bf3e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/3784899233 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3da1b0 0x5641bf3de690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3784899233 >> 172.21.15.66:0/3784899233 conn(0x5641bf3caef0 msgr2=0x5641bf3d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.780+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3784899233 shutdown_connections 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.781+0000 7f713bfd9b80 1 -- 172.21.15.66:0/3784899233 wait complete. 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.781+0000 7f713bfd9b80 1 -- start start 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.781+0000 7f713bfd9b80 1 Processor -- start 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3d5930 0x5641bf3e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3da1b0 0x5641bf3e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3e0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf3e1df0 con 0x5641bf3d5930 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf3e1f60 con 0x5641bf3daac0 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641bf3e1180 con 0x5641bf3da1b0 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f713bfd9b80 10 client.0 _ll_register_callbacks cb 0x7f711c005b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f7126b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3e0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f712639b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3d5930 0x5641bf3e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f7126b9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3e0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47596/0 (socket says 172.21.15.66:47596) 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f7126b9c700 1 -- 172.21.15.66:0/343346497 learned_addr learned my addr 172.21.15.66:0/343346497 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.782+0000 7f7125b9a700 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3da1b0 0x5641bf3e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7126b9c700 1 -- 172.21.15.66:0/343346497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3da1b0 msgr2=0x5641bf3e0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7126b9c700 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3da1b0 0x5641bf3e0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7126b9c700 1 -- 172.21.15.66:0/343346497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3d5930 msgr2=0x5641bf3e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7126b9c700 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3d5930 0x5641bf3e17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7126b9c700 1 -- 172.21.15.66:0/343346497 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711802c030 con 0x5641bf3daac0 2023-07-21T22:30:21.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7126b9c700 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3e0c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7118038040 tx=0x7f711800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f36ee8841f13f0a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:21.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7109ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:30:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7109ffb700 1 -- 172.21.15.66:0/343346497 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180079a0 con 0x5641bf3daac0 2023-07-21T22:30:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71180074e0 con 0x5641bf3daac0 2023-07-21T22:30:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7109ffb700 1 -- 172.21.15.66:0/343346497 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7118021040 con 0x5641bf3daac0 2023-07-21T22:30:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f7109ffb700 1 -- 172.21.15.66:0/343346497 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71180050c0 con 0x5641bf3daac0 2023-07-21T22:30:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f713bfd9b80 10 client.26765 fetch_fsmap learned FSMap version 38 2023-07-21T22:30:21.837 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.783+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5641bf17b250 con 0x5641bf3daac0 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f7109ffb700 1 -- 172.21.15.66:0/343346497 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71180074e0 con 0x5641bf3daac0 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f7109ffb700 1 -- 172.21.15.66:0/343346497 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 38) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7118046070 con 0x5641bf3daac0 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f713bfd9b80 10 client.26765 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f713bfd9b80 10 client.26765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f713bfd9b80 20 client.26765 populate_metadata read hostname 'smithi066' 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf444140 0x5641bf446550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:21.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f713bfd9b80 4 client.26765 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5641bf2e2050 con 0x5641bf444140 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.784+0000 7f712639b700 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf444140 0x5641bf446550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f712639b700 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf444140 0x5641bf446550 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb47bb5fd7a1170b server_cookie=1874106af5f3daaa in_seq=0 out_seq=0 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f7109ffb700 10 client.26765 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f7109ffb700 1 -- 172.21.15.66:0/343346497 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5641bf2e2050 con 0x5641bf444140 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f7109ffb700 10 client.26765 handle_command_reply: tid=0 2023-07-21T22:30:21.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 client.26765 shutdown 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 msgr2=0x5641bf3e0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3e0c40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7118038040 tx=0x7f711800bf60 comp rx=0 tx=0).stop 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 shutdown_connections 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5641bf3daac0 0x5641bf3e0c40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5641bf444140 0x5641bf446550 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5641bf3d5930 0x5641bf3e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.840 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 --2- 172.21.15.66:0/343346497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5641bf3da1b0 0x5641bf3e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 >> 172.21.15.66:0/343346497 conn(0x5641bf3caef0 msgr2=0x5641bf3de690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f7109ffb700 0 client.26765 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 shutdown_connections 2023-07-21T22:30:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 1 -- 172.21.15.66:0/343346497 wait complete. 2023-07-21T22:30:21.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.785+0000 7f713bfd9b80 20 client.26765 trim_cache size 0 max 16384 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:30:21.842 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:30:21.843 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:30:21.843 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:30:21.843 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:30:21.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f710c0a1340 msgr2=0x7f710c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f710c0a1340 0x7f710c0a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7118027000 tx=0x7f7118028910 comp rx=0 tx=0).stop 2023-07-21T22:30:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 msgr2=0x7f7120197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 0x7f7120197ea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f711c00e9a0 tx=0x7f711c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 shutdown_connections 2023-07-21T22:30:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71200ffad0 0x7f7120197ea0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f710c0a1340 0x7f710c0a37f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7120105ae0 0x7f712019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 --2- 172.21.15.66:0/1503853552 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7120100480 0x7f71201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:21.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 >> 172.21.15.66:0/1503853552 conn(0x7f71200fb5f0 msgr2=0x7f712010af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:21.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 shutdown_connections 2023-07-21T22:30:21.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:21.786+0000 7f71285ff700 1 -- 172.21.15.66:0/1503853552 wait complete. 2023-07-21T22:30:21.852 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:30:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:22 smithi104 ceph-mon[129021]: pgmap v275: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 225 op/s 2023-07-21T22:30:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:22 smithi066 ceph-mon[126379]: pgmap v275: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 225 op/s 2023-07-21T22:30:22.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:22 smithi149 ceph-mon[129268]: pgmap v275: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 225 op/s 2023-07-21T22:30:22.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 382931 5.19 MB/sec execute 349 sec latency 4990.142 ms 2023-07-21T22:30:23.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 383988 5.19 MB/sec execute 350 sec latency 13.497 ms 2023-07-21T22:30:24.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:24 smithi104 ceph-mon[129021]: pgmap v276: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 133 op/s 2023-07-21T22:30:24.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:24 smithi066 ceph-mon[126379]: pgmap v276: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 133 op/s 2023-07-21T22:30:24.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:24 smithi149 ceph-mon[129268]: pgmap v276: 129 pgs: 129 active+clean; 317 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 133 op/s 2023-07-21T22:30:24.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 385911 5.21 MB/sec execute 351 sec latency 23.220 ms 2023-07-21T22:30:25.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 387207 5.21 MB/sec execute 352 sec latency 51.274 ms 2023-07-21T22:30:26.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:26 smithi104 ceph-mon[129021]: pgmap v277: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 193 op/s 2023-07-21T22:30:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:26 smithi066 ceph-mon[126379]: pgmap v277: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 193 op/s 2023-07-21T22:30:26.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:26 smithi149 ceph-mon[129268]: pgmap v277: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 193 op/s 2023-07-21T22:30:26.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 388502 5.21 MB/sec execute 353 sec latency 38.640 ms 2023-07-21T22:30:27.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 390570 5.24 MB/sec execute 354 sec latency 13.552 ms 2023-07-21T22:30:28.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:28 smithi104 ceph-mon[129021]: pgmap v278: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2023-07-21T22:30:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:28 smithi066 ceph-mon[126379]: pgmap v278: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2023-07-21T22:30:28.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:28 smithi149 ceph-mon[129268]: pgmap v278: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2023-07-21T22:30:28.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 391672 5.24 MB/sec execute 355 sec latency 25.850 ms 2023-07-21T22:30:29.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 393191 5.24 MB/sec execute 356 sec latency 16.671 ms 2023-07-21T22:30:30.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:30 smithi104 ceph-mon[129021]: pgmap v279: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2023-07-21T22:30:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:30 smithi066 ceph-mon[126379]: pgmap v279: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2023-07-21T22:30:30.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:30 smithi149 ceph-mon[129268]: pgmap v279: 129 pgs: 129 active+clean; 321 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 1.9 MiB/s wr, 103 op/s 2023-07-21T22:30:30.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 394359 5.25 MB/sec execute 357 sec latency 19.334 ms 2023-07-21T22:30:31.556 INFO:tasks.workunit.client.0.smithi066.stdout: 1 395683 5.25 MB/sec execute 358 sec latency 53.117 ms 2023-07-21T22:30:32.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:32 smithi104 ceph-mon[129021]: pgmap v280: 129 pgs: 129 active+clean; 328 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-07-21T22:30:32.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:32 smithi066 ceph-mon[126379]: pgmap v280: 129 pgs: 129 active+clean; 328 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-07-21T22:30:32.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:32 smithi149 ceph-mon[129268]: pgmap v280: 129 pgs: 129 active+clean; 328 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 191 op/s 2023-07-21T22:30:32.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 397661 5.27 MB/sec execute 359 sec latency 48.226 ms 2023-07-21T22:30:33.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 398802 5.27 MB/sec execute 360 sec latency 15.353 ms 2023-07-21T22:30:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:34 smithi104 ceph-mon[129021]: pgmap v281: 129 pgs: 129 active+clean; 328 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 148 op/s 2023-07-21T22:30:34.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:34 smithi066 ceph-mon[126379]: pgmap v281: 129 pgs: 129 active+clean; 328 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 148 op/s 2023-07-21T22:30:34.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:34 smithi149 ceph-mon[129268]: pgmap v281: 129 pgs: 129 active+clean; 328 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 148 op/s 2023-07-21T22:30:34.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 400504 5.28 MB/sec execute 361 sec latency 17.595 ms 2023-07-21T22:30:35.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 401603 5.28 MB/sec execute 362 sec latency 20.645 ms 2023-07-21T22:30:36.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:36 smithi104 ceph-mon[129021]: pgmap v282: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 234 op/s 2023-07-21T22:30:36.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:36 smithi066 ceph-mon[126379]: pgmap v282: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 234 op/s 2023-07-21T22:30:36.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:36 smithi149 ceph-mon[129268]: pgmap v282: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 234 op/s 2023-07-21T22:30:36.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 403243 5.29 MB/sec execute 363 sec latency 45.082 ms 2023-07-21T22:30:37.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:37 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:30:37.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:37 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:30:37.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:37 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:30:37.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 404827 5.30 MB/sec execute 364 sec latency 15.108 ms 2023-07-21T22:30:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:38 smithi104 ceph-mon[129021]: pgmap v283: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2023-07-21T22:30:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:38 smithi066 ceph-mon[126379]: pgmap v283: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2023-07-21T22:30:38.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:38 smithi149 ceph-mon[129268]: pgmap v283: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2023-07-21T22:30:38.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 405992 5.30 MB/sec execute 365 sec latency 24.545 ms 2023-07-21T22:30:39.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 407943 5.32 MB/sec execute 366 sec latency 16.947 ms 2023-07-21T22:30:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:40 smithi104 ceph-mon[129021]: pgmap v284: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2023-07-21T22:30:40.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:40 smithi066 ceph-mon[126379]: pgmap v284: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2023-07-21T22:30:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:40 smithi149 ceph-mon[129268]: pgmap v284: 129 pgs: 129 active+clean; 336 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2023-07-21T22:30:40.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 408861 5.32 MB/sec execute 367 sec latency 16.319 ms 2023-07-21T22:30:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:41 smithi066 ceph-mon[126379]: pgmap v285: 129 pgs: 129 active+clean; 343 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 258 op/s 2023-07-21T22:30:41.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:41 smithi149 ceph-mon[129268]: pgmap v285: 129 pgs: 129 active+clean; 343 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 258 op/s 2023-07-21T22:30:41.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 410708 5.33 MB/sec execute 368 sec latency 57.187 ms 2023-07-21T22:30:41.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:41 smithi104 ceph-mon[129021]: pgmap v285: 129 pgs: 129 active+clean; 343 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 258 op/s 2023-07-21T22:30:42.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 412066 5.34 MB/sec execute 369 sec latency 12.951 ms 2023-07-21T22:30:43.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 413197 5.34 MB/sec execute 370 sec latency 26.448 ms 2023-07-21T22:30:44.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:43 smithi149 ceph-mon[129268]: pgmap v286: 129 pgs: 129 active+clean; 343 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2023-07-21T22:30:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:43 smithi104 ceph-mon[129021]: pgmap v286: 129 pgs: 129 active+clean; 343 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2023-07-21T22:30:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:43 smithi066 ceph-mon[126379]: pgmap v286: 129 pgs: 129 active+clean; 343 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2023-07-21T22:30:44.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 415254 5.36 MB/sec execute 371 sec latency 13.202 ms 2023-07-21T22:30:45.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 416030 5.35 MB/sec execute 372 sec latency 72.652 ms 2023-07-21T22:30:46.241 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:45 smithi149 ceph-mon[129268]: pgmap v287: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2023-07-21T22:30:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:45 smithi104 ceph-mon[129021]: pgmap v287: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2023-07-21T22:30:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:45 smithi066 ceph-mon[126379]: pgmap v287: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 261 op/s 2023-07-21T22:30:46.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 417803 5.36 MB/sec execute 373 sec latency 49.531 ms 2023-07-21T22:30:47.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:46 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:47.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:46 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:47.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:46 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:30:47.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 419002 5.37 MB/sec execute 374 sec latency 73.068 ms 2023-07-21T22:30:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:47 smithi104 ceph-mon[129021]: pgmap v288: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2023-07-21T22:30:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:47 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:48.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:47 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:48.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:47 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:30:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:47 smithi066 ceph-mon[126379]: pgmap v288: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2023-07-21T22:30:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:47 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:47 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:47 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:30:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:47 smithi149 ceph-mon[129268]: pgmap v288: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2023-07-21T22:30:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:47 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:30:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:47 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:30:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:47 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:30:48.557 INFO:tasks.workunit.client.0.smithi066.stdout: 1 420251 5.37 MB/sec execute 375 sec latency 17.581 ms 2023-07-21T22:30:49.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 422399 5.39 MB/sec execute 376 sec latency 16.765 ms 2023-07-21T22:30:50.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:49 smithi104 ceph-mon[129021]: pgmap v289: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2023-07-21T22:30:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:49 smithi066 ceph-mon[126379]: pgmap v289: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2023-07-21T22:30:50.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:49 smithi149 ceph-mon[129268]: pgmap v289: 129 pgs: 129 active+clean; 350 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2023-07-21T22:30:50.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 423533 5.39 MB/sec execute 377 sec latency 26.516 ms 2023-07-21T22:30:51.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 424905 5.39 MB/sec execute 378 sec latency 17.257 ms 2023-07-21T22:30:51.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:30:52.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 -- 172.21.15.66:0/2585946957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 msgr2=0x7f053c109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 --2- 172.21.15.66:0/2585946957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c109b00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0530009990 tx=0x7f0530009ca0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 -- 172.21.15.66:0/2585946957 shutdown_connections 2023-07-21T22:30:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 --2- 172.21.15.66:0/2585946957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c100d00 0x7f053c1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 --2- 172.21.15.66:0/2585946957 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053c10a040 0x7f053c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 --2- 172.21.15.66:0/2585946957 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c109b00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.036+0000 7f05427ff700 1 -- 172.21.15.66:0/2585946957 >> 172.21.15.66:0/2585946957 conn(0x7f053c0fa4b0 msgr2=0x7f053c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.037+0000 7f05427ff700 1 -- 172.21.15.66:0/2585946957 shutdown_connections 2023-07-21T22:30:52.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.037+0000 7f05427ff700 1 -- 172.21.15.66:0/2585946957 wait complete. 2023-07-21T22:30:52.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 Processor -- start 2023-07-21T22:30:52.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 -- start start 2023-07-21T22:30:52.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053c100d00 0x7f053c1089a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c108ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c10a040 0x7f053c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c1a1220 con 0x7f053c100d00 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c1a1390 con 0x7f053c10a040 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f05427ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c1a1550 con 0x7f053c101610 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f053b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c108ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f0540d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c10a040 0x7f053c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f053b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c108ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:56276/0 (socket says 172.21.15.66:56276) 2023-07-21T22:30:52.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.038+0000 7f053b7fe700 1 -- 172.21.15.66:0/2816622783 learned_addr learned my addr 172.21.15.66:0/2816622783 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:52.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053bfff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053c100d00 0x7f053c1089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053b7fe700 1 -- 172.21.15.66:0/2816622783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c10a040 msgr2=0x7f053c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053b7fe700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c10a040 0x7f053c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053b7fe700 1 -- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053c100d00 msgr2=0x7f053c1089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053b7fe700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053c100d00 0x7f053c1089a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053b7fe700 1 -- 172.21.15.66:0/2816622783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300095f0 con 0x7f053c101610 2023-07-21T22:30:52.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f0540d9c700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c10a040 0x7f053c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:30:52.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f053b7fe700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c108ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0530005910 tx=0x7f0530004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:52.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.039+0000 7f05397fa700 1 -- 172.21.15.66:0/2816622783 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053001d070 con 0x7f053c101610 2023-07-21T22:30:52.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.040+0000 7f05397fa700 1 -- 172.21.15.66:0/2816622783 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0530022470 con 0x7f053c101610 2023-07-21T22:30:52.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.040+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c1a17d0 con 0x7f053c101610 2023-07-21T22:30:52.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.040+0000 7f05397fa700 1 -- 172.21.15.66:0/2816622783 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530017650 con 0x7f053c101610 2023-07-21T22:30:52.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.040+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c1a9070 con 0x7f053c101610 2023-07-21T22:30:52.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.043+0000 7f05397fa700 1 -- 172.21.15.66:0/2816622783 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f05300225e0 con 0x7f053c101610 2023-07-21T22:30:52.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.044+0000 7f05560a5b80 1 Processor -- start 2023-07-21T22:30:52.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 -- start start 2023-07-21T22:30:52.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed561b0 0x564baed56580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed51350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05397fa700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05240a12c0 0x7f05240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed51da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baec56cb0 con 0x564baed561b0 2023-07-21T22:30:52.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baeca9a40 con 0x564baed56ac0 2023-07-21T22:30:52.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.045+0000 7f05560a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baeca2ed0 con 0x564baed51930 2023-07-21T22:30:52.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f05397fa700 1 -- 172.21.15.66:0/2816622783 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f05300ce2d0 con 0x7f053c101610 2023-07-21T22:30:52.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed51350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed51350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38944/0 (socket says 172.21.15.66:38944) 2023-07-21T22:30:52.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 -- 172.21.15.66:0/278107466 learned_addr learned my addr 172.21.15.66:0/278107466 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:52.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f0540d9c700 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed561b0 0x564baed56580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053b7fe700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05240a12c0 0x7f05240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f0540d9c700 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed51da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 -- 172.21.15.66:0/278107466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 msgr2=0x564baed51da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed51da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 -- 172.21.15.66:0/278107466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed561b0 msgr2=0x564baed56580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed561b0 0x564baed56580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 -- 172.21.15.66:0/278107466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564baead0de0 con 0x564baed56ac0 2023-07-21T22:30:52.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053bfff700 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed51350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f052c009d80 tx=0x7f052c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcd5ef5b4f5b555 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:52.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.046+0000 7f053b7fe700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05240a12c0 0x7f05240a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05300cb680 tx=0x7f0530000bc0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.047+0000 7f05227fc700 1 -- 172.21.15.66:0/278107466 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c004230 con 0x564baed56ac0 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.047+0000 7f05227fc700 1 -- 172.21.15.66:0/278107466 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c004390 con 0x564baed56ac0 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.047+0000 7f05227fc700 1 -- 172.21.15.66:0/278107466 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c019880 con 0x564baed56ac0 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.047+0000 7f05560a5b80 1 -- 172.21.15.66:0/278107466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 msgr2=0x564baed51350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.047+0000 7f05560a5b80 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed51350 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f052c009d80 tx=0x7f052c011720 comp rx=0 tx=0).stop 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.047+0000 7f05560a5b80 1 -- 172.21.15.66:0/278107466 shutdown_connections 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed51da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed561b0 0x564baed56580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 --2- 172.21.15.66:0/278107466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed51350 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 -- 172.21.15.66:0/278107466 >> 172.21.15.66:0/278107466 conn(0x564baed46ef0 msgr2=0x564baed4c830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:52.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 -- 172.21.15.66:0/278107466 shutdown_connections 2023-07-21T22:30:52.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 -- 172.21.15.66:0/278107466 wait complete. 2023-07-21T22:30:52.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.048+0000 7f05560a5b80 1 -- start start 2023-07-21T22:30:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.049+0000 7f05560a5b80 1 Processor -- start 2023-07-21T22:30:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.049+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed51930 0x564baed5db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.049+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baed5adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.049+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed5b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f05560a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baec42ac0 con 0x564baed51930 2023-07-21T22:30:52.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f05560a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baec427c0 con 0x564baed56ac0 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f05560a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baeb26920 con 0x564baed561b0 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f05560a5b80 10 client.0 _ll_register_callbacks cb 0x564baec40760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baed5adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baed5adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:56286/0 (socket says 172.21.15.66:56286) 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 -- 172.21.15.66:0/1923546263 learned_addr learned my addr 172.21.15.66:0/1923546263 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f0540d9c700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed51930 0x564baed5db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f0540d9c700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed5b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 -- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 msgr2=0x564baed5b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed5b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 -- 172.21.15.66:0/1923546263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed51930 msgr2=0x564baed5db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed51930 0x564baed5db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f053bfff700 1 -- 172.21.15.66:0/1923546263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c0099e0 con 0x564baed561b0 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.050+0000 7f0540d9c700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed5b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f053bfff700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baed5adf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f052c009770 tx=0x7f052c0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19d9ee3dfe211d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f051b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:30:52.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f051b7fe700 1 -- 172.21.15.66:0/1923546263 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c005450 con 0x564baed561b0 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f051b7fe700 1 -- 172.21.15.66:0/1923546263 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f052c003ad0 con 0x564baed561b0 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564baebfc4f0 con 0x564baed561b0 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f051b7fe700 1 -- 172.21.15.66:0/1923546263 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f052c009ed0 con 0x564baed561b0 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f05560a5b80 10 client.26752 fetch_fsmap learned FSMap version 39 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.051+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564baeab9c60 con 0x564baed561b0 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f051b7fe700 1 -- 172.21.15.66:0/1923546263 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564baebfc4f0 con 0x564baed561b0 2023-07-21T22:30:52.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f051b7fe700 1 -- 172.21.15.66:0/1923546263 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f052c028020 con 0x564baed561b0 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f05560a5b80 10 client.26752 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f05560a5b80 10 client.26752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f05560a5b80 20 client.26752 populate_metadata read hostname 'smithi066' 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f05560a5b80 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baed6b3d0 0x564baed6d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f05560a5b80 4 client.26752 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564baec9beb0 con 0x564baed6b3d0 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.052+0000 7f0540d9c700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baed6b3d0 0x564baed6d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.053+0000 7f051b7fe700 10 client.26752 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.053+0000 7f0540d9c700 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baed6b3d0 0x564baed6d7e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee0e6cebb1ebc4e5 server_cookie=14bcc864991db179 in_seq=0 out_seq=0 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.053+0000 7f051b7fe700 1 -- 172.21.15.66:0/1923546263 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564baec9beb0 con 0x564baed6b3d0 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.053+0000 7f051b7fe700 10 client.26752 handle_command_reply: tid=0 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 client.26752 shutdown 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 msgr2=0x564baed5adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baed5adf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f052c009770 tx=0x7f052c0038b0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 shutdown_connections 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed56ac0 0x564baed5b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baed6b3d0 0x564baed6d7e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed51930 0x564baed5db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 --2- 172.21.15.66:0/1923546263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baed5adf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f051b7fe700 0 client.26752 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 >> 172.21.15.66:0/1923546263 conn(0x564baed46ef0 msgr2=0x564baed4c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 shutdown_connections 2023-07-21T22:30:52.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 1 -- 172.21.15.66:0/1923546263 wait complete. 2023-07-21T22:30:52.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.054+0000 7f05560a5b80 20 client.26752 trim_cache size 0 max 16384 2023-07-21T22:30:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.072+0000 7f05560a5b80 1 Processor -- start 2023-07-21T22:30:52.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.072+0000 7f05560a5b80 1 -- start start 2023-07-21T22:30:52.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed5d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 0x564baed5a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed5ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f05560a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baed5c410 con 0x564baed56ac0 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f05560a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baed67ec0 con 0x564baed561b0 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f05560a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baed526a0 con 0x564baed51930 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f0540d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed5d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 0x564baed5a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 0x564baed5a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38954/0 (socket says 172.21.15.66:38954) 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 -- 172.21.15.66:0/596002279 learned_addr learned my addr 172.21.15.66:0/596002279 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f0540d9c700 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed5ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 -- 172.21.15.66:0/596002279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 msgr2=0x564baed5d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed5d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 -- 172.21.15.66:0/596002279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 msgr2=0x564baed5ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.122 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed5ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 -- 172.21.15.66:0/596002279 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564baeaf7250 con 0x564baed561b0 2023-07-21T22:30:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f0540d9c700 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed5ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:30:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.073+0000 7f053bfff700 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 0x564baed5a600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f052c000c00 tx=0x7f052c022520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23531fcebf4e84b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f0522ffd700 1 -- 172.21.15.66:0/596002279 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c009ed0 con 0x564baed561b0 2023-07-21T22:30:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f0522ffd700 1 -- 172.21.15.66:0/596002279 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c022670 con 0x564baed561b0 2023-07-21T22:30:52.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f0522ffd700 1 -- 172.21.15.66:0/596002279 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c018b70 con 0x564baed561b0 2023-07-21T22:30:52.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 -- 172.21.15.66:0/596002279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 msgr2=0x564baed5a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 0x564baed5a600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f052c000c00 tx=0x7f052c022520 comp rx=0 tx=0).stop 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 -- 172.21.15.66:0/596002279 shutdown_connections 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed5ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed51930 0x564baed5d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 --2- 172.21.15.66:0/596002279 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed561b0 0x564baed5a600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.074+0000 7f05560a5b80 1 -- 172.21.15.66:0/596002279 >> 172.21.15.66:0/596002279 conn(0x564baed46ef0 msgr2=0x564baed4f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:52.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.075+0000 7f05560a5b80 1 -- 172.21.15.66:0/596002279 shutdown_connections 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.075+0000 7f05560a5b80 1 -- 172.21.15.66:0/596002279 wait complete. 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.075+0000 7f05560a5b80 1 -- start start 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.075+0000 7f05560a5b80 1 Processor -- start 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed51930 0x564baedb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baedb8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baed60b30 con 0x564baed56ac0 2023-07-21T22:30:52.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baed60c70 con 0x564baed51930 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564baed60e30 con 0x564baed561b0 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f05560a5b80 10 client.0 _ll_register_callbacks cb 0x7f0534003c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baedb8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baedb8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:56296/0 (socket says 172.21.15.66:56296) 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 -- 172.21.15.66:0/2785637986 learned_addr learned my addr 172.21.15.66:0/2785637986 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f0540d9c700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed51930 0x564baedb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f0540d9c700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 -- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed51930 msgr2=0x564baedb8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed51930 0x564baedb8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 -- 172.21.15.66:0/2785637986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 msgr2=0x564baed605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.076+0000 7f053bfff700 1 -- 172.21.15.66:0/2785637986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564baeab9c60 con 0x564baed561b0 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f053bfff700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baedb8990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f052c018a70 tx=0x7f052c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aa0a052e47b3b25 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f051b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:30:52.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f051b7fe700 1 -- 172.21.15.66:0/2785637986 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0227e0 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f052c004390 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f051b7fe700 1 -- 172.21.15.66:0/2785637986 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f052c000e40 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f051b7fe700 1 -- 172.21.15.66:0/2785637986 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f052c022940 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f05560a5b80 10 client.26758 fetch_fsmap learned FSMap version 39 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.077+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564baeacf2b0 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f051b7fe700 1 -- 172.21.15.66:0/2785637986 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052c004390 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f051b7fe700 1 -- 172.21.15.66:0/2785637986 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f052c011790 con 0x564baed561b0 2023-07-21T22:30:52.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f05560a5b80 10 client.26758 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f05560a5b80 10 client.26758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f05560a5b80 20 client.26758 populate_metadata read hostname 'smithi066' 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f05560a5b80 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baedbfd60 0x564baedc2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f05560a5b80 4 client.26758 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564baec5e050 con 0x564baedbfd60 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.078+0000 7f0540d9c700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baedbfd60 0x564baedc2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:30:52.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f0540d9c700 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baedbfd60 0x564baedc2140 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241531314650075 server_cookie=9a8811cb4c72b161 in_seq=0 out_seq=0 2023-07-21T22:30:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f051b7fe700 10 client.26758 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f051b7fe700 1 -- 172.21.15.66:0/2785637986 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564baec5e050 con 0x564baedbfd60 2023-07-21T22:30:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f051b7fe700 10 client.26758 handle_command_reply: tid=0 2023-07-21T22:30:52.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 client.26758 shutdown 2023-07-21T22:30:52.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 msgr2=0x564baedb8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baedb8990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f052c018a70 tx=0x7f052c005e50 comp rx=0 tx=0).stop 2023-07-21T22:30:52.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 shutdown_connections 2023-07-21T22:30:52.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564baed51930 0x564baedb8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564baedbfd60 0x564baedc2140 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564baed56ac0 0x564baed605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 --2- 172.21.15.66:0/2785637986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564baed561b0 0x564baedb8990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f051b7fe700 0 client.26758 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 >> 172.21.15.66:0/2785637986 conn(0x564baed46ef0 msgr2=0x564baed508e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 shutdown_connections 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.079+0000 7f05560a5b80 1 -- 172.21.15.66:0/2785637986 wait complete. 2023-07-21T22:30:52.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05560a5b80 20 client.26758 trim_cache size 0 max 16384 2023-07-21T22:30:52.135 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:30:52.135 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:30:52.135 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:30:52.135 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:30:52.135 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:30:52.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:30:52.136 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:30:52.136 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:30:52.136 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:30:52.136 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:30:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05240a12c0 msgr2=0x7f05240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05240a12c0 0x7f05240a3770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f05300cb680 tx=0x7f0530000bc0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 msgr2=0x7f053c108ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:30:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c108ee0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0530005910 tx=0x7f0530004710 comp rx=0 tx=0).stop 2023-07-21T22:30:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 shutdown_connections 2023-07-21T22:30:52.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f053c10a040 0x7f053c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f05240a12c0 0x7f05240a3770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f053c100d00 0x7f053c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 --2- 172.21.15.66:0/2816622783 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f053c101610 0x7f053c108ee0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:30:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 >> 172.21.15.66:0/2816622783 conn(0x7f053c0fa4b0 msgr2=0x7f053c10c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:30:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 shutdown_connections 2023-07-21T22:30:52.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:30:52.080+0000 7f05427ff700 1 -- 172.21.15.66:0/2816622783 wait complete. 2023-07-21T22:30:52.145 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:30:52.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:51 smithi066 ceph-mon[126379]: pgmap v290: 129 pgs: 129 active+clean; 358 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 263 op/s 2023-07-21T22:30:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:51 smithi104 ceph-mon[129021]: pgmap v290: 129 pgs: 129 active+clean; 358 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 263 op/s 2023-07-21T22:30:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:51 smithi149 ceph-mon[129268]: pgmap v290: 129 pgs: 129 active+clean; 358 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 263 op/s 2023-07-21T22:30:52.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 426231 5.40 MB/sec execute 379 sec latency 17.392 ms 2023-07-21T22:30:53.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 427534 5.40 MB/sec execute 380 sec latency 50.061 ms 2023-07-21T22:30:54.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:53 smithi104 ceph-mon[129021]: pgmap v291: 129 pgs: 129 active+clean; 358 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2023-07-21T22:30:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:53 smithi066 ceph-mon[126379]: pgmap v291: 129 pgs: 129 active+clean; 358 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2023-07-21T22:30:54.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:53 smithi149 ceph-mon[129268]: pgmap v291: 129 pgs: 129 active+clean; 358 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2023-07-21T22:30:54.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 429527 5.42 MB/sec execute 381 sec latency 30.232 ms 2023-07-21T22:30:55.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 430624 5.42 MB/sec execute 382 sec latency 28.371 ms 2023-07-21T22:30:56.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:55 smithi104 ceph-mon[129021]: pgmap v292: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2023-07-21T22:30:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:55 smithi066 ceph-mon[126379]: pgmap v292: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2023-07-21T22:30:56.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:55 smithi149 ceph-mon[129268]: pgmap v292: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 250 op/s 2023-07-21T22:30:56.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 432120 5.42 MB/sec execute 383 sec latency 30.727 ms 2023-07-21T22:30:57.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 433318 5.43 MB/sec execute 384 sec latency 32.616 ms 2023-07-21T22:30:58.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:57 smithi104 ceph-mon[129021]: pgmap v293: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2023-07-21T22:30:58.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:57 smithi066 ceph-mon[126379]: pgmap v293: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2023-07-21T22:30:58.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:57 smithi149 ceph-mon[129268]: pgmap v293: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2023-07-21T22:30:58.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 434745 5.43 MB/sec execute 385 sec latency 31.540 ms 2023-07-21T22:30:59.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 436650 5.45 MB/sec execute 386 sec latency 14.810 ms 2023-07-21T22:31:00.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:30:59 smithi104 ceph-mon[129021]: pgmap v294: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2023-07-21T22:31:00.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:30:59 smithi066 ceph-mon[126379]: pgmap v294: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2023-07-21T22:31:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:30:59 smithi149 ceph-mon[129268]: pgmap v294: 129 pgs: 129 active+clean; 368 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 160 op/s 2023-07-21T22:31:00.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 437766 5.45 MB/sec execute 387 sec latency 24.789 ms 2023-07-21T22:31:01.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 438584 5.44 MB/sec execute 388 sec latency 54.295 ms 2023-07-21T22:31:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:01 smithi104 ceph-mon[129021]: pgmap v295: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-07-21T22:31:02.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:01 smithi104 ceph-mon[129021]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T22:31:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:01 smithi066 ceph-mon[126379]: pgmap v295: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-07-21T22:31:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:01 smithi066 ceph-mon[126379]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T22:31:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:01 smithi149 ceph-mon[129268]: pgmap v295: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 242 op/s 2023-07-21T22:31:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:01 smithi149 ceph-mon[129268]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T22:31:02.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 439233 5.44 MB/sec execute 389 sec latency 96.836 ms 2023-07-21T22:31:03.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 440448 5.44 MB/sec execute 390 sec latency 22.301 ms 2023-07-21T22:31:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:03 smithi104 ceph-mon[129021]: pgmap v297: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2023-07-21T22:31:04.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:03 smithi066 ceph-mon[126379]: pgmap v297: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2023-07-21T22:31:04.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:03 smithi149 ceph-mon[129268]: pgmap v297: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2023-07-21T22:31:04.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 442190 5.44 MB/sec execute 391 sec latency 20.124 ms 2023-07-21T22:31:05.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 443811 5.46 MB/sec execute 392 sec latency 11.049 ms 2023-07-21T22:31:06.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:05 smithi104 ceph-mon[129021]: pgmap v298: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-07-21T22:31:06.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:05 smithi066 ceph-mon[126379]: pgmap v298: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-07-21T22:31:06.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:05 smithi149 ceph-mon[129268]: pgmap v298: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-07-21T22:31:06.558 INFO:tasks.workunit.client.0.smithi066.stdout: 1 444934 5.46 MB/sec execute 393 sec latency 39.764 ms 2023-07-21T22:31:07.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 447087 5.48 MB/sec execute 394 sec latency 18.960 ms 2023-07-21T22:31:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:07 smithi104 ceph-mon[129021]: pgmap v299: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-07-21T22:31:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:07 smithi104 ceph-mon[129021]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T22:31:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:07 smithi066 ceph-mon[126379]: pgmap v299: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-07-21T22:31:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:07 smithi066 ceph-mon[126379]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T22:31:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:07 smithi149 ceph-mon[129268]: pgmap v299: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 231 op/s 2023-07-21T22:31:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:07 smithi149 ceph-mon[129268]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T22:31:08.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 447927 5.47 MB/sec execute 395 sec latency 27.446 ms 2023-07-21T22:31:09.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 449336 5.47 MB/sec execute 396 sec latency 16.409 ms 2023-07-21T22:31:10.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:09 smithi104 ceph-mon[129021]: pgmap v301: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2023-07-21T22:31:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:09 smithi066 ceph-mon[126379]: pgmap v301: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2023-07-21T22:31:10.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:09 smithi149 ceph-mon[129268]: pgmap v301: 129 pgs: 129 active+clean; 380 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2023-07-21T22:31:10.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 450883 5.49 MB/sec execute 397 sec latency 19.227 ms 2023-07-21T22:31:11.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 452014 5.49 MB/sec execute 398 sec latency 31.329 ms 2023-07-21T22:31:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:11 smithi104 ceph-mon[129021]: pgmap v302: 129 pgs: 129 active+clean; 388 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 239 op/s 2023-07-21T22:31:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:11 smithi066 ceph-mon[126379]: pgmap v302: 129 pgs: 129 active+clean; 388 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 239 op/s 2023-07-21T22:31:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:11 smithi149 ceph-mon[129268]: pgmap v302: 129 pgs: 129 active+clean; 388 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 239 op/s 2023-07-21T22:31:12.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 454045 5.51 MB/sec execute 399 sec latency 78.884 ms 2023-07-21T22:31:13.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 455030 5.50 MB/sec execute 400 sec latency 24.980 ms 2023-07-21T22:31:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:13 smithi104 ceph-mon[129021]: pgmap v303: 129 pgs: 129 active+clean; 388 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 236 op/s 2023-07-21T22:31:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:13 smithi066 ceph-mon[126379]: pgmap v303: 129 pgs: 129 active+clean; 388 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 236 op/s 2023-07-21T22:31:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:13 smithi149 ceph-mon[129268]: pgmap v303: 129 pgs: 129 active+clean; 388 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 236 op/s 2023-07-21T22:31:14.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 456374 5.50 MB/sec execute 401 sec latency 15.027 ms 2023-07-21T22:31:15.559 INFO:tasks.workunit.client.0.smithi066.stdout: 1 457949 5.52 MB/sec execute 402 sec latency 96.893 ms 2023-07-21T22:31:16.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:15 smithi104 ceph-mon[129021]: pgmap v304: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-07-21T22:31:16.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:15 smithi066 ceph-mon[126379]: pgmap v304: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-07-21T22:31:16.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:15 smithi149 ceph-mon[129268]: pgmap v304: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-07-21T22:31:16.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 458693 5.53 MB/sec execute 403 sec latency 70.636 ms 2023-07-21T22:31:17.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:17 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:17.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:17 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:17.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:17 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:17.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 458866 5.53 MB/sec execute 404 sec latency 602.326 ms 2023-07-21T22:31:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:18 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:31:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:18 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:18.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:18 smithi104 ceph-mon[129021]: pgmap v305: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-07-21T22:31:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:18 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:31:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:18 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:18 smithi066 ceph-mon[126379]: pgmap v305: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-07-21T22:31:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:18 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:31:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:18 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:18 smithi149 ceph-mon[129268]: pgmap v305: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2023-07-21T22:31:18.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 460767 5.55 MB/sec execute 405 sec latency 17.272 ms 2023-07-21T22:31:19.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 462372 5.56 MB/sec execute 406 sec latency 11.095 ms 2023-07-21T22:31:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:20 smithi104 ceph-mon[129021]: pgmap v306: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 193 op/s 2023-07-21T22:31:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:20 smithi066 ceph-mon[126379]: pgmap v306: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 193 op/s 2023-07-21T22:31:20.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:20 smithi149 ceph-mon[129268]: pgmap v306: 129 pgs: 129 active+clean; 396 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 193 op/s 2023-07-21T22:31:20.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 463764 5.56 MB/sec execute 407 sec latency 35.189 ms 2023-07-21T22:31:21.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 465740 5.58 MB/sec execute 408 sec latency 14.963 ms 2023-07-21T22:31:22.146 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:31:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:22 smithi104 ceph-mon[129021]: pgmap v307: 129 pgs: 129 active+clean; 405 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2023-07-21T22:31:22.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 -- 172.21.15.66:0/823139964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80728b0 msgr2=0x7f22b8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/823139964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80728b0 0x7f22b8072c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009990 tx=0x7f22a8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 -- 172.21.15.66:0/823139964 shutdown_connections 2023-07-21T22:31:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/823139964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80728b0 0x7f22b8072c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/823139964 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b8110af0 0x7f22b8112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/823139964 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22b80731c0 0x7f22b81105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 -- 172.21.15.66:0/823139964 >> 172.21.15.66:0/823139964 conn(0x7f22b80fb5e0 msgr2=0x7f22b80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 -- 172.21.15.66:0/823139964 shutdown_connections 2023-07-21T22:31:22.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.352+0000 7f22bd5ff700 1 -- 172.21.15.66:0/823139964 wait complete. 2023-07-21T22:31:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.353+0000 7f22bd5ff700 1 Processor -- start 2023-07-21T22:31:22.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 -- start start 2023-07-21T22:31:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 0x7f22b819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80731c0 0x7f22b819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22b8110af0 0x7f22b81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b81a1200 con 0x7f22b8110af0 2023-07-21T22:31:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b81a1370 con 0x7f22b80731c0 2023-07-21T22:31:22.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22bd5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b81a1530 con 0x7f22b80728b0 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22b6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 0x7f22b819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22b6ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 0x7f22b819c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50740/0 (socket says 172.21.15.66:50740) 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22b6ffd700 1 -- 172.21.15.66:0/426527647 learned_addr learned my addr 172.21.15.66:0/426527647 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22b67fc700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80731c0 0x7f22b819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.354+0000 7f22b77fe700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22b8110af0 0x7f22b81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b6ffd700 1 -- 172.21.15.66:0/426527647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80731c0 msgr2=0x7f22b819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b6ffd700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80731c0 0x7f22b819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b6ffd700 1 -- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22b8110af0 msgr2=0x7f22b81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b6ffd700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22b8110af0 0x7f22b81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b6ffd700 1 -- 172.21.15.66:0/426527647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0009c40 con 0x7f22b80728b0 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b67fc700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80731c0 0x7f22b819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22b6ffd700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 0x7f22b819c220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005910 tx=0x7f22a8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f229ffff700 1 -- 172.21.15.66:0/426527647 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a801d070 con 0x7f22b80728b0 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a80095f0 con 0x7f22b80728b0 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f229ffff700 1 -- 172.21.15.66:0/426527647 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22a8022470 con 0x7f22b80728b0 2023-07-21T22:31:22.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f229ffff700 1 -- 172.21.15.66:0/426527647 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8017650 con 0x7f22b80728b0 2023-07-21T22:31:22.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.355+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b81a8e70 con 0x7f22b80728b0 2023-07-21T22:31:22.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.359+0000 7f229ffff700 1 -- 172.21.15.66:0/426527647 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f22a80225e0 con 0x7f22b80728b0 2023-07-21T22:31:22.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 Processor -- start 2023-07-21T22:31:22.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 -- start start 2023-07-21T22:31:22.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267ef6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef6ac0 0x564267ef1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f229ffff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f22a00a12c0 0x7f22a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef1930 0x564267ef1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267df6cb0 con 0x564267ef6ac0 2023-07-21T22:31:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267e49a40 con 0x564267ef1930 2023-07-21T22:31:22.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.361+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267e42ed0 con 0x564267ef61b0 2023-07-21T22:31:22.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267ef6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267ef6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50748/0 (socket says 172.21.15.66:50748) 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 -- 172.21.15.66:0/2125793802 learned_addr learned my addr 172.21.15.66:0/2125793802 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f229ffff700 1 -- 172.21.15.66:0/426527647 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f22a80ce420 con 0x7f22b80728b0 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef1930 0x564267ef1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b6ffd700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f22a00a12c0 0x7f22a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b67fc700 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef6ac0 0x564267ef1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 -- 172.21.15.66:0/2125793802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef1930 msgr2=0x564267ef1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef1930 0x564267ef1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 -- 172.21.15.66:0/2125793802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef6ac0 msgr2=0x564267ef1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef6ac0 0x564267ef1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b77fe700 1 -- 172.21.15.66:0/2125793802 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564267c70de0 con 0x564267ef61b0 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.362+0000 7f22b67fc700 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef6ac0 0x564267ef1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f22b6ffd700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f22a00a12c0 0x7f22a00a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f22a80a3010 tx=0x7f22a8022f90 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267ef6580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009b50 tx=0x7f22b0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d029a7237c99c02 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f229d7fa700 1 -- 172.21.15.66:0/2125793802 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000e070 con 0x564267ef61b0 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f229d7fa700 1 -- 172.21.15.66:0/2125793802 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b000f040 con 0x564267ef61b0 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f229d7fa700 1 -- 172.21.15.66:0/2125793802 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0004340 con 0x564267ef61b0 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2125793802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 msgr2=0x564267ef6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.363+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267ef6580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009b50 tx=0x7f22b0009920 comp rx=0 tx=0).stop 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2125793802 shutdown_connections 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef1930 0x564267ef1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267ef6580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2125793802 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef6ac0 0x564267ef1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2125793802 >> 172.21.15.66:0/2125793802 conn(0x564267ee6ef0 msgr2=0x564267eec830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2125793802 shutdown_connections 2023-07-21T22:31:22.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2125793802 wait complete. 2023-07-21T22:31:22.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 -- start start 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.364+0000 7f22d0e53b80 1 Processor -- start 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267efdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267efae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267efb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267de2ac0 con 0x564267ef1930 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267de27c0 con 0x564267ef6ac0 2023-07-21T22:31:22.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267cc6920 con 0x564267ef61b0 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22d0e53b80 10 client.0 _ll_register_callbacks cb 0x564267de0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22b67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267efae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22b77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267efdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22b67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267efae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50764/0 (socket says 172.21.15.66:50764) 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.365+0000 7f22b67fc700 1 -- 172.21.15.66:0/2414547825 learned_addr learned my addr 172.21.15.66:0/2414547825 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267efb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.417 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 -- 172.21.15.66:0/2414547825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 msgr2=0x564267efae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267efae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 -- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 msgr2=0x564267efb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267efb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 -- 172.21.15.66:0/2414547825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b0008a10 con 0x564267ef1930 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22b77fe700 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267efdc00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22b0006080 tx=0x7f22b00094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be93f1c5cb7f05e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f229aff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f229aff5700 1 -- 172.21.15.66:0/2414547825 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000e040 con 0x564267ef1930 2023-07-21T22:31:22.418 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564267d9c4f0 con 0x564267ef1930 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.366+0000 7f229aff5700 1 -- 172.21.15.66:0/2414547825 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f22b0008030 con 0x564267ef1930 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f229aff5700 1 -- 172.21.15.66:0/2414547825 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22b00081f0 con 0x564267ef1930 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 10 client.16737 fetch_fsmap learned FSMap version 39 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564267c59c60 con 0x564267ef1930 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f229aff5700 1 -- 172.21.15.66:0/2414547825 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564267d9c4f0 con 0x564267ef1930 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f229aff5700 1 -- 172.21.15.66:0/2414547825 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f22b0012050 con 0x564267ef1930 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 10 client.16737 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:31:22.419 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 20 client.16737 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f0b250 0x564267f0d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 4 client.16737 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564267e3beb0 con 0x564267f0b250 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.367+0000 7f22b67fc700 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f0b250 0x564267f0d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.368+0000 7f22b67fc700 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f0b250 0x564267f0d630 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6b7168d01b9d3 server_cookie=16ffef419d6a06ad in_seq=0 out_seq=0 2023-07-21T22:31:22.420 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.368+0000 7f229aff5700 10 client.16737 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f229aff5700 1 -- 172.21.15.66:0/2414547825 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564267e3beb0 con 0x564267f0b250 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f229aff5700 10 client.16737 handle_command_reply: tid=0 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 client.16737 shutdown 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 msgr2=0x564267efdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267efdc00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22b0006080 tx=0x7f22b00094a0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 shutdown_connections 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267efb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f0b250 0x564267f0d630 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267efdc00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/2414547825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267efae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f229aff5700 0 client.16737 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 >> 172.21.15.66:0/2414547825 conn(0x564267ee6ef0 msgr2=0x564267eec3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 shutdown_connections 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 1 -- 172.21.15.66:0/2414547825 wait complete. 2023-07-21T22:31:22.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.369+0000 7f22d0e53b80 20 client.16737 trim_cache size 0 max 16384 2023-07-21T22:31:22.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 Processor -- start 2023-07-21T22:31:22.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 -- start start 2023-07-21T22:31:22.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef1930 0x564267f025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef61b0 0x564267efd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267eec2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267ee9080 con 0x564267ef61b0 2023-07-21T22:31:22.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267efe390 con 0x564267ef6ac0 2023-07-21T22:31:22.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.388+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267efbde0 con 0x564267ef1930 2023-07-21T22:31:22.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267eec2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267eec2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:39740/0 (socket says 172.21.15.66:39740) 2023-07-21T22:31:22.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 -- 172.21.15.66:0/4289868616 learned_addr learned my addr 172.21.15.66:0/4289868616 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:22.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef1930 0x564267f025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b67fc700 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef61b0 0x564267efd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 -- 172.21.15.66:0/4289868616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef1930 msgr2=0x564267f025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef1930 0x564267f025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 -- 172.21.15.66:0/4289868616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef61b0 msgr2=0x564267efd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef61b0 0x564267efd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 -- 172.21.15.66:0/4289868616 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564267c97250 con 0x564267ef6ac0 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.389+0000 7f22b77fe700 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267eec2d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22b000b000 tx=0x7f22b0009430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea4fd7869661e2dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.390+0000 7f229dffb700 1 -- 172.21.15.66:0/4289868616 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000e070 con 0x564267ef6ac0 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.390+0000 7f229dffb700 1 -- 172.21.15.66:0/4289868616 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b001b5a0 con 0x564267ef6ac0 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.390+0000 7f229dffb700 1 -- 172.21.15.66:0/4289868616 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b001bad0 con 0x564267ef6ac0 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 -- 172.21.15.66:0/4289868616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 msgr2=0x564267eec2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267eec2d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f22b000b000 tx=0x7f22b0009430 comp rx=0 tx=0).stop 2023-07-21T22:31:22.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 -- 172.21.15.66:0/4289868616 shutdown_connections 2023-07-21T22:31:22.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267eec2d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef1930 0x564267f025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/4289868616 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef61b0 0x564267efd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 -- 172.21.15.66:0/4289868616 >> 172.21.15.66:0/4289868616 conn(0x564267ee6ef0 msgr2=0x564267eefaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 -- 172.21.15.66:0/4289868616 shutdown_connections 2023-07-21T22:31:22.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.391+0000 7f22d0e53b80 1 -- 172.21.15.66:0/4289868616 wait complete. 2023-07-21T22:31:22.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.392+0000 7f22d0e53b80 1 -- start start 2023-07-21T22:31:22.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.392+0000 7f22d0e53b80 1 Processor -- start 2023-07-21T22:31:22.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267f025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267f00a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267f00f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267f01660 con 0x564267ef1930 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267f017d0 con 0x564267ef6ac0 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564267f5f0b0 con 0x564267ef61b0 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22d0e53b80 10 client.0 _ll_register_callbacks cb 0x7f22b0008670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267f00a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267f00a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50776/0 (socket says 172.21.15.66:50776) 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 -- 172.21.15.66:0/977926967 learned_addr learned my addr 172.21.15.66:0/977926967 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b77fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267f025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b77fe700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267f00f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 -- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 msgr2=0x564267f00f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267f00f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 -- 172.21.15.66:0/977926967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 msgr2=0x564267f025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:22.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267f025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.393+0000 7f22b67fc700 1 -- 172.21.15.66:0/977926967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564267c59c60 con 0x564267ef61b0 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.394+0000 7f22b77fe700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267f025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.394+0000 7f22b67fc700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267f00a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x564267f0dea0 tx=0x7f22ac013320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6d085358c9dfab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.394+0000 7f229aff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.394+0000 7f229aff5700 1 -- 172.21.15.66:0/977926967 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac018040 con 0x564267ef61b0 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.394+0000 7f229aff5700 1 -- 172.21.15.66:0/977926967 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f22ac004e20 con 0x564267ef61b0 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.394+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b000f040 con 0x564267ef61b0 2023-07-21T22:31:22.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f229aff5700 1 -- 172.21.15.66:0/977926967 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ac01db50 con 0x564267ef61b0 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 10 client.26776 fetch_fsmap learned FSMap version 39 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x564267c6f2b0 con 0x564267ef61b0 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f229aff5700 1 -- 172.21.15.66:0/977926967 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22b000f040 con 0x564267ef61b0 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f229aff5700 1 -- 172.21.15.66:0/977926967 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 39) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f22ac013800 con 0x564267ef61b0 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 10 client.26776 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 10 client.26776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 20 client.26776 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:22.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f60640 0x564267f62a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:22.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 4 client.26776 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:31:22.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564267dfe050 con 0x564267f60640 2023-07-21T22:31:22.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.395+0000 7f22b77fe700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f60640 0x564267f62a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.396+0000 7f22b77fe700 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f60640 0x564267f62a50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ad233e2b1b01d server_cookie=d411b5a25d14543 in_seq=0 out_seq=0 2023-07-21T22:31:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.396+0000 7f229aff5700 10 client.26776 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.396+0000 7f229aff5700 1 -- 172.21.15.66:0/977926967 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564267dfe050 con 0x564267f60640 2023-07-21T22:31:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.396+0000 7f229aff5700 10 client.26776 handle_command_reply: tid=0 2023-07-21T22:31:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.396+0000 7f22d0e53b80 1 client.26776 shutdown 2023-07-21T22:31:22.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 msgr2=0x564267f00a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267f00a10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x564267f0dea0 tx=0x7f22ac013320 comp rx=0 tx=0).stop 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 shutdown_connections 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564267ef6ac0 0x564267f00f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x564267f60640 0x564267f62a50 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x564267ef1930 0x564267f025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 --2- 172.21.15.66:0/977926967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564267ef61b0 0x564267f00a10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f229aff5700 0 client.26776 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 >> 172.21.15.66:0/977926967 conn(0x564267ee6ef0 msgr2=0x564267eec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 shutdown_connections 2023-07-21T22:31:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 1 -- 172.21.15.66:0/977926967 wait complete. 2023-07-21T22:31:22.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.397+0000 7f22d0e53b80 20 client.26776 trim_cache size 0 max 16384 2023-07-21T22:31:22.449 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:31:22.449 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: "4b68d0c7-b791-47de-84f5-c4e8b466bda2": { 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "4b68d0c7-b791-47de-84f5-c4e8b466bda2", 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:31:22.450 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:31:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f22a00a12c0 msgr2=0x7f22a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f22a00a12c0 0x7f22a00a3770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f22a80a3010 tx=0x7f22a8022f90 comp rx=0 tx=0).stop 2023-07-21T22:31:22.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 msgr2=0x7f22b819c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 0x7f22b819c220 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22a8005910 tx=0x7f22a8005720 comp rx=0 tx=0).stop 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 shutdown_connections 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f22a00a12c0 0x7f22a00a3770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f22b8110af0 0x7f22b81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22b80728b0 0x7f22b819c220 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 --2- 172.21.15.66:0/426527647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22b80731c0 0x7f22b819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:22.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 >> 172.21.15.66:0/426527647 conn(0x7f22b80fb5e0 msgr2=0x7f22b80fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:22.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 shutdown_connections 2023-07-21T22:31:22.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:22.398+0000 7f22bd5ff700 1 -- 172.21.15.66:0/426527647 wait complete. 2023-07-21T22:31:22.453 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:22 smithi066 ceph-mon[126379]: pgmap v307: 129 pgs: 129 active+clean; 405 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2023-07-21T22:31:22.463 INFO:tasks.cephfs.filesystem:scrub status for tag:4b68d0c7-b791-47de-84f5-c4e8b466bda2 - {'path': '/', 'tag': '4b68d0c7-b791-47de-84f5-c4e8b466bda2', 'options': 'recursive,force'} 2023-07-21T22:31:22.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:22 smithi149 ceph-mon[129268]: pgmap v307: 129 pgs: 129 active+clean; 405 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2023-07-21T22:31:22.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 466624 5.57 MB/sec execute 409 sec latency 48.124 ms 2023-07-21T22:31:23.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 468839 5.59 MB/sec execute 410 sec latency 21.907 ms 2023-07-21T22:31:24.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:24 smithi104 ceph-mon[129021]: pgmap v308: 129 pgs: 129 active+clean; 405 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 175 op/s 2023-07-21T22:31:24.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:24 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:31:24.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:24 smithi066 ceph-mon[126379]: pgmap v308: 129 pgs: 129 active+clean; 405 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 175 op/s 2023-07-21T22:31:24.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:24 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:31:24.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:24 smithi149 ceph-mon[129268]: pgmap v308: 129 pgs: 129 active+clean; 405 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.9 MiB/s wr, 175 op/s 2023-07-21T22:31:24.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:24 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:31:24.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 469966 5.59 MB/sec execute 411 sec latency 11.659 ms 2023-07-21T22:31:25.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 472203 5.61 MB/sec execute 412 sec latency 16.432 ms 2023-07-21T22:31:26.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:26 smithi104 ceph-mon[129021]: pgmap v309: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2023-07-21T22:31:26.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:26 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:31:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:26 smithi066 ceph-mon[126379]: pgmap v309: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2023-07-21T22:31:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:26 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:31:26.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:26 smithi149 ceph-mon[129268]: pgmap v309: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2023-07-21T22:31:26.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:26 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:31:26.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 473277 5.61 MB/sec execute 413 sec latency 21.260 ms 2023-07-21T22:31:27.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:27 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:31:27.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:27 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:31:27.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:27 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:31:27.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:27 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:31:27.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:27 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:31:27.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:27 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:31:27.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 474956 5.61 MB/sec execute 414 sec latency 48.623 ms 2023-07-21T22:31:28.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:28 smithi104 ceph-mon[129021]: pgmap v310: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:31:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:28 smithi066 ceph-mon[126379]: pgmap v310: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:31:28.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:28 smithi149 ceph-mon[129268]: pgmap v310: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:31:28.560 INFO:tasks.workunit.client.0.smithi066.stdout: 1 476395 5.62 MB/sec execute 415 sec latency 201.947 ms 2023-07-21T22:31:29.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 477737 5.62 MB/sec execute 416 sec latency 12.578 ms 2023-07-21T22:31:30.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:30 smithi104 ceph-mon[129021]: pgmap v311: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:31:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:30 smithi066 ceph-mon[126379]: pgmap v311: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:31:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:30 smithi149 ceph-mon[129268]: pgmap v311: 129 pgs: 129 active+clean; 414 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 179 op/s 2023-07-21T22:31:30.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 479905 5.64 MB/sec execute 417 sec latency 58.034 ms 2023-07-21T22:31:31.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 481120 5.64 MB/sec execute 418 sec latency 12.109 ms 2023-07-21T22:31:32.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:32 smithi104 ceph-mon[129021]: pgmap v312: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-07-21T22:31:32.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:32 smithi066 ceph-mon[126379]: pgmap v312: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-07-21T22:31:32.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:32 smithi149 ceph-mon[129268]: pgmap v312: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 268 op/s 2023-07-21T22:31:32.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 483278 5.66 MB/sec execute 419 sec latency 29.857 ms 2023-07-21T22:31:33.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 484197 5.65 MB/sec execute 420 sec latency 70.388 ms 2023-07-21T22:31:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:34 smithi104 ceph-mon[129021]: pgmap v313: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 182 op/s 2023-07-21T22:31:34.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:34 smithi066 ceph-mon[126379]: pgmap v313: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 182 op/s 2023-07-21T22:31:34.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:34 smithi149 ceph-mon[129268]: pgmap v313: 129 pgs: 129 active+clean; 421 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 182 op/s 2023-07-21T22:31:34.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 486424 5.67 MB/sec execute 421 sec latency 15.674 ms 2023-07-21T22:31:35.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 487561 5.67 MB/sec execute 422 sec latency 11.453 ms 2023-07-21T22:31:36.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:36 smithi104 ceph-mon[129021]: pgmap v314: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 270 op/s 2023-07-21T22:31:36.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:36 smithi066 ceph-mon[126379]: pgmap v314: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 270 op/s 2023-07-21T22:31:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:36 smithi149 ceph-mon[129268]: pgmap v314: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 270 op/s 2023-07-21T22:31:36.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489163 5.67 MB/sec execute 423 sec latency 165.577 ms 2023-07-21T22:31:37.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:37 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:31:37.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:37 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:31:37.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:37 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:31:37.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489163 5.66 MB/sec execute 424 sec latency 1165.693 ms 2023-07-21T22:31:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:38 smithi104 ceph-mon[129021]: pgmap v315: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-21T22:31:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:38 smithi066 ceph-mon[126379]: pgmap v315: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-21T22:31:38.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:38 smithi149 ceph-mon[129268]: pgmap v315: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-21T22:31:38.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489163 5.64 MB/sec execute 425 sec latency 2165.833 ms 2023-07-21T22:31:39.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:39 smithi066 ceph-mon[126379]: pgmap v316: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-21T22:31:39.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:39 smithi149 ceph-mon[129268]: pgmap v316: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-21T22:31:39.561 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489163 5.63 MB/sec execute 426 sec latency 3165.959 ms 2023-07-21T22:31:39.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:39 smithi104 ceph-mon[129021]: pgmap v316: 129 pgs: 129 active+clean; 430 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 177 op/s 2023-07-21T22:31:40.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489203 5.62 MB/sec execute 427 sec latency 3852.139 ms 2023-07-21T22:31:41.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489203 5.61 MB/sec execute 428 sec latency 1267.637 ms 2023-07-21T22:31:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:41 smithi104 ceph-mon[129021]: pgmap v317: 129 pgs: 129 active+clean; 437 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:31:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:41 smithi066 ceph-mon[126379]: pgmap v317: 129 pgs: 129 active+clean; 437 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:31:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:41 smithi149 ceph-mon[129268]: pgmap v317: 129 pgs: 129 active+clean; 437 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 219 op/s 2023-07-21T22:31:42.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489203 5.59 MB/sec execute 429 sec latency 2267.814 ms 2023-07-21T22:31:43.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489203 5.58 MB/sec execute 430 sec latency 3267.920 ms 2023-07-21T22:31:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:43 smithi104 ceph-mon[129021]: pgmap v318: 129 pgs: 129 active+clean; 437 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-21T22:31:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:43 smithi066 ceph-mon[126379]: pgmap v318: 129 pgs: 129 active+clean; 437 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-21T22:31:44.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:43 smithi149 ceph-mon[129268]: pgmap v318: 129 pgs: 129 active+clean; 437 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 130 op/s 2023-07-21T22:31:44.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489203 5.57 MB/sec execute 431 sec latency 4268.107 ms 2023-07-21T22:31:45.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489320 5.56 MB/sec execute 432 sec latency 4899.097 ms 2023-07-21T22:31:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:45 smithi104 ceph-mon[129021]: pgmap v319: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 161 op/s 2023-07-21T22:31:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:45 smithi066 ceph-mon[126379]: pgmap v319: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 161 op/s 2023-07-21T22:31:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:45 smithi149 ceph-mon[129268]: pgmap v319: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 161 op/s 2023-07-21T22:31:46.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489320 5.54 MB/sec execute 433 sec latency 1282.847 ms 2023-07-21T22:31:47.562 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489320 5.53 MB/sec execute 434 sec latency 2282.937 ms 2023-07-21T22:31:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:47 smithi104 ceph-mon[129021]: pgmap v320: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2023-07-21T22:31:48.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:47 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:47 smithi066 ceph-mon[126379]: pgmap v320: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2023-07-21T22:31:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:47 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:47 smithi149 ceph-mon[129268]: pgmap v320: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2023-07-21T22:31:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:47 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:31:48.563 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489320 5.52 MB/sec execute 435 sec latency 3283.022 ms 2023-07-21T22:31:49.563 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489320 5.50 MB/sec execute 436 sec latency 4283.138 ms 2023-07-21T22:31:50.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:49 smithi104 ceph-mon[129021]: pgmap v321: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2023-07-21T22:31:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:49 smithi066 ceph-mon[126379]: pgmap v321: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2023-07-21T22:31:50.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:49 smithi149 ceph-mon[129268]: pgmap v321: 129 pgs: 129 active+clean; 433 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2023-07-21T22:31:50.563 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489357 5.49 MB/sec execute 437 sec latency 4919.627 ms 2023-07-21T22:31:51.563 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489357 5.48 MB/sec execute 438 sec latency 1335.660 ms 2023-07-21T22:31:51.947 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: pgmap v322: 129 pgs: 129 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 3.4 MiB/s wr, 95 op/s 2023-07-21T22:31:51.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.949 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.949 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.949 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.949 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.949 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:51 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: pgmap v322: 129 pgs: 129 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 3.4 MiB/s wr, 95 op/s 2023-07-21T22:31:51.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:51.993 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:51.994 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:51 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: pgmap v322: 129 pgs: 129 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 3.4 MiB/s wr, 95 op/s 2023-07-21T22:31:52.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:51 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.464 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:31:52.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489357 5.47 MB/sec execute 439 sec latency 2336.449 ms 2023-07-21T22:31:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 -- 172.21.15.66:0/3770355712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d40ffad0 msgr2=0x7f88d40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 --2- 172.21.15.66:0/3770355712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d40ffad0 0x7f88d40fff40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009990 tx=0x7f88c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 -- 172.21.15.66:0/3770355712 shutdown_connections 2023-07-21T22:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 --2- 172.21.15.66:0/3770355712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88d4100480 0x7f88d410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 --2- 172.21.15.66:0/3770355712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d4105ae0 0x7f88d4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 --2- 172.21.15.66:0/3770355712 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d40ffad0 0x7f88d40fff40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 -- 172.21.15.66:0/3770355712 >> 172.21.15.66:0/3770355712 conn(0x7f88d40fb5f0 msgr2=0x7f88d40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 -- 172.21.15.66:0/3770355712 shutdown_connections 2023-07-21T22:31:52.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.682+0000 7f88da9ff700 1 -- 172.21.15.66:0/3770355712 wait complete. 2023-07-21T22:31:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.684+0000 7f88da9ff700 1 Processor -- start 2023-07-21T22:31:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.684+0000 7f88da9ff700 1 -- start start 2023-07-21T22:31:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88da9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 0x7f88d419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88da9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d4100480 0x7f88d419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88da9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88d4105ae0 0x7f88d4195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88da9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4196080 con 0x7f88d4100480 2023-07-21T22:31:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88da9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d41961f0 con 0x7f88d4105ae0 2023-07-21T22:31:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88da9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d41963b0 con 0x7f88d40ffad0 2023-07-21T22:31:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 0x7f88d419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 0x7f88d419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37796/0 (socket says 172.21.15.66:37796) 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 -- 172.21.15.66:0/620253287 learned_addr learned my addr 172.21.15.66:0/620253287 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d37fe700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d4100480 0x7f88d419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d8f9c700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88d4105ae0 0x7f88d4195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 -- 172.21.15.66:0/620253287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88d4105ae0 msgr2=0x7f88d4195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88d4105ae0 0x7f88d4195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 -- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d4100480 msgr2=0x7f88d419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.685+0000 7f88d3fff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d4100480 0x7f88d419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88d3fff700 1 -- 172.21.15.66:0/620253287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c4009520 con 0x7f88d40ffad0 2023-07-21T22:31:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88d3fff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 0x7f88d419b9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88c400eb30 tx=0x7f88c400ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88d17fa700 1 -- 172.21.15.66:0/620253287 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4009880 con 0x7f88d40ffad0 2023-07-21T22:31:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88d17fa700 1 -- 172.21.15.66:0/620253287 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c40044e0 con 0x7f88d40ffad0 2023-07-21T22:31:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88d17fa700 1 -- 172.21.15.66:0/620253287 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4003d80 con 0x7f88d40ffad0 2023-07-21T22:31:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c80095f0 con 0x7f88d40ffad0 2023-07-21T22:31:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.686+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d41a27b0 con 0x7f88d40ffad0 2023-07-21T22:31:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.691+0000 7f88d17fa700 1 -- 172.21.15.66:0/620253287 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f88c4003f00 con 0x7f88d40ffad0 2023-07-21T22:31:52.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.692+0000 7f88ee229b80 1 Processor -- start 2023-07-21T22:31:52.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.692+0000 7f88ee229b80 1 -- start start 2023-07-21T22:31:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448ca1b0 0x55ba448ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448c5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88ee229b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba447cacb0 con 0x55ba448caac0 2023-07-21T22:31:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88ee229b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4481da40 con 0x55ba448c5930 2023-07-21T22:31:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88ee229b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba44816ed0 con 0x55ba448ca1b0 2023-07-21T22:31:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d17fa700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88bc0a1340 0x7f88bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48600/0 (socket says 172.21.15.66:48600) 2023-07-21T22:31:52.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d3fff700 1 -- 172.21.15.66:0/3837303268 learned_addr learned my addr 172.21.15.66:0/3837303268 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d8f9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448ca1b0 0x55ba448ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d17fa700 1 -- 172.21.15.66:0/620253287 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88c4014070 con 0x7f88d40ffad0 2023-07-21T22:31:52.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.693+0000 7f88d8f9c700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88bc0a1340 0x7f88bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d37fe700 1 -- 172.21.15.66:0/3837303268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448ca1b0 msgr2=0x55ba448ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d37fe700 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448ca1b0 0x55ba448ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d37fe700 1 -- 172.21.15.66:0/3837303268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 msgr2=0x55ba448c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d37fe700 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d37fe700 1 -- 172.21.15.66:0/3837303268 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba44644de0 con 0x55ba448c5930 2023-07-21T22:31:52.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d3fff700 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d37fe700 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448c5da0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88c800b2e0 tx=0x7f88c8005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913461b44725fc2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88d8f9c700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88bc0a1340 0x7f88bc0a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009ec0 tx=0x7f88cc00b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88ba7fc700 1 -- 172.21.15.66:0/3837303268 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c801d070 con 0x55ba448c5930 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88ba7fc700 1 -- 172.21.15.66:0/3837303268 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c801e040 con 0x55ba448c5930 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.694+0000 7f88ba7fc700 1 -- 172.21.15.66:0/3837303268 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8004880 con 0x55ba448c5930 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 -- 172.21.15.66:0/3837303268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 msgr2=0x55ba448c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448c5da0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88c800b2e0 tx=0x7f88c8005ea0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 -- 172.21.15.66:0/3837303268 shutdown_connections 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448c5da0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448ca1b0 0x55ba448ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 --2- 172.21.15.66:0/3837303268 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 -- 172.21.15.66:0/3837303268 >> 172.21.15.66:0/3837303268 conn(0x55ba448baef0 msgr2=0x55ba448c0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 -- 172.21.15.66:0/3837303268 shutdown_connections 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.695+0000 7f88ee229b80 1 -- 172.21.15.66:0/3837303268 wait complete. 2023-07-21T22:31:52.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.696+0000 7f88ee229b80 1 -- start start 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.696+0000 7f88ee229b80 1 Processor -- start 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.696+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448d1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.696+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 0x55ba448cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448caac0 0x55ba448cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88ee229b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba447b6ac0 con 0x55ba448ca1b0 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88ee229b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba447b67c0 con 0x55ba448c5930 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88ee229b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba4469a920 con 0x55ba448caac0 2023-07-21T22:31:52.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88ee229b80 10 client.0 _ll_register_callbacks cb 0x55ba447b4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 0x55ba448cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448d1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 0x55ba448cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48614/0 (socket says 172.21.15.66:48614) 2023-07-21T22:31:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 -- 172.21.15.66:0/1810573759 learned_addr learned my addr 172.21.15.66:0/1810573759 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:52.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d8f9c700 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448caac0 0x55ba448cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 -- 172.21.15.66:0/1810573759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448caac0 msgr2=0x55ba448cf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448caac0 0x55ba448cf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 -- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 msgr2=0x55ba448d1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448d1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.697+0000 7f88d37fe700 1 -- 172.21.15.66:0/1810573759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d41a27b0 con 0x55ba448ca1b0 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88d37fe700 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 0x55ba448cee30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f88c80098e0 tx=0x7f88c8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563e43c24aad7cc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88b37fe700 1 -- 172.21.15.66:0/1810573759 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c801d040 con 0x55ba448ca1b0 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88b37fe700 1 -- 172.21.15.66:0/1810573759 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88c8005240 con 0x55ba448ca1b0 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba447704f0 con 0x55ba448ca1b0 2023-07-21T22:31:52.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88b37fe700 1 -- 172.21.15.66:0/1810573759 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c8005bc0 con 0x55ba448ca1b0 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88ee229b80 10 client.16755 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f88d410d210 con 0x55ba448ca1b0 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.698+0000 7f88b37fe700 1 -- 172.21.15.66:0/1810573759 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba447704f0 con 0x55ba448ca1b0 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88b37fe700 1 -- 172.21.15.66:0/1810573759 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f88c8004940 con 0x55ba448ca1b0 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88ee229b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88ee229b80 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88ee229b80 20 client.16755 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:52.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba448deef0 0x55ba448e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88ee229b80 4 client.16755 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba4480feb0 con 0x55ba448deef0 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.699+0000 7f88d3fff700 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba448deef0 0x55ba448e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.700+0000 7f88d3fff700 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba448deef0 0x55ba448e12d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e8aca4279c87b3 server_cookie=246573526f5984e in_seq=0 out_seq=0 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.700+0000 7f88b37fe700 10 client.16755 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.700+0000 7f88b37fe700 1 -- 172.21.15.66:0/1810573759 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba4480feb0 con 0x55ba448deef0 2023-07-21T22:31:52.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.700+0000 7f88b37fe700 10 client.16755 handle_command_reply: tid=0 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 client.16755 shutdown 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 msgr2=0x55ba448cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 0x55ba448cee30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f88c80098e0 tx=0x7f88c8005f10 comp rx=0 tx=0).stop 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 shutdown_connections 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448c5930 0x55ba448d1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba448deef0 0x55ba448e12d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448caac0 0x55ba448cf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1810573759 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448ca1b0 0x55ba448cee30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88b37fe700 0 client.16755 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 >> 172.21.15.66:0/1810573759 conn(0x55ba448baef0 msgr2=0x55ba448c03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 shutdown_connections 2023-07-21T22:31:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 1 -- 172.21.15.66:0/1810573759 wait complete. 2023-07-21T22:31:52.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.701+0000 7f88ee229b80 20 client.16755 trim_cache size 0 max 16384 2023-07-21T22:31:52.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.718+0000 7f88ee229b80 1 Processor -- start 2023-07-21T22:31:52.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 -- start start 2023-07-21T22:31:52.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448ca1b0 0x55ba448ce610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba447704f0 con 0x55ba448caac0 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba448d2120 con 0x55ba448ca1b0 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88ee229b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba448c1ca0 con 0x55ba448c5930 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88d8f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37822/0 (socket says 172.21.15.66:37822) 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88d3fff700 1 -- 172.21.15.66:0/2085237167 learned_addr learned my addr 172.21.15.66:0/2085237167 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.719+0000 7f88d37fe700 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448ca1b0 0x55ba448ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88d37fe700 1 -- 172.21.15.66:0/2085237167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 msgr2=0x55ba448d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88d37fe700 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88d37fe700 1 -- 172.21.15.66:0/2085237167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 msgr2=0x55ba448ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88d37fe700 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88d37fe700 1 -- 172.21.15.66:0/2085237167 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4462dc60 con 0x55ba448ca1b0 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88d37fe700 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448ca1b0 0x55ba448ce610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88c8005780 tx=0x7f88c8004fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd783a448ac9eaf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88baffd700 1 -- 172.21.15.66:0/2085237167 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800e3e0 con 0x55ba448ca1b0 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88baffd700 1 -- 172.21.15.66:0/2085237167 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c8004500 con 0x55ba448ca1b0 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.720+0000 7f88baffd700 1 -- 172.21.15.66:0/2085237167 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800faf0 con 0x55ba448ca1b0 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 -- 172.21.15.66:0/2085237167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448ca1b0 msgr2=0x55ba448ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448ca1b0 0x55ba448ce610 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88c8005780 tx=0x7f88c8004fc0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 -- 172.21.15.66:0/2085237167 shutdown_connections 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 --2- 172.21.15.66:0/2085237167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448ca1b0 0x55ba448ce610 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 -- 172.21.15.66:0/2085237167 >> 172.21.15.66:0/2085237167 conn(0x55ba448baef0 msgr2=0x55ba448bef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 -- 172.21.15.66:0/2085237167 shutdown_connections 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 -- 172.21.15.66:0/2085237167 wait complete. 2023-07-21T22:31:52.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.721+0000 7f88ee229b80 1 -- start start 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 Processor -- start 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448d4c00 0x55ba448d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba44932d30 con 0x55ba448caac0 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba44932e70 con 0x55ba448d4c00 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba44932fb0 con 0x55ba448c5930 2023-07-21T22:31:52.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88ee229b80 10 client.0 _ll_register_callbacks cb 0x7f88c4003720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37826/0 (socket says 172.21.15.66:37826) 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88d3fff700 1 -- 172.21.15.66:0/1864101892 learned_addr learned my addr 172.21.15.66:0/1864101892 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88d37fe700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.722+0000 7f88d8f9c700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448d4c00 0x55ba448d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88d3fff700 1 -- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448d4c00 msgr2=0x55ba448d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88d3fff700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448d4c00 0x55ba448d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88d3fff700 1 -- 172.21.15.66:0/1864101892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 msgr2=0x55ba448d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88d3fff700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448d46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88d3fff700 1 -- 172.21.15.66:0/1864101892 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d410d210 con 0x55ba448c5930 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88d3fff700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d6490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f88c40037d0 tx=0x7f88c40cd700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd941805551d35dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88b37fe700 1 -- 172.21.15.66:0/1864101892 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c409e270 con 0x55ba448c5930 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88c801e040 con 0x55ba448c5930 2023-07-21T22:31:52.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.723+0000 7f88b37fe700 1 -- 172.21.15.66:0/1864101892 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f88c40a2690 con 0x55ba448c5930 2023-07-21T22:31:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88b37fe700 1 -- 172.21.15.66:0/1864101892 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c400cdd0 con 0x55ba448c5930 2023-07-21T22:31:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 10 client.26794 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ba4466b250 con 0x55ba448c5930 2023-07-21T22:31:52.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88b37fe700 1 -- 172.21.15.66:0/1864101892 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c801e040 con 0x55ba448c5930 2023-07-21T22:31:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88b37fe700 1 -- 172.21.15.66:0/1864101892 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f88c4099730 con 0x55ba448c5930 2023-07-21T22:31:52.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 10 client.26794 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 10 client.26794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 20 client.26794 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba44934570 0x55ba44936980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 4 client.26794 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.724+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba447d2050 con 0x55ba44934570 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88d37fe700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba44934570 0x55ba44936980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:52.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88d37fe700 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba44934570 0x55ba44936980 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d237c4834f07da server_cookie=592e636204454267 in_seq=0 out_seq=0 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88b37fe700 10 client.26794 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88b37fe700 1 -- 172.21.15.66:0/1864101892 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ba447d2050 con 0x55ba44934570 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88b37fe700 10 client.26794 handle_command_reply: tid=0 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 client.26794 shutdown 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 msgr2=0x55ba448d6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d6490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f88c40037d0 tx=0x7f88c40cd700 comp rx=0 tx=0).stop 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 shutdown_connections 2023-07-21T22:31:52.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ba448d4c00 0x55ba448d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55ba44934570 0x55ba44936980 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ba448c5930 0x55ba448d6490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 --2- 172.21.15.66:0/1864101892 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ba448caac0 0x55ba448d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88b37fe700 0 client.26794 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.725+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 >> 172.21.15.66:0/1864101892 conn(0x55ba448baef0 msgr2=0x55ba448c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 shutdown_connections 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88ee229b80 1 -- 172.21.15.66:0/1864101892 wait complete. 2023-07-21T22:31:52.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88ee229b80 20 client.26794 trim_cache size 0 max 16384 2023-07-21T22:31:52.780 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:31:52.780 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:31:52.780 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:31:52.780 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:31:52.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88bc0a1340 msgr2=0x7f88bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88bc0a1340 0x7f88bc0a37f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009ec0 tx=0x7f88cc00b040 comp rx=0 tx=0).stop 2023-07-21T22:31:52.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 msgr2=0x7f88d419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:52.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 0x7f88d419b9a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f88c400eb30 tx=0x7f88c400ee40 comp rx=0 tx=0).stop 2023-07-21T22:31:52.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 shutdown_connections 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88d4105ae0 0x7f88d4195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f88bc0a1340 0x7f88bc0a37f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88d40ffad0 0x7f88d419b9a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 --2- 172.21.15.66:0/620253287 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f88d4100480 0x7f88d419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 >> 172.21.15.66:0/620253287 conn(0x7f88d40fb5f0 msgr2=0x7f88d40fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.726+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 shutdown_connections 2023-07-21T22:31:52.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.727+0000 7f88da9ff700 1 -- 172.21.15.66:0/620253287 wait complete. 2023-07-21T22:31:52.790 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:31:52.791 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:31:52.852 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi104 to 196.2M 2023-07-21T22:31:52.853 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: Unable to set osd_memory_target on smithi104 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-21T22:31:52.853 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.853 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.853 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.853 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.854 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:52.854 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:52.854 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:52 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:52.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi104 to 196.2M 2023-07-21T22:31:52.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: Unable to set osd_memory_target on smithi104 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-21T22:31:52.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.991 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:52.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:52.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:52.992 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:52 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:53.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.955+0000 7fc3967ff700 1 -- 172.21.15.66:0/1255952303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3900ffad0 msgr2=0x7fc3900fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.955+0000 7fc3967ff700 1 --2- 172.21.15.66:0/1255952303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3900ffad0 0x7fc3900fff40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc384009990 tx=0x7fc384009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 -- 172.21.15.66:0/1255952303 shutdown_connections 2023-07-21T22:31:53.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 --2- 172.21.15.66:0/1255952303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390105ae0 0x7fc390105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 --2- 172.21.15.66:0/1255952303 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc390100480 0x7fc39010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 --2- 172.21.15.66:0/1255952303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3900ffad0 0x7fc3900fff40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 -- 172.21.15.66:0/1255952303 >> 172.21.15.66:0/1255952303 conn(0x7fc3900fb590 msgr2=0x7fc3900fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 -- 172.21.15.66:0/1255952303 shutdown_connections 2023-07-21T22:31:53.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.956+0000 7fc3967ff700 1 -- 172.21.15.66:0/1255952303 wait complete. 2023-07-21T22:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 Processor -- start 2023-07-21T22:31:53.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 -- start start 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3900ffad0 0x7fc390072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390100480 0x7fc390072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc390105ae0 0x7fc3901a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390073580 con 0x7fc3900ffad0 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3901a42d0 con 0x7fc390100480 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc3967ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3901a4410 con 0x7fc390105ae0 2023-07-21T22:31:53.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.957+0000 7fc38ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3900ffad0 0x7fc390072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390100480 0x7fc390072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc390105ae0 0x7fc3901a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc38f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390100480 0x7fc390072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34362/0 (socket says 172.21.15.66:34362) 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc38f7fe700 1 -- 172.21.15.66:0/2177238700 learned_addr learned my addr 172.21.15.66:0/2177238700 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 -- 172.21.15.66:0/2177238700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390100480 msgr2=0x7fc390072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390100480 0x7fc390072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 -- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3900ffad0 msgr2=0x7fc390072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3900ffad0 0x7fc390072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 -- 172.21.15.66:0/2177238700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc388009c40 con 0x7fc390105ae0 2023-07-21T22:31:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc394d9c700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc390105ae0 0x7fc3901a3d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc38800efb0 tx=0x7fc38800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388003a00 con 0x7fc390105ae0 2023-07-21T22:31:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.958+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc38800f040 con 0x7fc390105ae0 2023-07-21T22:31:53.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.959+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3840095f0 con 0x7fc390105ae0 2023-07-21T22:31:53.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.959+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc388004540 con 0x7fc390105ae0 2023-07-21T22:31:53.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.959+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3901a4a10 con 0x7fc390105ae0 2023-07-21T22:31:53.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.960+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc390067300 con 0x7fc390105ae0 2023-07-21T22:31:53.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.963+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc388003bf0 con 0x7fc390105ae0 2023-07-21T22:31:53.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.964+0000 7fc38d7fa700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc3780a1380 0x7fc3780a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.965+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc3880ccc90 con 0x7fc390105ae0 2023-07-21T22:31:53.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.965+0000 7fc38ffff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc3780a1380 0x7fc3780a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.966+0000 7fc38ffff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc3780a1380 0x7fc3780a3830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc380009be0 tx=0x7fc380009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:52.971+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc388095d30 con 0x7fc390105ae0 2023-07-21T22:31:53.120 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi104 to 196.2M 2023-07-21T22:31:53.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: Unable to set osd_memory_target on smithi104 to 205766758: error parsing value: Value '205766758' is below minimum 939524096 2023-07-21T22:31:53.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:53.121 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:53.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:53.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T22:31:53.122 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:31:53.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:31:53.123 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:52 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:31:53.285 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.239+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3901a4c30 con 0x7fc390105ae0 2023-07-21T22:31:53.286 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.240+0000 7fc38d7fa700 1 -- 172.21.15.66:0/2177238700 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fc388099350 con 0x7fc390105ae0 2023-07-21T22:31:53.286 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:31:53.287 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:31:36.037177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:31:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc3780a1380 msgr2=0x7fc3780a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc3780a1380 0x7fc3780a3830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc380009be0 tx=0x7fc380009260 comp rx=0 tx=0).stop 2023-07-21T22:31:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc390105ae0 msgr2=0x7fc3901a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc390105ae0 0x7fc3901a3d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc38800efb0 tx=0x7fc38800c3a0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 shutdown_connections 2023-07-21T22:31:53.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc3780a1380 0x7fc3780a3830 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc390105ae0 0x7fc3901a3d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc3900ffad0 0x7fc390072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 --2- 172.21.15.66:0/2177238700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc390100480 0x7fc390072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 >> 172.21.15.66:0/2177238700 conn(0x7fc3900fb590 msgr2=0x7fc3901035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 shutdown_connections 2023-07-21T22:31:53.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.243+0000 7fc3967ff700 1 -- 172.21.15.66:0/2177238700 wait complete. 2023-07-21T22:31:53.291 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 41 2023-07-21T22:31:53.308 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:31:53.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 -- 172.21.15.66:0/998161374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea40ff9d0 msgr2=0x7f8ea40ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/998161374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea40ffe40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009990 tx=0x7f8e98009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 -- 172.21.15.66:0/998161374 shutdown_connections 2023-07-21T22:31:53.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/998161374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea41059e0 0x7f8ea4105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/998161374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea4100380 0x7f8ea410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/998161374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea40ffe40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 -- 172.21.15.66:0/998161374 >> 172.21.15.66:0/998161374 conn(0x7f8ea40fb4b0 msgr2=0x7f8ea40fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 -- 172.21.15.66:0/998161374 shutdown_connections 2023-07-21T22:31:53.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.474+0000 7f8ea3fff700 1 -- 172.21.15.66:0/998161374 wait complete. 2023-07-21T22:31:53.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.475+0000 7f8ea3fff700 1 Processor -- start 2023-07-21T22:31:53.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.475+0000 7f8ea3fff700 1 -- start start 2023-07-21T22:31:53.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.475+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.475+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea4100380 0x7f8ea419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea41059e0 0x7f8ea4197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea3fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4197fe0 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea3fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4198150 con 0x7f8ea4100380 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea3fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4198310 con 0x7f8ea41059e0 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea41059e0 0x7f8ea4197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea419d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48656/0 (socket says 172.21.15.66:48656) 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/150914705 learned_addr learned my addr 172.21.15.66:0/150914705 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea4100380 0x7f8ea419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/150914705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea41059e0 msgr2=0x7f8ea4197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea41059e0 0x7f8ea4197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/150914705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea4100380 msgr2=0x7f8ea419de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea4100380 0x7f8ea419de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/150914705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980095f0 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.476+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea41059e0 0x7f8ea4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:53.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.477+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea419d900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e970 tx=0x7f8e9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.477+0000 7f8e8bfff700 1 -- 172.21.15.66:0/150914705 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94009790 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.477+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea41985f0 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.477+0000 7f8e8bfff700 1 -- 172.21.15.66:0/150914705 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e9400c7e0 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.477+0000 7f8e8bfff700 1 -- 172.21.15.66:0/150914705 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94003c70 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.477+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41a48f0 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.480+0000 7f8e8bfff700 1 -- 172.21.15.66:0/150914705 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8e9400cc50 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.481+0000 7f8ebcd2ab80 1 Processor -- start 2023-07-21T22:31:53.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.481+0000 7f8ebcd2ab80 1 -- start start 2023-07-21T22:31:53.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41e99860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41ea7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41d9fcb0 con 0x563f41e99490 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41df2a40 con 0x563f41e99da0 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41debed0 con 0x563f41ea4c40 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8e8bfff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8e8c0a1360 0x7f8e8c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41e99860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ea27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41ea7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ea27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41ea7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37888/0 (socket says 172.21.15.66:37888) 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2942035641 learned_addr learned my addr 172.21.15.66:0/2942035641 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8e8bfff700 1 -- 172.21.15.66:0/150914705 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8e94014070 con 0x7f8ea40ff9d0 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.482+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8e8c0a1360 0x7f8e8c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2942035641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 msgr2=0x563f41ea4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2942035641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 msgr2=0x563f41e99860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41e99860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2942035641 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f41c19de0 con 0x563f41ea4c40 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41e99860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41ea7020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e98015000 tx=0x7f8e98005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba5558cc891f1cef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8e8c0a1360 0x7f8e8c0a3810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c003fd0 tx=0x7f8e9c004b40 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8e897fa700 1 -- 172.21.15.66:0/2942035641 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801d070 con 0x563f41ea4c40 2023-07-21T22:31:53.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8e897fa700 1 -- 172.21.15.66:0/2942035641 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e9801e040 con 0x563f41ea4c40 2023-07-21T22:31:53.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.483+0000 7f8e897fa700 1 -- 172.21.15.66:0/2942035641 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e980048f0 con 0x563f41ea4c40 2023-07-21T22:31:53.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2942035641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 msgr2=0x563f41ea7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41ea7020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e98015000 tx=0x7f8e98005ea0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2942035641 shutdown_connections 2023-07-21T22:31:53.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41ea7020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41e99860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2942035641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2942035641 >> 172.21.15.66:0/2942035641 conn(0x563f41e8fef0 msgr2=0x563f41e95860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2942035641 shutdown_connections 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.484+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2942035641 wait complete. 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.485+0000 7f8ebcd2ab80 1 -- start start 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.485+0000 7f8ebcd2ab80 1 Processor -- start 2023-07-21T22:31:53.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99490 0x563f41ea45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 0x563f41ea17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41ea4c40 0x563f41ea1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41d8bac0 con 0x563f41e99da0 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41d8b7c0 con 0x563f41ea4c40 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41c6f920 con 0x563f41e99490 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ebcd2ab80 10 client.0 _ll_register_callbacks cb 0x563f41d89760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 0x563f41ea17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 0x563f41ea17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48670/0 (socket says 172.21.15.66:48670) 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99490 0x563f41ea45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2872025212 learned_addr learned my addr 172.21.15.66:0/2872025212 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41ea4c40 0x563f41ea1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99490 msgr2=0x563f41ea45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99490 0x563f41ea45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2872025212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41ea4c40 msgr2=0x563f41ea1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41ea4c40 0x563f41ea1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.486+0000 7f8ea27fc700 1 -- 172.21.15.66:0/2872025212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41a48f0 con 0x563f41e99da0 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 0x563f41ea17e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c010 tx=0x7f8e9800fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbfe95c83e40058c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8e86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8e86ff5700 1 -- 172.21.15.66:0/2872025212 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9801d040 con 0x563f41e99da0 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f41d454f0 con 0x563f41e99da0 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8e86ff5700 1 -- 172.21.15.66:0/2872025212 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8e980058e0 con 0x563f41e99da0 2023-07-21T22:31:53.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8e86ff5700 1 -- 172.21.15.66:0/2872025212 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e98005610 con 0x563f41e99da0 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8ebcd2ab80 10 client.16779 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f8ea410d110 con 0x563f41e99da0 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.487+0000 7f8e86ff5700 1 -- 172.21.15.66:0/2872025212 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f41d454f0 con 0x563f41e99da0 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8e86ff5700 1 -- 172.21.15.66:0/2872025212 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8e98004b80 con 0x563f41e99da0 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ebcd2ab80 10 client.16779 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ebcd2ab80 10 client.16779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:31:53.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ebcd2ab80 20 client.16779 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41eb4050 0x563f41eb6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ebcd2ab80 4 client.16779 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f41de4eb0 con 0x563f41eb4050 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.488+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41eb4050 0x563f41eb6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.489+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41eb4050 0x563f41eb6460 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6750f44d4a6bb77a server_cookie=97d00782d5fc9c22 in_seq=0 out_seq=0 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.489+0000 7f8e86ff5700 10 client.16779 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.489+0000 7f8e86ff5700 1 -- 172.21.15.66:0/2872025212 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f41de4eb0 con 0x563f41eb4050 2023-07-21T22:31:53.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.489+0000 7f8e86ff5700 10 client.16779 handle_command_reply: tid=0 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.489+0000 7f8ebcd2ab80 1 client.16779 shutdown 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 msgr2=0x563f41ea17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 0x563f41ea17e0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c010 tx=0x7f8e9800fb80 comp rx=0 tx=0).stop 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 shutdown_connections 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41eb4050 0x563f41eb6460 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41ea4c40 0x563f41ea1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99490 0x563f41ea45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/2872025212 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99da0 0x563f41ea17e0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 >> 172.21.15.66:0/2872025212 conn(0x563f41e8fef0 msgr2=0x563f41e953f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8e86ff5700 0 client.16779 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:31:53.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 shutdown_connections 2023-07-21T22:31:53.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/2872025212 wait complete. 2023-07-21T22:31:53.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.490+0000 7f8ebcd2ab80 20 client.16779 trim_cache size 0 max 16384 2023-07-21T22:31:53.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 Processor -- start 2023-07-21T22:31:53.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 -- start start 2023-07-21T22:31:53.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99490 0x563f41eab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 0x563f41eb6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 0x563f41ea4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41ea2380 con 0x563f41ea4c40 2023-07-21T22:31:53.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41e9a170 con 0x563f41e99490 2023-07-21T22:31:53.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41e9fd40 con 0x563f41e99da0 2023-07-21T22:31:53.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 0x563f41ea4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ea27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 0x563f41eb6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99490 0x563f41eab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ea27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 0x563f41eb6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37902/0 (socket says 172.21.15.66:37902) 2023-07-21T22:31:53.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ea27fc700 1 -- 172.21.15.66:0/1284742293 learned_addr learned my addr 172.21.15.66:0/1284742293 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.504+0000 7f8ea37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 0x563f41ea4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48680/0 (socket says 172.21.15.66:48680) 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea37fe700 1 -- 172.21.15.66:0/1284742293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 msgr2=0x563f41eb6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 0x563f41eb6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea37fe700 1 -- 172.21.15.66:0/1284742293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99490 msgr2=0x563f41eab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99490 0x563f41eab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea37fe700 1 -- 172.21.15.66:0/1284742293 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f41e91730 con 0x563f41ea4c40 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 0x563f41eb6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:53.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 0x563f41ea4100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009040 tx=0x7f8e9c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a09119a0ad6876 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.505+0000 7f8e89ffb700 1 -- 172.21.15.66:0/1284742293 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c028070 con 0x563f41ea4c40 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8e89ffb700 1 -- 172.21.15.66:0/1284742293 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e9c00f040 con 0x563f41ea4c40 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8e89ffb700 1 -- 172.21.15.66:0/1284742293 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00ea10 con 0x563f41ea4c40 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/1284742293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 msgr2=0x563f41ea4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 0x563f41ea4100 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c009040 tx=0x7f8e9c009920 comp rx=0 tx=0).stop 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/1284742293 shutdown_connections 2023-07-21T22:31:53.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99490 0x563f41eab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41ea4c40 0x563f41ea4100 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/1284742293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41e99da0 0x563f41eb6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/1284742293 >> 172.21.15.66:0/1284742293 conn(0x563f41e8fef0 msgr2=0x563f41eafe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/1284742293 shutdown_connections 2023-07-21T22:31:53.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.506+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/1284742293 wait complete. 2023-07-21T22:31:53.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 -- start start 2023-07-21T22:31:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 Processor -- start 2023-07-21T22:31:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41ea7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41f06f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41ea8360 con 0x563f41e99490 2023-07-21T22:31:53.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41ea84d0 con 0x563f41e99da0 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f41f07470 con 0x563f41ea4c40 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.507+0000 7f8ebcd2ab80 10 client.0 _ll_register_callbacks cb 0x7f8e940962e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41ea7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41ea7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48692/0 (socket says 172.21.15.66:48692) 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/508260236 learned_addr learned my addr 172.21.15.66:0/508260236 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea37fe700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41f06f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 msgr2=0x563f41f06f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41f06f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/508260236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 msgr2=0x563f41ea7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 -- 172.21.15.66:0/508260236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea410d110 con 0x563f41e99490 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ea2ffd700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41ea7650 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e94096a10 tx=0x7f8e94003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89afb745b795e35d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8e86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8e86ff5700 1 -- 172.21.15.66:0/508260236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e940a23f0 con 0x563f41e99490 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8e86ff5700 1 -- 172.21.15.66:0/508260236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8e940ca8a0 con 0x563f41e99490 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.508+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e9801e040 con 0x563f41e99490 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8e86ff5700 1 -- 172.21.15.66:0/508260236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e940caa60 con 0x563f41e99490 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 10 client.16785 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f8e9c009180 con 0x563f41e99490 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8e86ff5700 1 -- 172.21.15.66:0/508260236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e9801e040 con 0x563f41e99490 2023-07-21T22:31:53.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8e86ff5700 1 -- 172.21.15.66:0/508260236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8e94099490 con 0x563f41e99490 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 10 client.16785 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 10 client.16785 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 20 client.16785 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41f08b80 0x563f41f0af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 4 client.16785 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f41da7050 con 0x563f41f08b80 2023-07-21T22:31:53.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.509+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41f08b80 0x563f41f0af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ea27fc700 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41f08b80 0x563f41f0af90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b4b74eec0a6da35 server_cookie=78401d1091699bdc in_seq=0 out_seq=0 2023-07-21T22:31:53.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8e86ff5700 10 client.16785 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:31:53.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8e86ff5700 1 -- 172.21.15.66:0/508260236 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f41da7050 con 0x563f41f08b80 2023-07-21T22:31:53.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8e86ff5700 10 client.16785 handle_command_reply: tid=0 2023-07-21T22:31:53.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 client.16785 shutdown 2023-07-21T22:31:53.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 msgr2=0x563f41ea7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41ea7650 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8e94096a10 tx=0x7f8e94003710 comp rx=0 tx=0).stop 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 shutdown_connections 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f41f08b80 0x563f41f0af90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f41ea4c40 0x563f41f06f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f41e99490 0x563f41ea7650 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8e86ff5700 0 client.16785 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 --2- 172.21.15.66:0/508260236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f41e99da0 0x563f41ea7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 >> 172.21.15.66:0/508260236 conn(0x563f41e8fef0 msgr2=0x563f41eb6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 shutdown_connections 2023-07-21T22:31:53.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.510+0000 7f8ebcd2ab80 1 -- 172.21.15.66:0/508260236 wait complete. 2023-07-21T22:31:53.566 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ebcd2ab80 20 client.16785 trim_cache size 0 max 16384 2023-07-21T22:31:53.566 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:31:53.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8e8c0a1360 msgr2=0x7f8e8c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8e8c0a1360 0x7f8e8c0a3810 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c003fd0 tx=0x7f8e9c004b40 comp rx=0 tx=0).stop 2023-07-21T22:31:53.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 msgr2=0x7f8ea419d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea419d900 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8e9400e970 tx=0x7f8e9400ec80 comp rx=0 tx=0).stop 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 shutdown_connections 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8e8c0a1360 0x7f8e8c0a3810 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ea41059e0 0x7f8ea4197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8ea40ff9d0 0x7f8ea419d900 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 --2- 172.21.15.66:0/150914705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ea4100380 0x7f8ea419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 >> 172.21.15.66:0/150914705 conn(0x7f8ea40fb4b0 msgr2=0x7f8ea4072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 shutdown_connections 2023-07-21T22:31:53.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.511+0000 7f8ea3fff700 1 -- 172.21.15.66:0/150914705 wait complete. 2023-07-21T22:31:53.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489357 5.45 MB/sec execute 440 sec latency 3336.499 ms 2023-07-21T22:31:53.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:31:53.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.739+0000 7f91c79ff700 1 -- 172.21.15.66:0/2089634825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c0072850 msgr2=0x7f91c0072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.739+0000 7f91c79ff700 1 --2- 172.21.15.66:0/2089634825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c0072850 0x7f91c0072c20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009990 tx=0x7f91bc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 -- 172.21.15.66:0/2089634825 shutdown_connections 2023-07-21T22:31:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 --2- 172.21.15.66:0/2089634825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c010c3a0 0x7f91c010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 --2- 172.21.15.66:0/2089634825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c0072850 0x7f91c0072c20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 --2- 172.21.15.66:0/2089634825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91c0073160 0x7f91c010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 -- 172.21.15.66:0/2089634825 >> 172.21.15.66:0/2089634825 conn(0x7f91c00fb600 msgr2=0x7f91c00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 -- 172.21.15.66:0/2089634825 shutdown_connections 2023-07-21T22:31:53.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.740+0000 7f91c79ff700 1 -- 172.21.15.66:0/2089634825 wait complete. 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 Processor -- start 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 -- start start 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 0x7f91c0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91c0073160 0x7f91c0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c010c3a0 0x7f91c019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c019cea0 con 0x7f91c010c3a0 2023-07-21T22:31:53.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c019d010 con 0x7f91c0072850 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c79ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c019d1d0 con 0x7f91c0073160 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c579b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 0x7f91c0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c579b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 0x7f91c0197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34444/0 (socket says 172.21.15.66:34444) 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.741+0000 7f91c579b700 1 -- 172.21.15.66:0/1461265871 learned_addr learned my addr 172.21.15.66:0/1461265871 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c010c3a0 0x7f91c019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91c0073160 0x7f91c0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c579b700 1 -- 172.21.15.66:0/1461265871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91c0073160 msgr2=0x7f91c0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c579b700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91c0073160 0x7f91c0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c579b700 1 -- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c010c3a0 msgr2=0x7f91c019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c579b700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c010c3a0 0x7f91c019c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c579b700 1 -- 172.21.15.66:0/1461265871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0095f0 con 0x7f91c0072850 2023-07-21T22:31:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c579b700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 0x7f91c0197ec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f91bc005910 tx=0x7f91bc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91b67fc700 1 -- 172.21.15.66:0/1461265871 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc01d070 con 0x7f91c0072850 2023-07-21T22:31:53.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91b67fc700 1 -- 172.21.15.66:0/1461265871 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91bc022470 con 0x7f91c0072850 2023-07-21T22:31:53.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c019d450 con 0x7f91c0072850 2023-07-21T22:31:53.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.742+0000 7f91b67fc700 1 -- 172.21.15.66:0/1461265871 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc017650 con 0x7f91c0072850 2023-07-21T22:31:53.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.743+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c01a4cf0 con 0x7f91c0072850 2023-07-21T22:31:53.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.746+0000 7f91b67fc700 1 -- 172.21.15.66:0/1461265871 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f91bc0225e0 con 0x7f91c0072850 2023-07-21T22:31:53.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.747+0000 7f91db360b80 1 Processor -- start 2023-07-21T22:31:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.747+0000 7f91db360b80 1 -- start start 2023-07-21T22:31:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91b67fc700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91ac0a12c0 0x7f91ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91db360b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d19f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91db360b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d19a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91db360b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91db360b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d09fcb0 con 0x55ce8d19a930 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91db360b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d0f2a40 con 0x55ce8d19f1b0 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91db360b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d0ebed0 con 0x55ce8d19fac0 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91b67fc700 1 -- 172.21.15.66:0/1461265871 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91bc0ce480 con 0x7f91c0072850 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91ac0a12c0 0x7f91ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d19a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37952/0 (socket says 172.21.15.66:37952) 2023-07-21T22:31:53.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2531567392 learned_addr learned my addr 172.21.15.66:0/2531567392 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c579b700 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2531567392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 msgr2=0x55ce8d19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2531567392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 msgr2=0x55ce8d19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.748+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2531567392 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce8cf19de0 con 0x55ce8d19fac0 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.749+0000 7f91c579b700 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.749+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91ac0a12c0 0x7f91ac0a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91c0069a90 tx=0x7f91b8015420 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.749+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d19a350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009cb0 tx=0x7f91b00113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73995aa9e3baf1f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.749+0000 7f91ab7fe700 1 -- 172.21.15.66:0/2531567392 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b00097b0 con 0x55ce8d19fac0 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.749+0000 7f91ab7fe700 1 -- 172.21.15.66:0/2531567392 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b0011c60 con 0x55ce8d19fac0 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.749+0000 7f91ab7fe700 1 -- 172.21.15.66:0/2531567392 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000f6a0 con 0x55ce8d19fac0 2023-07-21T22:31:53.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 -- 172.21.15.66:0/2531567392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 msgr2=0x55ce8d19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d19a350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009cb0 tx=0x7f91b00113e0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 -- 172.21.15.66:0/2531567392 shutdown_connections 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 --2- 172.21.15.66:0/2531567392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d19a350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 -- 172.21.15.66:0/2531567392 >> 172.21.15.66:0/2531567392 conn(0x55ce8d18fef0 msgr2=0x55ce8d195830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 -- 172.21.15.66:0/2531567392 shutdown_connections 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 -- 172.21.15.66:0/2531567392 wait complete. 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.750+0000 7f91db360b80 1 -- start start 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 Processor -- start 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19a930 0x55ce8d1a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 0x55ce8d1a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d1a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d08bac0 con 0x55ce8d19f1b0 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d08b7c0 con 0x55ce8d19a930 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8cf6f920 con 0x55ce8d19fac0 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91db360b80 10 client.0 _ll_register_callbacks cb 0x55ce8d089760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.751+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 0x55ce8d1a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 0x55ce8d1a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48722/0 (socket says 172.21.15.66:48722) 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3487049632 learned_addr learned my addr 172.21.15.66:0/3487049632 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c579b700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19a930 0x55ce8d1a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d1a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3487049632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 msgr2=0x55ce8d1a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d1a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19a930 msgr2=0x55ce8d1a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19a930 0x55ce8d1a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3487049632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c01a4cf0 con 0x55ce8d19f1b0 2023-07-21T22:31:53.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c579b700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19a930 0x55ce8d1a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 0x55ce8d1a3de0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f91b0012d60 tx=0x7f91b0012c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42069eea79776fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:53.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.752+0000 7f91a8ff9700 1 -- 172.21.15.66:0/3487049632 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0012880 con 0x55ce8d19f1b0 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce8d0454f0 con 0x55ce8d19f1b0 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91a8ff9700 1 -- 172.21.15.66:0/3487049632 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91b0011a60 con 0x55ce8d19f1b0 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91a8ff9700 1 -- 172.21.15.66:0/3487049632 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91b0004260 con 0x55ce8d19f1b0 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 10 client.16797 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f91b0009910 con 0x55ce8d19f1b0 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91a8ff9700 1 -- 172.21.15.66:0/3487049632 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce8d0454f0 con 0x55ce8d19f1b0 2023-07-21T22:31:53.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91a8ff9700 1 -- 172.21.15.66:0/3487049632 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f91b0022070 con 0x55ce8d19f1b0 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 10 client.16797 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 20 client.16797 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d1b4350 0x55ce8d1b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 4 client.16797 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.753+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce8d0e4eb0 con 0x55ce8d1b4350 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.754+0000 7f91c579b700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d1b4350 0x55ce8d1b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.754+0000 7f91c579b700 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d1b4350 0x55ce8d1b6760 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=786776523f762f19 server_cookie=3fdb26d99a3b8dd1 in_seq=0 out_seq=0 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.754+0000 7f91a8ff9700 10 client.16797 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91a8ff9700 1 -- 172.21.15.66:0/3487049632 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ce8d0e4eb0 con 0x55ce8d1b4350 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91a8ff9700 10 client.16797 handle_command_reply: tid=0 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 client.16797 shutdown 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 msgr2=0x55ce8d1a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 0x55ce8d1a3de0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f91b0012d60 tx=0x7f91b0012c20 comp rx=0 tx=0).stop 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 shutdown_connections 2023-07-21T22:31:53.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d1b4350 0x55ce8d1b6760 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19a930 0x55ce8d1a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d19fac0 0x55ce8d1a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91a8ff9700 0 client.16797 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 --2- 172.21.15.66:0/3487049632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19f1b0 0x55ce8d1a3de0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 >> 172.21.15.66:0/3487049632 conn(0x55ce8d18fef0 msgr2=0x55ce8d195420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 shutdown_connections 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.755+0000 7f91db360b80 1 -- 172.21.15.66:0/3487049632 wait complete. 2023-07-21T22:31:53.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.756+0000 7f91db360b80 20 client.16797 trim_cache size 0 max 16384 2023-07-21T22:31:53.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.769+0000 7f91db360b80 1 Processor -- start 2023-07-21T22:31:53.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.769+0000 7f91db360b80 1 -- start start 2023-07-21T22:31:53.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.769+0000 7f91db360b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.769+0000 7f91db360b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91db360b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d198f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91db360b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d196e70 con 0x55ce8d19a930 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91db360b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d19b6f0 con 0x55ce8d19f1b0 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91db360b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d1bb650 con 0x55ce8d18fef0 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48724/0 (socket says 172.21.15.66:48724) 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3500759525 learned_addr learned my addr 172.21.15.66:0/3500759525 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c579b700 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3500759525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 msgr2=0x55ce8d1a6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3500759525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 msgr2=0x55ce8d198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 -- 172.21.15.66:0/3500759525 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce8cf02c60 con 0x55ce8d19a930 2023-07-21T22:31:53.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.770+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1989c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009c00 tx=0x7f91b00113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e150550d3cfa20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91abfff700 1 -- 172.21.15.66:0/3500759525 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b003a070 con 0x55ce8d19a930 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91abfff700 1 -- 172.21.15.66:0/3500759525 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b0005c80 con 0x55ce8d19a930 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91abfff700 1 -- 172.21.15.66:0/3500759525 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b001a7c0 con 0x55ce8d19a930 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 -- 172.21.15.66:0/3500759525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 msgr2=0x55ce8d1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1989c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009c00 tx=0x7f91b00113e0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 -- 172.21.15.66:0/3500759525 shutdown_connections 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 --2- 172.21.15.66:0/3500759525 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1989c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.771+0000 7f91db360b80 1 -- 172.21.15.66:0/3500759525 >> 172.21.15.66:0/3500759525 conn(0x55ce8d19b060 msgr2=0x55ce8d1b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.772+0000 7f91db360b80 1 -- 172.21.15.66:0/3500759525 shutdown_connections 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.772+0000 7f91db360b80 1 -- 172.21.15.66:0/3500759525 wait complete. 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.772+0000 7f91db360b80 1 -- start start 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.772+0000 7f91db360b80 1 Processor -- start 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d2071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d207760 con 0x55ce8d19a930 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d2078d0 con 0x55ce8d19f1b0 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8d207a90 con 0x55ce8d18fef0 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91db360b80 10 client.0 _ll_register_callbacks cb 0x7f91b80158f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1a9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48730/0 (socket says 172.21.15.66:48730) 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2119736430 learned_addr learned my addr 172.21.15.66:0/2119736430 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c579b700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c5f9c700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d2071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2119736430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 msgr2=0x55ce8d1a9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 msgr2=0x55ce8d2071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:53.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d2071f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c4f9a700 1 -- 172.21.15.66:0/2119736430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b0009910 con 0x55ce8d19a930 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.773+0000 7f91c579b700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91c4f9a700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1a9cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f91b0005e10 tx=0x7f91b00094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa1e76aec3affad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91a8ff9700 1 -- 172.21.15.66:0/2119736430 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b003a040 con 0x55ce8d19a930 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91b0011c60 con 0x55ce8d19a930 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91a8ff9700 1 -- 172.21.15.66:0/2119736430 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91b0000e40 con 0x55ce8d19a930 2023-07-21T22:31:53.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91a8ff9700 1 -- 172.21.15.66:0/2119736430 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91b0004c80 con 0x55ce8d19a930 2023-07-21T22:31:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91db360b80 10 client.16803 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.774+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ce8cf40250 con 0x55ce8d19a930 2023-07-21T22:31:53.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91a8ff9700 1 -- 172.21.15.66:0/2119736430 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b0011c60 con 0x55ce8d19a930 2023-07-21T22:31:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91a8ff9700 1 -- 172.21.15.66:0/2119736430 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f91b0011450 con 0x55ce8d19a930 2023-07-21T22:31:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91db360b80 10 client.16803 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91db360b80 10 client.16803 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:31:53.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91db360b80 20 client.16803 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91db360b80 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d19fac0 0x55ce8d20ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91db360b80 4 client.16803 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce8d0a7050 con 0x55ce8d19fac0 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.775+0000 7f91c579b700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d19fac0 0x55ce8d20ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91c579b700 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d19fac0 0x55ce8d20ae70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7446d79cdcd169e6 server_cookie=d7be4e65bf244946 in_seq=0 out_seq=0 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91a8ff9700 10 client.16803 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:31:53.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91a8ff9700 1 -- 172.21.15.66:0/2119736430 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce8d0a7050 con 0x55ce8d19fac0 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91a8ff9700 10 client.16803 handle_command_reply: tid=0 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 client.16803 shutdown 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 msgr2=0x55ce8d1a9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1a9cc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f91b0005e10 tx=0x7f91b00094b0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 shutdown_connections 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55ce8d19fac0 0x55ce8d20ae70 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ce8d19f1b0 0x55ce8d2071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce8d18fef0 0x55ce8d1a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 --2- 172.21.15.66:0/2119736430 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55ce8d19a930 0x55ce8d1a9cc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91a8ff9700 0 client.16803 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:31:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.776+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 >> 172.21.15.66:0/2119736430 conn(0x55ce8d19b060 msgr2=0x55ce8d1989c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 shutdown_connections 2023-07-21T22:31:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91db360b80 1 -- 172.21.15.66:0/2119736430 wait complete. 2023-07-21T22:31:53.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91db360b80 20 client.16803 trim_cache size 0 max 16384 2023-07-21T22:31:53.831 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:31:53.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91ac0a12c0 msgr2=0x7f91ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91ac0a12c0 0x7f91ac0a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f91c0069a90 tx=0x7f91b8015420 comp rx=0 tx=0).stop 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 msgr2=0x7f91c0197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 0x7f91c0197ec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f91bc005910 tx=0x7f91bc0047c0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 shutdown_connections 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91c0072850 0x7f91c0197ec0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f91ac0a12c0 0x7f91ac0a3770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f91c010c3a0 0x7f91c019c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 --2- 172.21.15.66:0/1461265871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91c0073160 0x7f91c0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.777+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 >> 172.21.15.66:0/1461265871 conn(0x7f91c00fb600 msgr2=0x7f91c010e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.778+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 shutdown_connections 2023-07-21T22:31:53.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:53.778+0000 7f91c79ff700 1 -- 172.21.15.66:0/1461265871 wait complete. 2023-07-21T22:31:53.842 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:31:54.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 -- 172.21.15.66:0/1663358700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac1015b0 msgr2=0x7f47ac109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 --2- 172.21.15.66:0/1663358700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac1015b0 0x7f47ac109aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009990 tx=0x7f47a0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 -- 172.21.15.66:0/1663358700 shutdown_connections 2023-07-21T22:31:54.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 --2- 172.21.15.66:0/1663358700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47ac10a070 0x7f47ac10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 --2- 172.21.15.66:0/1663358700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 --2- 172.21.15.66:0/1663358700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac1015b0 0x7f47ac109aa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 -- 172.21.15.66:0/1663358700 >> 172.21.15.66:0/1663358700 conn(0x7f47ac0fa4b0 msgr2=0x7f47ac0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 -- 172.21.15.66:0/1663358700 shutdown_connections 2023-07-21T22:31:54.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.006+0000 7f47b23ff700 1 -- 172.21.15.66:0/1663358700 wait complete. 2023-07-21T22:31:54.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.007+0000 7f47b23ff700 1 Processor -- start 2023-07-21T22:31:54.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 -- start start 2023-07-21T22:31:54.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47ac1015b0 0x7f47ac0fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac10a070 0x7f47ac1a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0ff660 con 0x7f47ac10a070 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1a4350 con 0x7f47ac1015b0 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b23ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1a4490 con 0x7f47ac100ca0 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47b099c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac10a070 0x7f47ac1a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac0fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38006/0 (socket says 172.21.15.66:38006) 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47abfff700 1 -- 172.21.15.66:0/322496984 learned_addr learned my addr 172.21.15.66:0/322496984 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.008+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47ac1015b0 0x7f47ac0fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47abfff700 1 -- 172.21.15.66:0/322496984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47ac1015b0 msgr2=0x7f47ac0fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47abfff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47ac1015b0 0x7f47ac0fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47abfff700 1 -- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac10a070 msgr2=0x7f47ac1a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47abfff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac10a070 0x7f47ac1a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47abfff700 1 -- 172.21.15.66:0/322496984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009520 con 0x7f47ac100ca0 2023-07-21T22:31:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47abfff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac0fe8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f479c00eb30 tx=0x7f479c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47a97fa700 1 -- 172.21.15.66:0/322496984 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c009880 con 0x7f47ac100ca0 2023-07-21T22:31:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a00095f0 con 0x7f47ac100ca0 2023-07-21T22:31:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47a97fa700 1 -- 172.21.15.66:0/322496984 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f479c0044e0 con 0x7f47ac100ca0 2023-07-21T22:31:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47a97fa700 1 -- 172.21.15.66:0/322496984 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c003d80 con 0x7f47ac100ca0 2023-07-21T22:31:54.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.009+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac1a4a90 con 0x7f47ac100ca0 2023-07-21T22:31:54.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.013+0000 7f47a97fa700 1 -- 172.21.15.66:0/322496984 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f479c003f20 con 0x7f47ac100ca0 2023-07-21T22:31:54.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 Processor -- start 2023-07-21T22:31:54.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 -- start start 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 0x562282fa7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47a97fa700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f47940a12c0 0x7f47940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fa7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282eaccb0 con 0x562282fa7930 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282effa40 con 0x562282facac0 2023-07-21T22:31:54.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282ef8ed0 con 0x562282fac1b0 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47a97fa700 1 -- 172.21.15.66:0/322496984 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f479c014070 con 0x7f47ac100ca0 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 0x562282fa7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47abfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 0x562282fa7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34526/0 (socket says 172.21.15.66:34526) 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.014+0000 7f47abfff700 1 -- 172.21.15.66:0/2913962709 learned_addr learned my addr 172.21.15.66:0/2913962709 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f47940a12c0 0x7f47940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47b099c700 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fa7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47b099c700 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47abfff700 1 -- 172.21.15.66:0/2913962709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 msgr2=0x562282fac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47abfff700 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47abfff700 1 -- 172.21.15.66:0/2913962709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 msgr2=0x562282fa7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47abfff700 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fa7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47abfff700 1 -- 172.21.15.66:0/2913962709 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562282d26de0 con 0x562282facac0 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47b099c700 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fa7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f47940a12c0 0x7f47940a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47ac1000e0 tx=0x7f47a0005fd0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47abfff700 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 0x562282fa7350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f479c00eb00 tx=0x7f479c09a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c687d143e35b6424 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.015+0000 7f47927fc700 1 -- 172.21.15.66:0/2913962709 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c099100 con 0x562282facac0 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47927fc700 1 -- 172.21.15.66:0/2913962709 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f479c09e170 con 0x562282facac0 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47927fc700 1 -- 172.21.15.66:0/2913962709 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0cd470 con 0x562282facac0 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/2913962709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 msgr2=0x562282fa7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 0x562282fa7350 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f479c00eb00 tx=0x7f479c09a1a0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/2913962709 shutdown_connections 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fa7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/2913962709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282facac0 0x562282fa7350 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/2913962709 >> 172.21.15.66:0/2913962709 conn(0x562282f9cef0 msgr2=0x562282fa2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/2913962709 shutdown_connections 2023-07-21T22:31:54.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.016+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/2913962709 wait complete. 2023-07-21T22:31:54.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.017+0000 7f47c5dbfb80 1 -- start start 2023-07-21T22:31:54.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.017+0000 7f47c5dbfb80 1 Processor -- start 2023-07-21T22:31:54.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fb0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fb1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282facac0 0x562282fb1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282e98ac0 con 0x562282fa7930 2023-07-21T22:31:54.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282e987c0 con 0x562282fac1b0 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282d7c920 con 0x562282facac0 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47c5dbfb80 10 client.0 _ll_register_callbacks cb 0x562282e96760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fb0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fb1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47abfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fb0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48756/0 (socket says 172.21.15.66:48756) 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47abfff700 1 -- 172.21.15.66:0/615465179 learned_addr learned my addr 172.21.15.66:0/615465179 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.018+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fb1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34534/0 (socket says 172.21.15.66:34534) 2023-07-21T22:31:54.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47b099c700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282facac0 0x562282fb1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47abfff700 1 -- 172.21.15.66:0/615465179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282facac0 msgr2=0x562282fb1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47abfff700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282facac0 0x562282fb1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47abfff700 1 -- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 msgr2=0x562282fb1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47abfff700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fb1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47abfff700 1 -- 172.21.15.66:0/615465179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac1a4a90 con 0x562282fa7930 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47b099c700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282facac0 0x562282fb1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fb1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:54.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47abfff700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fb0ef0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c0caf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8be3181fa534464 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f478b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f478b7fe700 1 -- 172.21.15.66:0/615465179 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c09a780 con 0x562282fa7930 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f478b7fe700 1 -- 172.21.15.66:0/615465179 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f479c0cd050 con 0x562282fa7930 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.019+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562282ef1eb0 con 0x562282fa7930 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f478b7fe700 1 -- 172.21.15.66:0/615465179 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479c09e170 con 0x562282fa7930 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 10 client.16815 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:54.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562282d0fc60 con 0x562282fa7930 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f478b7fe700 1 -- 172.21.15.66:0/615465179 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562282ef1eb0 con 0x562282fa7930 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f478b7fe700 1 -- 172.21.15.66:0/615465179 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f479c099810 con 0x562282fa7930 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 10 client.16815 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 20 client.16815 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562282fc11a0 0x562282fc3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 4 client.16815 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:54.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.020+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562282eb4050 con 0x562282fc11a0 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.021+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562282fc11a0 0x562282fc3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.021+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562282fc11a0 0x562282fc3580 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=627340776f4f34a0 server_cookie=52f8330c7ddf82d1 in_seq=0 out_seq=0 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.021+0000 7f478b7fe700 10 client.16815 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f478b7fe700 1 -- 172.21.15.66:0/615465179 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562282eb4050 con 0x562282fc11a0 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f478b7fe700 10 client.16815 handle_command_reply: tid=0 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 client.16815 shutdown 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 msgr2=0x562282fb0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fb0ef0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c0caf00 comp rx=0 tx=0).stop 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 shutdown_connections 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562282fc11a0 0x562282fc3580 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282facac0 0x562282fb1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282fa7930 0x562282fb0ef0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/615465179 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fb1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 >> 172.21.15.66:0/615465179 conn(0x562282f9cef0 msgr2=0x562282fa23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f478b7fe700 0 client.16815 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:54.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 shutdown_connections 2023-07-21T22:31:54.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/615465179 wait complete. 2023-07-21T22:31:54.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.022+0000 7f47c5dbfb80 20 client.16815 trim_cache size 0 max 16384 2023-07-21T22:31:54.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.036+0000 7f47c5dbfb80 1 Processor -- start 2023-07-21T22:31:54.082 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.036+0000 7f47c5dbfb80 1 -- start start 2023-07-21T22:31:54.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.036+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fa7930 0x562282fb8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.083 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fa7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282fa3320 con 0x562282facac0 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282fb8ab0 con 0x562282fac1b0 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282fa6330 con 0x562282fa7930 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fa7930 0x562282fb8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fa7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.084 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47abfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fa7930 0x562282fb8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38042/0 (socket says 172.21.15.66:38042) 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47abfff700 1 -- 172.21.15.66:0/739779924 learned_addr learned my addr 172.21.15.66:0/739779924 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47b099c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47ab7fe700 1 -- 172.21.15.66:0/739779924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fa7930 msgr2=0x562282fb8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fa7930 0x562282fb8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47ab7fe700 1 -- 172.21.15.66:0/739779924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 msgr2=0x562282fb17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.037+0000 7f47ab7fe700 1 -- 172.21.15.66:0/739779924 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562282d4d250 con 0x562282fac1b0 2023-07-21T22:31:54.085 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fa7130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b2e0 tx=0x7f47a0023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc23573a77dcf495 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f4792ffd700 1 -- 172.21.15.66:0/739779924 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a002b030 con 0x562282fac1b0 2023-07-21T22:31:54.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f4792ffd700 1 -- 172.21.15.66:0/739779924 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a002c040 con 0x562282fac1b0 2023-07-21T22:31:54.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f4792ffd700 1 -- 172.21.15.66:0/739779924 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a00332b0 con 0x562282fac1b0 2023-07-21T22:31:54.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/739779924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 msgr2=0x562282fa7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fa7130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b2e0 tx=0x7f47a0023000 comp rx=0 tx=0).stop 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/739779924 shutdown_connections 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fa7930 0x562282fb8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/739779924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fac1b0 0x562282fa7130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.038+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/739779924 >> 172.21.15.66:0/739779924 conn(0x562282f9cef0 msgr2=0x562282fa6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.039+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/739779924 shutdown_connections 2023-07-21T22:31:54.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.039+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/739779924 wait complete. 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.039+0000 7f47c5dbfb80 1 -- start start 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.039+0000 7f47c5dbfb80 1 Processor -- start 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fa7930 0x562282fb7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fb81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282fb66e0 con 0x562282facac0 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282fb6850 con 0x562282fa7930 2023-07-21T22:31:54.090 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562282fb6a10 con 0x562282fac1b0 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47c5dbfb80 10 client.0 _ll_register_callbacks cb 0x7f47a4009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fb81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47b099c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fb81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38050/0 (socket says 172.21.15.66:38050) 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 -- 172.21.15.66:0/3206960524 learned_addr learned my addr 172.21.15.66:0/3206960524 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47abfff700 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fa7930 0x562282fb7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 -- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fa7930 msgr2=0x562282fb7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.091 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fa7930 0x562282fb7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 -- 172.21.15.66:0/3206960524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 msgr2=0x562282fb6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.040+0000 7f47ab7fe700 1 -- 172.21.15.66:0/3206960524 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac10d170 con 0x562282fac1b0 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f47ab7fe700 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fb81c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47a0000c00 tx=0x7f47a0005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c2a70a47bf5fe7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f478b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f478b7fe700 1 -- 172.21.15.66:0/3206960524 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a002b050 con 0x562282fac1b0 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562282e524f0 con 0x562282fac1b0 2023-07-21T22:31:54.092 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f478b7fe700 1 -- 172.21.15.66:0/3206960524 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f47a0034de0 con 0x562282fac1b0 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f478b7fe700 1 -- 172.21.15.66:0/3206960524 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a00048e0 con 0x562282fac1b0 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f47c5dbfb80 10 client.26854 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562282d252b0 con 0x562282fac1b0 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.041+0000 7f478b7fe700 1 -- 172.21.15.66:0/3206960524 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562282e524f0 con 0x562282fac1b0 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f478b7fe700 1 -- 172.21.15.66:0/3206960524 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f47a0046080 con 0x562282fac1b0 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47c5dbfb80 10 client.26854 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:54.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47c5dbfb80 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:54.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47c5dbfb80 20 client.26854 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:54.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562283015d30 0x562283018140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:54.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47c5dbfb80 4 client.26854 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:31:54.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562282d47f70 con 0x562283015d30 2023-07-21T22:31:54.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47abfff700 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562283015d30 0x562283018140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:54.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f47abfff700 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562283015d30 0x562283018140 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca0ea22b653a82d3 server_cookie=1a83dff2a5b87218 in_seq=0 out_seq=0 2023-07-21T22:31:54.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.042+0000 7f478b7fe700 10 client.26854 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:54.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f478b7fe700 1 -- 172.21.15.66:0/3206960524 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562282d47f70 con 0x562283015d30 2023-07-21T22:31:54.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f478b7fe700 10 client.26854 handle_command_reply: tid=0 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 client.26854 shutdown 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 msgr2=0x562282fb81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fb81c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f47a0000c00 tx=0x7f47a0005f10 comp rx=0 tx=0).stop 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 shutdown_connections 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562282fa7930 0x562282fb7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x562283015d30 0x562283018140 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f478b7fe700 0 client.26854 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x562282facac0 0x562282fb6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 --2- 172.21.15.66:0/3206960524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562282fac1b0 0x562282fb81c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 >> 172.21.15.66:0/3206960524 conn(0x562282f9cef0 msgr2=0x562282fb1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 shutdown_connections 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 1 -- 172.21.15.66:0/3206960524 wait complete. 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.043+0000 7f47c5dbfb80 20 client.26854 trim_cache size 0 max 16384 2023-07-21T22:31:54.097 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:31:54.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f47940a12c0 msgr2=0x7f47940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f47940a12c0 0x7f47940a3770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47ac1000e0 tx=0x7f47a0005fd0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 msgr2=0x7f47ac0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac0fe8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f479c00eb30 tx=0x7f479c00ee40 comp rx=0 tx=0).stop 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 shutdown_connections 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f47940a12c0 0x7f47940a3770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f47ac10a070 0x7f47ac1a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ac100ca0 0x7f47ac0fe8f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 --2- 172.21.15.66:0/322496984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47ac1015b0 0x7f47ac0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:54.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 >> 172.21.15.66:0/322496984 conn(0x7f47ac0fa4b0 msgr2=0x7f47ac10c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 shutdown_connections 2023-07-21T22:31:54.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:54.044+0000 7f47b23ff700 1 -- 172.21.15.66:0/322496984 wait complete. 2023-07-21T22:31:54.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:53 smithi066 ceph-mon[126379]: Adjusting osd_memory_target on smithi066 to 220.9M 2023-07-21T22:31:54.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:53 smithi066 ceph-mon[126379]: Unable to set osd_memory_target on smithi066 to 231653273: error parsing value: Value '231653273' is below minimum 939524096 2023-07-21T22:31:54.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:53 smithi066 ceph-mon[126379]: pgmap v323: 129 pgs: 129 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 2.5 MiB/s wr, 52 op/s 2023-07-21T22:31:54.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:53 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2177238700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:31:54.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:53 smithi104 ceph-mon[129021]: Adjusting osd_memory_target on smithi066 to 220.9M 2023-07-21T22:31:54.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:53 smithi104 ceph-mon[129021]: Unable to set osd_memory_target on smithi066 to 231653273: error parsing value: Value '231653273' is below minimum 939524096 2023-07-21T22:31:54.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:53 smithi104 ceph-mon[129021]: pgmap v323: 129 pgs: 129 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 2.5 MiB/s wr, 52 op/s 2023-07-21T22:31:54.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:53 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2177238700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:31:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:53 smithi149 ceph-mon[129268]: Adjusting osd_memory_target on smithi066 to 220.9M 2023-07-21T22:31:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:53 smithi149 ceph-mon[129268]: Unable to set osd_memory_target on smithi066 to 231653273: error parsing value: Value '231653273' is below minimum 939524096 2023-07-21T22:31:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:53 smithi149 ceph-mon[129268]: pgmap v323: 129 pgs: 129 active+clean; 453 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 2.5 MiB/s wr, 52 op/s 2023-07-21T22:31:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:53 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2177238700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:31:54.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489357 5.44 MB/sec execute 441 sec latency 4336.584 ms 2023-07-21T22:31:55.110 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:31:55.110 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:31:55.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 -- 172.21.15.66:0/4064341758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e00731c0 msgr2=0x7fd4e01105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/4064341758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e00731c0 0x7fd4e01105a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009990 tx=0x7fd4d0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 -- 172.21.15.66:0/4064341758 shutdown_connections 2023-07-21T22:31:55.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/4064341758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 0x7fd4e0072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/4064341758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e0110ae0 0x7fd4e0112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/4064341758 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e00731c0 0x7fd4e01105a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 -- 172.21.15.66:0/4064341758 >> 172.21.15.66:0/4064341758 conn(0x7fd4e00fb610 msgr2=0x7fd4e00fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 -- 172.21.15.66:0/4064341758 shutdown_connections 2023-07-21T22:31:55.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.293+0000 7fd4e87ff700 1 -- 172.21.15.66:0/4064341758 wait complete. 2023-07-21T22:31:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 Processor -- start 2023-07-21T22:31:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 -- start start 2023-07-21T22:31:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 0x7fd4e019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 0x7fd4e019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e0110ae0 0x7fd4e01a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a11b0 con 0x7fd4e0110ae0 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a1320 con 0x7fd4e00728b0 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.294+0000 7fd4e87ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01a14e0 con 0x7fd4e00731c0 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 0x7fd4e019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 0x7fd4e019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e6d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e0110ae0 0x7fd4e01a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 0x7fd4e019c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38056/0 (socket says 172.21.15.66:38056) 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 -- 172.21.15.66:0/2738214554 learned_addr learned my addr 172.21.15.66:0/2738214554 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 -- 172.21.15.66:0/2738214554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 msgr2=0x7fd4e019c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 0x7fd4e019c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 -- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e0110ae0 msgr2=0x7fd4e01a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e0110ae0 0x7fd4e01a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 -- 172.21.15.66:0/2738214554 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00095f0 con 0x7fd4e00731c0 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e6d9c700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e0110ae0 0x7fd4e01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 0x7fd4e019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:55.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.295+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 0x7fd4e019c780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005910 tx=0x7fd4d0005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.296+0000 7fd4d77fe700 1 -- 172.21.15.66:0/2738214554 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d001d070 con 0x7fd4e00731c0 2023-07-21T22:31:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.296+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e01a1760 con 0x7fd4e00731c0 2023-07-21T22:31:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.296+0000 7fd4d77fe700 1 -- 172.21.15.66:0/2738214554 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4d0022470 con 0x7fd4e00731c0 2023-07-21T22:31:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.296+0000 7fd4d77fe700 1 -- 172.21.15.66:0/2738214554 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0017650 con 0x7fd4e00731c0 2023-07-21T22:31:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.296+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e01a9000 con 0x7fd4e00731c0 2023-07-21T22:31:55.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.299+0000 7fd4d77fe700 1 -- 172.21.15.66:0/2738214554 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd4d00225e0 con 0x7fd4e00731c0 2023-07-21T22:31:55.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.300+0000 7fd4fc06bb80 1 Processor -- start 2023-07-21T22:31:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 -- start start 2023-07-21T22:31:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7925600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 0x560dd79203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd7920e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7825d30 con 0x560dd79209b0 2023-07-21T22:31:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7878ac0 con 0x560dd7925230 2023-07-21T22:31:55.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7871f50 con 0x560dd7925b40 2023-07-21T22:31:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 0x560dd79203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 0x560dd79203d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38062/0 (socket says 172.21.15.66:38062) 2023-07-21T22:31:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4e659b700 1 -- 172.21.15.66:0/2505713405 learned_addr learned my addr 172.21.15.66:0/2505713405 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4e6d9c700 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7925600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.301+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd7920e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e659b700 1 -- 172.21.15.66:0/2505713405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 msgr2=0x560dd7925600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7925600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e659b700 1 -- 172.21.15.66:0/2505713405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 msgr2=0x560dd7920e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4d77fe700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd4cc0a1340 0x7fd4cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd7920e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e659b700 1 -- 172.21.15.66:0/2505713405 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dd769fe60 con 0x560dd7925b40 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd7920e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:55.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4d77fe700 1 -- 172.21.15.66:0/2738214554 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4d00ce380 con 0x7fd4e00731c0 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 0x560dd79203d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0115f0 tx=0x7fd4dc011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8460faf7e477855f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4e6d9c700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd4cc0a1340 0x7fd4cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4d4ff9700 1 -- 172.21.15.66:0/2505713405 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc019630 con 0x560dd7925b40 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4d4ff9700 1 -- 172.21.15.66:0/2505713405 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4dc019c50 con 0x560dd7925b40 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.302+0000 7fd4d4ff9700 1 -- 172.21.15.66:0/2505713405 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00eb70 con 0x560dd7925b40 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2505713405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 msgr2=0x560dd79203d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4e6d9c700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd4cc0a1340 0x7fd4cc0a37f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0069bc0 tx=0x7fd4d800b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 0x560dd79203d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0115f0 tx=0x7fd4dc011900 comp rx=0 tx=0).stop 2023-07-21T22:31:55.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2505713405 shutdown_connections 2023-07-21T22:31:55.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7925600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd7920e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2505713405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7925b40 0x560dd79203d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2505713405 >> 172.21.15.66:0/2505713405 conn(0x560dd7915f70 msgr2=0x560dd791b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2505713405 shutdown_connections 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.303+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2505713405 wait complete. 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.304+0000 7fd4fc06bb80 1 -- start start 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.304+0000 7fd4fc06bb80 1 Processor -- start 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.304+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 0x560dd792ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7929f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925b40 0x560dd792a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7811b40 con 0x560dd7925b40 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7811840 con 0x560dd7925230 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd76f59a0 con 0x560dd79209b0 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4fc06bb80 10 client.0 _ll_register_callbacks cb 0x560dd780f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 0x560dd792ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e6d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925b40 0x560dd792a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 0x560dd792ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38068/0 (socket says 172.21.15.66:38068) 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e6d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925b40 0x560dd792a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48828/0 (socket says 172.21.15.66:48828) 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 -- 172.21.15.66:0/3106598850 learned_addr learned my addr 172.21.15.66:0/3106598850 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7929f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 -- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 msgr2=0x560dd7929f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7929f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 -- 172.21.15.66:0/3106598850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925b40 msgr2=0x560dd792a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925b40 0x560dd792a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.305+0000 7fd4e659b700 1 -- 172.21.15.66:0/3106598850 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc009dd0 con 0x560dd79209b0 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4e659b700 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 0x560dd792ccc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc004680 tx=0x7fd4dc00e3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345c1fcd46e812e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/3106598850 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005460 con 0x560dd79209b0 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dd77cb570 con 0x560dd79209b0 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/3106598850 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4dc005ce0 con 0x560dd79209b0 2023-07-21T22:31:55.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/3106598850 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4dc0055c0 con 0x560dd79209b0 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4fc06bb80 10 client.26866 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560dd7688ce0 con 0x560dd79209b0 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.306+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/3106598850 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dd77cb570 con 0x560dd79209b0 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/3106598850 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd4dc004c40 con 0x560dd79209b0 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4fc06bb80 10 client.26866 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4fc06bb80 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:55.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4fc06bb80 20 client.26866 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd793a400 0x560dd793c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4fc06bb80 4 client.26866 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dd786af30 con 0x560dd793a400 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd793a400 0x560dd793c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd793a400 0x560dd793c810 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1abef5e1733b2bb server_cookie=f59e3b5ef824602 in_seq=0 out_seq=0 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.307+0000 7fd4c9ffb700 10 client.26866 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/3106598850 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560dd786af30 con 0x560dd793a400 2023-07-21T22:31:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4c9ffb700 10 client.26866 handle_command_reply: tid=0 2023-07-21T22:31:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 client.26866 shutdown 2023-07-21T22:31:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 msgr2=0x560dd792ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 0x560dd792ccc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc004680 tx=0x7fd4dc00e3b0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 shutdown_connections 2023-07-21T22:31:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd793a400 0x560dd793c810 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925b40 0x560dd792a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4c9ffb700 0 client.26866 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.308+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd79209b0 0x560dd792ccc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.309+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/3106598850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd7929f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.309+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 >> 172.21.15.66:0/3106598850 conn(0x560dd7915f70 msgr2=0x560dd791b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.309+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 shutdown_connections 2023-07-21T22:31:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.309+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/3106598850 wait complete. 2023-07-21T22:31:55.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.309+0000 7fd4fc06bb80 20 client.26866 trim_cache size 0 max 16384 2023-07-21T22:31:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.326+0000 7fd4fc06bb80 1 Processor -- start 2023-07-21T22:31:55.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 -- start start 2023-07-21T22:31:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd792c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd792cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd792d3e0 con 0x560dd79209b0 2023-07-21T22:31:55.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd791c3a0 con 0x560dd7925230 2023-07-21T22:31:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd791cef0 con 0x560dd7915f70 2023-07-21T22:31:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4e5d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd792c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4e6d9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd792cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4e5d9a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd792c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48844/0 (socket says 172.21.15.66:48844) 2023-07-21T22:31:55.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.327+0000 7fd4e5d9a700 1 -- 172.21.15.66:0/2581022520 learned_addr learned my addr 172.21.15.66:0/2581022520 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4e659b700 1 -- 172.21.15.66:0/2581022520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 msgr2=0x560dd792cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd792cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4e659b700 1 -- 172.21.15.66:0/2581022520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 msgr2=0x560dd792c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd792c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4e659b700 1 -- 172.21.15.66:0/2581022520 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dd76c62d0 con 0x560dd7915f70 2023-07-21T22:31:55.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4e659b700 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79201b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc011510 tx=0x7fd4dc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eef88764413e034 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4d57fa700 1 -- 172.21.15.66:0/2581022520 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc011d00 con 0x560dd7915f70 2023-07-21T22:31:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4d57fa700 1 -- 172.21.15.66:0/2581022520 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4dc00e940 con 0x560dd7915f70 2023-07-21T22:31:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.328+0000 7fd4d57fa700 1 -- 172.21.15.66:0/2581022520 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc019430 con 0x560dd7915f70 2023-07-21T22:31:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2581022520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 msgr2=0x560dd79201b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79201b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc011510 tx=0x7fd4dc000bc0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2581022520 shutdown_connections 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd7925230 0x560dd792cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79201b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/2581022520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd79209b0 0x560dd792c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2581022520 >> 172.21.15.66:0/2581022520 conn(0x560dd79210e0 msgr2=0x560dd791faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2581022520 shutdown_connections 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/2581022520 wait complete. 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.329+0000 7fd4fc06bb80 1 -- start start 2023-07-21T22:31:55.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 Processor -- start 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79879f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd79209b0 0x560dd7987f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 0x560dd7930970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd79310b0 con 0x560dd7925230 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd7931220 con 0x560dd79209b0 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dd79313e0 con 0x560dd7915f70 2023-07-21T22:31:55.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4fc06bb80 10 client.0 _ll_register_callbacks cb 0x7fd4d8009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4e6d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 0x560dd7930970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79879f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4e6d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 0x560dd7930970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48846/0 (socket says 172.21.15.66:48846) 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4e659b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79879f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38082/0 (socket says 172.21.15.66:38082) 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4e6d9c700 1 -- 172.21.15.66:0/560887021 learned_addr learned my addr 172.21.15.66:0/560887021 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.330+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd79209b0 0x560dd7987f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e659b700 1 -- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd79209b0 msgr2=0x560dd7987f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e659b700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd79209b0 0x560dd7987f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e659b700 1 -- 172.21.15.66:0/560887021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 msgr2=0x560dd7930970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e659b700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 0x560dd7930970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e659b700 1 -- 172.21.15.66:0/560887021 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dd7688ce0 con 0x560dd7915f70 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e6d9c700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 0x560dd7930970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4e659b700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79879f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc011510 tx=0x7fd4dc00e820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80f6265d3d01199 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.331+0000 7fd4c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:31:55.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/560887021 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005460 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4dc019c50 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/560887021 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd4dc000e40 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/560887021 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4dc019dc0 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4fc06bb80 10 client.26872 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560dd769e330 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.332+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/560887021 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4dc019c50 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/560887021 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd4dc004850 con 0x560dd7915f70 2023-07-21T22:31:55.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4fc06bb80 10 client.26872 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4fc06bb80 10 client.26872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4fc06bb80 20 client.26872 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd7925b40 0x560dd79924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4fc06bb80 4 client.26872 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560dd782d0d0 con 0x560dd7925b40 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd7925b40 0x560dd79924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4c9ffb700 10 client.26872 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.333+0000 7fd4e5d9a700 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd7925b40 0x560dd79924a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa23bfe9798cb1b5 server_cookie=6b3f95b689ff6082 in_seq=0 out_seq=0 2023-07-21T22:31:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.334+0000 7fd4c9ffb700 1 -- 172.21.15.66:0/560887021 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560dd782d0d0 con 0x560dd7925b40 2023-07-21T22:31:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.334+0000 7fd4c9ffb700 10 client.26872 handle_command_reply: tid=0 2023-07-21T22:31:55.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.334+0000 7fd4fc06bb80 1 client.26872 shutdown 2023-07-21T22:31:55.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 msgr2=0x560dd79879f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79879f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc011510 tx=0x7fd4dc00e820 comp rx=0 tx=0).stop 2023-07-21T22:31:55.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 shutdown_connections 2023-07-21T22:31:55.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560dd7925b40 0x560dd79924a0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560dd7925230 0x560dd7930970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560dd7915f70 0x560dd79879f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 --2- 172.21.15.66:0/560887021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560dd79209b0 0x560dd7987f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4c9ffb700 0 client.26872 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 >> 172.21.15.66:0/560887021 conn(0x560dd79210e0 msgr2=0x560dd79201a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 shutdown_connections 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 1 -- 172.21.15.66:0/560887021 wait complete. 2023-07-21T22:31:55.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.335+0000 7fd4fc06bb80 20 client.26872 trim_cache size 0 max 16384 2023-07-21T22:31:55.392 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:31:55.392 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:31:55.392 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "b2a48f23-b933-41fc-bda9-5d9512b32f98", 2023-07-21T22:31:55.392 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:31:55.392 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:31:55.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd4cc0a1340 msgr2=0x7fd4cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd4cc0a1340 0x7fd4cc0a37f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0069bc0 tx=0x7fd4d800b040 comp rx=0 tx=0).stop 2023-07-21T22:31:55.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 msgr2=0x7fd4e019c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 0x7fd4e019c780 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005910 tx=0x7fd4d0005720 comp rx=0 tx=0).stop 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 shutdown_connections 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd4e00728b0 0x7fd4e019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd4cc0a1340 0x7fd4cc0a37f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd4e0110ae0 0x7fd4e01a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 --2- 172.21.15.66:0/2738214554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4e00731c0 0x7fd4e019c780 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 >> 172.21.15.66:0/2738214554 conn(0x7fd4e00fb610 msgr2=0x7fd4e00fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 shutdown_connections 2023-07-21T22:31:55.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.336+0000 7fd4e87ff700 1 -- 172.21.15.66:0/2738214554 wait complete. 2023-07-21T22:31:55.402 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:31:55.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489520 5.44 MB/sec execute 442 sec latency 4974.741 ms 2023-07-21T22:31:55.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 -- 172.21.15.66:0/3060368458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40fe880 msgr2=0x7f8be40fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 --2- 172.21.15.66:0/3060368458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40fe880 0x7f8be40fec50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009990 tx=0x7f8bd4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 -- 172.21.15.66:0/3060368458 shutdown_connections 2023-07-21T22:31:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 --2- 172.21.15.66:0/3060368458 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 0x7f8be410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 --2- 172.21.15.66:0/3060368458 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40fe880 0x7f8be40fec50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 --2- 172.21.15.66:0/3060368458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be40ff190 0x7f8be410bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 -- 172.21.15.66:0/3060368458 >> 172.21.15.66:0/3060368458 conn(0x7f8be40fa4b0 msgr2=0x7f8be40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 -- 172.21.15.66:0/3060368458 shutdown_connections 2023-07-21T22:31:55.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.573+0000 7f8be9dff700 1 -- 172.21.15.66:0/3060368458 wait complete. 2023-07-21T22:31:55.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.574+0000 7f8be9dff700 1 Processor -- start 2023-07-21T22:31:55.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.574+0000 7f8be9dff700 1 -- start start 2023-07-21T22:31:55.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be9dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be40fe880 0x7f8be4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be9dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40ff190 0x7f8be4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be9dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 0x7f8be419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be9dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be419ce20 con 0x7f8be410c430 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be9dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be419cf90 con 0x7f8be40fe880 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be9dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be419d150 con 0x7f8be40ff190 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 0x7f8be419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40ff190 0x7f8be4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40ff190 0x7f8be4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38096/0 (socket says 172.21.15.66:38096) 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be2ffd700 1 -- 172.21.15.66:0/1804942125 learned_addr learned my addr 172.21.15.66:0/1804942125 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be37fe700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be40fe880 0x7f8be4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be3fff700 1 -- 172.21.15.66:0/1804942125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40ff190 msgr2=0x7f8be4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be3fff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40ff190 0x7f8be4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be3fff700 1 -- 172.21.15.66:0/1804942125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be40fe880 msgr2=0x7f8be4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be3fff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be40fe880 0x7f8be4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.575+0000 7f8be3fff700 1 -- 172.21.15.66:0/1804942125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40095f0 con 0x7f8be410c430 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.576+0000 7f8be3fff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 0x7f8be419c7e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8be4100a60 tx=0x7f8bdc00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.576+0000 7f8be0ff9700 1 -- 172.21.15.66:0/1804942125 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc018070 con 0x7f8be410c430 2023-07-21T22:31:55.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.576+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be419d3d0 con 0x7f8be410c430 2023-07-21T22:31:55.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.576+0000 7f8be0ff9700 1 -- 172.21.15.66:0/1804942125 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bdc00f040 con 0x7f8be410c430 2023-07-21T22:31:55.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.576+0000 7f8be0ff9700 1 -- 172.21.15.66:0/1804942125 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00ccd0 con 0x7f8be410c430 2023-07-21T22:31:55.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.576+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be41a4c70 con 0x7f8be410c430 2023-07-21T22:31:55.625 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.579+0000 7f8be0ff9700 1 -- 172.21.15.66:0/1804942125 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8bdc003a00 con 0x7f8be410c430 2023-07-21T22:31:55.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 Processor -- start 2023-07-21T22:31:55.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8be0ff9700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 -- start start 2023-07-21T22:31:55.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8be0ff9700 1 -- 172.21.15.66:0/1804942125 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8bdc0ccbd0 con 0x7f8be410c430 2023-07-21T22:31:55.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa34398580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa34393350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa34393da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa34298cb0 con 0x55aa34398ac0 2023-07-21T22:31:55.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa342eba40 con 0x55aa343981b0 2023-07-21T22:31:55.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.581+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa342e4ed0 con 0x55aa34393930 2023-07-21T22:31:55.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa34398580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa34398580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34606/0 (socket says 172.21.15.66:34606) 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 -- 172.21.15.66:0/1944625466 learned_addr learned my addr 172.21.15.66:0/1944625466 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa34393350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be3fff700 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa34393da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 -- 172.21.15.66:0/1944625466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 msgr2=0x55aa34393da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa34393da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 -- 172.21.15.66:0/1944625466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 msgr2=0x55aa34393350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa34393350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 -- 172.21.15.66:0/1944625466 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa34112de0 con 0x55aa343981b0 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa34393350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8be4199450 tx=0x7f8bd80139e0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.582+0000 7f8be37fe700 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa34398580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4005ea0 tx=0x7f8bd4005ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68d6307968b9e367 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bc9ffb700 1 -- 172.21.15.66:0/1944625466 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd401d070 con 0x55aa343981b0 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bc9ffb700 1 -- 172.21.15.66:0/1944625466 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd401e040 con 0x55aa343981b0 2023-07-21T22:31:55.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bc9ffb700 1 -- 172.21.15.66:0/1944625466 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4004840 con 0x55aa343981b0 2023-07-21T22:31:55.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/1944625466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 msgr2=0x55aa34398580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa34398580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4005ea0 tx=0x7f8bd4005ed0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/1944625466 shutdown_connections 2023-07-21T22:31:55.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa34398580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa34393da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/1944625466 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa34393350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.583+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/1944625466 >> 172.21.15.66:0/1944625466 conn(0x55aa34388ef0 msgr2=0x55aa3438e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.584+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/1944625466 shutdown_connections 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.584+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/1944625466 wait complete. 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.584+0000 7f8bfd6ebb80 1 -- start start 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.584+0000 7f8bfd6ebb80 1 Processor -- start 2023-07-21T22:31:55.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 0x55aa3439fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa343981b0 0x55aa3439ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34398ac0 0x55aa3439d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa34284ac0 con 0x55aa34393930 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa342847c0 con 0x55aa34398ac0 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa34168920 con 0x55aa343981b0 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8bfd6ebb80 10 client.0 _ll_register_callbacks cb 0x55aa34282760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:55.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 0x55aa3439fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa343981b0 0x55aa3439ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 0x55aa3439fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48882/0 (socket says 172.21.15.66:48882) 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be37fe700 1 -- 172.21.15.66:0/2211889501 learned_addr learned my addr 172.21.15.66:0/2211889501 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be3fff700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34398ac0 0x55aa3439d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be37fe700 1 -- 172.21.15.66:0/2211889501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa343981b0 msgr2=0x55aa3439ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be37fe700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa343981b0 0x55aa3439ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.585+0000 7f8be37fe700 1 -- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34398ac0 msgr2=0x55aa3439d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8be37fe700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34398ac0 0x55aa3439d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8be37fe700 1 -- 172.21.15.66:0/2211889501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be410f390 con 0x55aa34393930 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8be3fff700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34398ac0 0x55aa3439d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8be37fe700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 0x55aa3439fc30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40098e0 tx=0x7f8bd400fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167bf0b56c00cdae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2211889501 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd401d040 con 0x55aa34393930 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa3423e4f0 con 0x55aa34393930 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2211889501 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8bd400fe20 con 0x55aa34393930 2023-07-21T22:31:55.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2211889501 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bd4005980 con 0x55aa34393930 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bfd6ebb80 10 client.16851 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.586+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55aa340fbc60 con 0x55aa34393930 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2211889501 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa3423e4f0 con 0x55aa34393930 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2211889501 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8bd4004ae0 con 0x55aa34393930 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bfd6ebb80 10 client.16851 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bfd6ebb80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:55.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bfd6ebb80 20 client.16851 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa343acf20 0x55aa343af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bfd6ebb80 4 client.16851 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:31:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa342ddeb0 con 0x55aa343acf20 2023-07-21T22:31:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.587+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa343acf20 0x55aa343af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.588+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa343acf20 0x55aa343af330 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2361bebd3f0ec46 server_cookie=99c6cbd2adc64d46 in_seq=0 out_seq=0 2023-07-21T22:31:55.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.588+0000 7f8bc2ffd700 10 client.16851 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2211889501 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa342ddeb0 con 0x55aa343acf20 2023-07-21T22:31:55.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bc2ffd700 10 client.16851 handle_command_reply: tid=0 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 client.16851 shutdown 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 msgr2=0x55aa3439fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 0x55aa3439fc30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40098e0 tx=0x7f8bd400fb10 comp rx=0 tx=0).stop 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 shutdown_connections 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34398ac0 0x55aa3439d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa343acf20 0x55aa343af330 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34393930 0x55aa3439fc30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2211889501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa343981b0 0x55aa3439ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bc2ffd700 0 client.16851 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 >> 172.21.15.66:0/2211889501 conn(0x55aa34388ef0 msgr2=0x55aa3438e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 shutdown_connections 2023-07-21T22:31:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.589+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2211889501 wait complete. 2023-07-21T22:31:55.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.590+0000 7f8bfd6ebb80 20 client.16851 trim_cache size 0 max 16384 2023-07-21T22:31:55.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.607+0000 7f8bfd6ebb80 1 Processor -- start 2023-07-21T22:31:55.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.607+0000 7f8bfd6ebb80 1 -- start start 2023-07-21T22:31:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.607+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa3439f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa3439c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa3439cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3438b010 con 0x55aa34398ac0 2023-07-21T22:31:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa34394400 con 0x55aa343981b0 2023-07-21T22:31:55.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa3439dae0 con 0x55aa34393930 2023-07-21T22:31:55.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa3439cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa3439f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa3439c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa3439f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38118/0 (socket says 172.21.15.66:38118) 2023-07-21T22:31:55.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 -- 172.21.15.66:0/3284113111 learned_addr learned my addr 172.21.15.66:0/3284113111 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 -- 172.21.15.66:0/3284113111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 msgr2=0x55aa3439c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa3439c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 -- 172.21.15.66:0/3284113111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 msgr2=0x55aa3439cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa3439cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.608+0000 7f8be37fe700 1 -- 172.21.15.66:0/3284113111 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa34139250 con 0x55aa34393930 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8be37fe700 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa3439f760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4004fb0 tx=0x7f8bd4005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f080bbc4e54fdec7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bca7fc700 1 -- 172.21.15.66:0/3284113111 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400e5e0 con 0x55aa34393930 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bca7fc700 1 -- 172.21.15.66:0/3284113111 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd4005950 con 0x55aa34393930 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bca7fc700 1 -- 172.21.15.66:0/3284113111 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd40055b0 con 0x55aa34393930 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/3284113111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 msgr2=0x55aa3439f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa3439f760 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4004fb0 tx=0x7f8bd4005e80 comp rx=0 tx=0).stop 2023-07-21T22:31:55.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/3284113111 shutdown_connections 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa34398ac0 0x55aa3439cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34393930 0x55aa3439f760 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/3284113111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa343981b0 0x55aa3439c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.609+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/3284113111 >> 172.21.15.66:0/3284113111 conn(0x55aa34388ef0 msgr2=0x55aa34391aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.610+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/3284113111 shutdown_connections 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.610+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/3284113111 wait complete. 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.610+0000 7f8bfd6ebb80 1 -- start start 2023-07-21T22:31:55.659 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.610+0000 7f8bfd6ebb80 1 Processor -- start 2023-07-21T22:31:55.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34393930 0x55aa343a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa343981b0 0x55aa343a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 0x55aa343f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa343fa2f0 con 0x55aa343981b0 2023-07-21T22:31:55.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa343fa460 con 0x55aa34393930 2023-07-21T22:31:55.660 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa343fa620 con 0x55aa34398ac0 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8bfd6ebb80 10 client.0 _ll_register_callbacks cb 0x7f8bdc009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34393930 0x55aa343a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be2ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa343981b0 0x55aa343a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 0x55aa343f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34393930 0x55aa343a26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:34632/0 (socket says 172.21.15.66:34632) 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be37fe700 1 -- 172.21.15.66:0/2303929261 learned_addr learned my addr 172.21.15.66:0/2303929261 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:31:55.661 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 0x55aa343f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38128/0 (socket says 172.21.15.66:38128) 2023-07-21T22:31:55.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.611+0000 7f8be2ffd700 1 -- 172.21.15.66:0/2303929261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 msgr2=0x55aa343f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 0x55aa343f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8be2ffd700 1 -- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34393930 msgr2=0x55aa343a26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:31:55.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34393930 0x55aa343a26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8be2ffd700 1 -- 172.21.15.66:0/2303929261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa340fbc60 con 0x55aa343981b0 2023-07-21T22:31:55.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8be3fff700 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 0x55aa343f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:31:55.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8be2ffd700 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa343981b0 0x55aa343a2c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55aa3439e3f0 tx=0x7f8bd80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97814ca1f59973c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:31:55.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:31:55.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2303929261 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8026040 con 0x55aa343981b0 2023-07-21T22:31:55.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2303929261 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8bd8010430 con 0x55aa343981b0 2023-07-21T22:31:55.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bd401e040 con 0x55aa343981b0 2023-07-21T22:31:55.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2303929261 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bd8018040 con 0x55aa343981b0 2023-07-21T22:31:55.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bfd6ebb80 10 client.16857 fetch_fsmap learned FSMap version 41 2023-07-21T22:31:55.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.612+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55aa341112b0 con 0x55aa343981b0 2023-07-21T22:31:55.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2303929261 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bd401e040 con 0x55aa343981b0 2023-07-21T22:31:55.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2303929261 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8bd8031030 con 0x55aa343981b0 2023-07-21T22:31:55.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bfd6ebb80 10 client.16857 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:31:55.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bfd6ebb80 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:31:55.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bfd6ebb80 20 client.16857 populate_metadata read hostname 'smithi066' 2023-07-21T22:31:55.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa34401de0 0x55aa344041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:31:55.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bfd6ebb80 4 client.16857 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:31:55.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aa342a0050 con 0x55aa34401de0 2023-07-21T22:31:55.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.613+0000 7f8be37fe700 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa34401de0 0x55aa344041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:31:55.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8be37fe700 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa34401de0 0x55aa344041f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e5315ffc86baa3 server_cookie=c4fe298d00be8450 in_seq=0 out_seq=0 2023-07-21T22:31:55.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bc2ffd700 10 client.16857 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bc2ffd700 1 -- 172.21.15.66:0/2303929261 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aa342a0050 con 0x55aa34401de0 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bc2ffd700 10 client.16857 handle_command_reply: tid=0 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 client.16857 shutdown 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa343981b0 msgr2=0x55aa343a2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa343981b0 0x55aa343a2c20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55aa3439e3f0 tx=0x7f8bd80047c0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 shutdown_connections 2023-07-21T22:31:55.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aa34393930 0x55aa343a26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55aa34401de0 0x55aa344041f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55aa34398ac0 0x55aa343f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 --2- 172.21.15.66:0/2303929261 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55aa343981b0 0x55aa343a2c20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bc2ffd700 0 client.16857 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 >> 172.21.15.66:0/2303929261 conn(0x55aa34388ef0 msgr2=0x55aa3439c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 shutdown_connections 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.614+0000 7f8bfd6ebb80 1 -- 172.21.15.66:0/2303929261 wait complete. 2023-07-21T22:31:55.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8bfd6ebb80 20 client.16857 trim_cache size 0 max 16384 2023-07-21T22:31:55.674 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:31:55.674 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:31:55.674 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:31:55.674 INFO:teuthology.orchestra.run.smithi066.stdout: "b2a48f23-b933-41fc-bda9-5d9512b32f98": { 2023-07-21T22:31:55.674 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:31:55.675 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "b2a48f23-b933-41fc-bda9-5d9512b32f98", 2023-07-21T22:31:55.675 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:31:55.675 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:31:55.675 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:31:55.675 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:31:55.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8bcc0a12c0 msgr2=0x7f8bcc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8be4199450 tx=0x7f8bd80139e0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 msgr2=0x7f8be419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:31:55.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 0x7f8be419c7e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8be4100a60 tx=0x7f8bdc00ef90 comp rx=0 tx=0).stop 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 shutdown_connections 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8be40fe880 0x7f8be4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f8bcc0a12c0 0x7f8bcc0a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8be410c430 0x7f8be419c7e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 --2- 172.21.15.66:0/1804942125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8be40ff190 0x7f8be4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 >> 172.21.15.66:0/1804942125 conn(0x7f8be40fa4b0 msgr2=0x7f8be410d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:31:55.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 shutdown_connections 2023-07-21T22:31:55.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:31:55.615+0000 7f8be9dff700 1 -- 172.21.15.66:0/1804942125 wait complete. 2023-07-21T22:31:55.679 INFO:tasks.cephfs.filesystem:scrub status for tag:b2a48f23-b933-41fc-bda9-5d9512b32f98 - {'path': '/', 'tag': 'b2a48f23-b933-41fc-bda9-5d9512b32f98', 'options': 'recursive,force'} 2023-07-21T22:31:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: pgmap v324: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 3.3 MiB/s wr, 73 op/s 2023-07-21T22:31:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:31:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:31:56.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:31:56.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:31:56.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:31:56.374 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.374 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:55 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: pgmap v324: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 3.3 MiB/s wr, 73 op/s 2023-07-21T22:31:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:31:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:31:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:31:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:31:56.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:31:56.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:55 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: pgmap v324: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 3.3 MiB/s wr, 73 op/s 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:31:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:31:56.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:55 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:56.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489520 5.42 MB/sec execute 443 sec latency 1131.836 ms 2023-07-21T22:31:57.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:56 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:31:57.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:56 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:56 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:31:57.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:56 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:57.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:56 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:31:57.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:56 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:31:57.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489520 5.41 MB/sec execute 444 sec latency 2131.919 ms 2023-07-21T22:31:58.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:57 smithi104 ceph-mon[129021]: pgmap v325: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.6 MiB/s wr, 42 op/s 2023-07-21T22:31:58.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:57 smithi066 ceph-mon[126379]: pgmap v325: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.6 MiB/s wr, 42 op/s 2023-07-21T22:31:58.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:57 smithi149 ceph-mon[129268]: pgmap v325: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.6 MiB/s wr, 42 op/s 2023-07-21T22:31:58.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489520 5.40 MB/sec execute 445 sec latency 3132.078 ms 2023-07-21T22:31:59.564 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489520 5.39 MB/sec execute 446 sec latency 4132.223 ms 2023-07-21T22:32:00.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:31:59 smithi104 ceph-mon[129021]: pgmap v326: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.6 MiB/s wr, 42 op/s 2023-07-21T22:32:00.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:31:59 smithi066 ceph-mon[126379]: pgmap v326: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.6 MiB/s wr, 42 op/s 2023-07-21T22:32:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:31:59 smithi149 ceph-mon[129268]: pgmap v326: 129 pgs: 129 active+clean; 458 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 2.6 MiB/s wr, 42 op/s 2023-07-21T22:32:00.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489638 5.38 MB/sec execute 447 sec latency 4773.951 ms 2023-07-21T22:32:01.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:00 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:32:01.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:00 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:32:01.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:00 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:32:01.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489771 5.37 MB/sec execute 448 sec latency 639.003 ms 2023-07-21T22:32:02.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:01 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:32:02.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:01 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:32:02.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:01 smithi104 ceph-mon[129021]: pgmap v327: 129 pgs: 129 active+clean; 459 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.1 MiB/s wr, 67 op/s 2023-07-21T22:32:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:01 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:32:02.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:01 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:32:02.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:01 smithi066 ceph-mon[126379]: pgmap v327: 129 pgs: 129 active+clean; 459 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.1 MiB/s wr, 67 op/s 2023-07-21T22:32:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:01 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:32:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:01 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:32:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:01 smithi149 ceph-mon[129268]: pgmap v327: 129 pgs: 129 active+clean; 459 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.1 MiB/s wr, 67 op/s 2023-07-21T22:32:02.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489771 5.36 MB/sec execute 449 sec latency 1639.129 ms 2023-07-21T22:32:03.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489771 5.34 MB/sec execute 450 sec latency 2639.252 ms 2023-07-21T22:32:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:03 smithi104 ceph-mon[129021]: pgmap v328: 129 pgs: 129 active+clean; 459 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 1.3 MiB/s wr, 45 op/s 2023-07-21T22:32:04.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:03 smithi066 ceph-mon[126379]: pgmap v328: 129 pgs: 129 active+clean; 459 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 1.3 MiB/s wr, 45 op/s 2023-07-21T22:32:04.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:03 smithi149 ceph-mon[129268]: pgmap v328: 129 pgs: 129 active+clean; 459 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 1.3 MiB/s wr, 45 op/s 2023-07-21T22:32:04.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489771 5.33 MB/sec execute 451 sec latency 3639.420 ms 2023-07-21T22:32:05.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489935 5.32 MB/sec execute 452 sec latency 4281.661 ms 2023-07-21T22:32:06.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:05 smithi104 ceph-mon[129021]: pgmap v329: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.1 MiB/s wr, 89 op/s 2023-07-21T22:32:06.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:05 smithi066 ceph-mon[126379]: pgmap v329: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.1 MiB/s wr, 89 op/s 2023-07-21T22:32:06.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:05 smithi149 ceph-mon[129268]: pgmap v329: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.1 MiB/s wr, 89 op/s 2023-07-21T22:32:06.565 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489935 5.31 MB/sec execute 453 sec latency 1315.791 ms 2023-07-21T22:32:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:07 smithi104 ceph-mon[129021]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T22:32:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:07 smithi066 ceph-mon[126379]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T22:32:07.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:07 smithi149 ceph-mon[129268]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T22:32:07.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489935 5.30 MB/sec execute 454 sec latency 2315.903 ms 2023-07-21T22:32:08.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:08 smithi104 ceph-mon[129021]: pgmap v331: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2023-07-21T22:32:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:08 smithi066 ceph-mon[126379]: pgmap v331: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2023-07-21T22:32:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:08 smithi149 ceph-mon[129268]: pgmap v331: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2023-07-21T22:32:08.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489935 5.29 MB/sec execute 455 sec latency 3316.050 ms 2023-07-21T22:32:09.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489935 5.27 MB/sec execute 456 sec latency 4316.119 ms 2023-07-21T22:32:10.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:10 smithi104 ceph-mon[129021]: pgmap v332: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2023-07-21T22:32:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:10 smithi066 ceph-mon[126379]: pgmap v332: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2023-07-21T22:32:10.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:10 smithi149 ceph-mon[129268]: pgmap v332: 129 pgs: 129 active+clean; 460 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.6 MiB/s wr, 81 op/s 2023-07-21T22:32:10.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 489968 5.26 MB/sec execute 457 sec latency 4945.701 ms 2023-07-21T22:32:11.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 490764 5.26 MB/sec execute 458 sec latency 647.935 ms 2023-07-21T22:32:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:12 smithi104 ceph-mon[129021]: pgmap v333: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2023-07-21T22:32:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:12 smithi066 ceph-mon[126379]: pgmap v333: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2023-07-21T22:32:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:12 smithi149 ceph-mon[129268]: pgmap v333: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2023-07-21T22:32:12.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492084 5.26 MB/sec execute 459 sec latency 23.801 ms 2023-07-21T22:32:13.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492644 5.26 MB/sec execute 460 sec latency 846.323 ms 2023-07-21T22:32:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:13 smithi104 ceph-mon[129021]: pgmap v334: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2023-07-21T22:32:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:13 smithi104 ceph-mon[129021]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T22:32:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:13 smithi066 ceph-mon[126379]: pgmap v334: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2023-07-21T22:32:14.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:13 smithi066 ceph-mon[126379]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T22:32:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:13 smithi149 ceph-mon[129268]: pgmap v334: 129 pgs: 129 active+clean; 451 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.4 MiB/s wr, 83 op/s 2023-07-21T22:32:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:13 smithi149 ceph-mon[129268]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T22:32:14.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492644 5.24 MB/sec execute 461 sec latency 1846.464 ms 2023-07-21T22:32:15.566 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492799 5.24 MB/sec execute 462 sec latency 2469.972 ms 2023-07-21T22:32:16.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:15 smithi104 ceph-mon[129021]: pgmap v336: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1021 KiB/s wr, 88 op/s 2023-07-21T22:32:16.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:15 smithi066 ceph-mon[126379]: pgmap v336: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1021 KiB/s wr, 88 op/s 2023-07-21T22:32:16.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:15 smithi149 ceph-mon[129268]: pgmap v336: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1021 KiB/s wr, 88 op/s 2023-07-21T22:32:16.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492799 5.22 MB/sec execute 463 sec latency 1200.110 ms 2023-07-21T22:32:17.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492799 5.21 MB/sec execute 464 sec latency 2200.233 ms 2023-07-21T22:32:18.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:17 smithi104 ceph-mon[129021]: pgmap v337: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 910 KiB/s wr, 78 op/s 2023-07-21T22:32:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:17 smithi066 ceph-mon[126379]: pgmap v337: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 910 KiB/s wr, 78 op/s 2023-07-21T22:32:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:17 smithi149 ceph-mon[129268]: pgmap v337: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 910 KiB/s wr, 78 op/s 2023-07-21T22:32:18.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492799 5.20 MB/sec execute 465 sec latency 3200.373 ms 2023-07-21T22:32:19.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 492799 5.19 MB/sec execute 466 sec latency 4200.520 ms 2023-07-21T22:32:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:19 smithi104 ceph-mon[129021]: pgmap v338: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 910 KiB/s wr, 78 op/s 2023-07-21T22:32:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:19 smithi066 ceph-mon[126379]: pgmap v338: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 910 KiB/s wr, 78 op/s 2023-07-21T22:32:20.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:19 smithi149 ceph-mon[129268]: pgmap v338: 129 pgs: 129 active+clean; 453 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 910 KiB/s wr, 78 op/s 2023-07-21T22:32:20.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 493639 5.19 MB/sec execute 467 sec latency 4826.249 ms 2023-07-21T22:32:21.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 494664 5.19 MB/sec execute 468 sec latency 12.624 ms 2023-07-21T22:32:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:21 smithi104 ceph-mon[129021]: pgmap v339: 129 pgs: 129 active+clean; 455 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 607 KiB/s wr, 59 op/s 2023-07-21T22:32:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:21 smithi066 ceph-mon[126379]: pgmap v339: 129 pgs: 129 active+clean; 455 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 607 KiB/s wr, 59 op/s 2023-07-21T22:32:22.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:21 smithi149 ceph-mon[129268]: pgmap v339: 129 pgs: 129 active+clean; 455 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 607 KiB/s wr, 59 op/s 2023-07-21T22:32:22.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 496342 5.20 MB/sec execute 469 sec latency 18.653 ms 2023-07-21T22:32:23.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 497717 5.21 MB/sec execute 470 sec latency 75.416 ms 2023-07-21T22:32:24.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:23 smithi104 ceph-mon[129021]: pgmap v340: 129 pgs: 129 active+clean; 455 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 607 KiB/s wr, 59 op/s 2023-07-21T22:32:24.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:23 smithi066 ceph-mon[126379]: pgmap v340: 129 pgs: 129 active+clean; 455 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 607 KiB/s wr, 59 op/s 2023-07-21T22:32:24.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:23 smithi149 ceph-mon[129268]: pgmap v340: 129 pgs: 129 active+clean; 455 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 607 KiB/s wr, 59 op/s 2023-07-21T22:32:24.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 498826 5.21 MB/sec execute 471 sec latency 16.070 ms 2023-07-21T22:32:25.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 500274 5.21 MB/sec execute 472 sec latency 22.600 ms 2023-07-21T22:32:25.680 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:32:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 -- 172.21.15.66:0/1104108565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc40ffad0 msgr2=0x7f9dc40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/1104108565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc40ffad0 0x7f9dc40fff40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009990 tx=0x7f9db8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 -- 172.21.15.66:0/1104108565 shutdown_connections 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/1104108565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 0x7f9dc410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/1104108565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc4105ae0 0x7f9dc4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/1104108565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc40ffad0 0x7f9dc40fff40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 -- 172.21.15.66:0/1104108565 >> 172.21.15.66:0/1104108565 conn(0x7f9dc40fb5f0 msgr2=0x7f9dc40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 -- 172.21.15.66:0/1104108565 shutdown_connections 2023-07-21T22:32:25.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.871+0000 7f9dc9bff700 1 -- 172.21.15.66:0/1104108565 wait complete. 2023-07-21T22:32:25.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.872+0000 7f9dc9bff700 1 Processor -- start 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.872+0000 7f9dc9bff700 1 -- start start 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc9bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 0x7f9dc4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc9bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 0x7f9dc4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc9bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc4105ae0 0x7f9dc419c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc9bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc419cdb0 con 0x7f9dc40ffad0 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc9bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc419cf20 con 0x7f9dc4105ae0 2023-07-21T22:32:25.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc9bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc419d0e0 con 0x7f9dc4100480 2023-07-21T22:32:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 0x7f9dc4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 0x7f9dc4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53028/0 (socket says 172.21.15.66:53028) 2023-07-21T22:32:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc37fe700 1 -- 172.21.15.66:0/3138523427 learned_addr learned my addr 172.21.15.66:0/3138523427 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc3fff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc4105ae0 0x7f9dc419c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.873+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 0x7f9dc4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/3138523427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc4105ae0 msgr2=0x7f9dc419c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc4105ae0 0x7f9dc419c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 msgr2=0x7f9dc4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 0x7f9dc4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/3138523427 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db80095f0 con 0x7f9dc4100480 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 0x7f9dc4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.874+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 0x7f9dc4198360 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009990 tx=0x7f9db8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.875+0000 7f9dc0ff9700 1 -- 172.21.15.66:0/3138523427 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db801d070 con 0x7f9dc4100480 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.875+0000 7f9dc0ff9700 1 -- 172.21.15.66:0/3138523427 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db8021410 con 0x7f9dc4100480 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.875+0000 7f9dc0ff9700 1 -- 172.21.15.66:0/3138523427 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8017650 con 0x7f9dc4100480 2023-07-21T22:32:25.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.875+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc419d360 con 0x7f9dc4100480 2023-07-21T22:32:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.875+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc419d5e0 con 0x7f9dc4100480 2023-07-21T22:32:25.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.879+0000 7f9dc0ff9700 1 -- 172.21.15.66:0/3138523427 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9db8021810 con 0x7f9dc4100480 2023-07-21T22:32:25.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 Processor -- start 2023-07-21T22:32:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 -- start start 2023-07-21T22:32:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018f1b0 0x55fe5018f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9dc0ff9700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9dac0a12c0 0x7f9dac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 0x55fe5018a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5018ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5008fcb0 con 0x55fe5018a930 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe500e2a40 con 0x55fe5018fac0 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.880+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe500dbed0 con 0x55fe5018f1b0 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc0ff9700 1 -- 172.21.15.66:0/3138523427 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9db80ce280 con 0x7f9dc4100480 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 0x55fe5018a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 0x55fe5018a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45022/0 (socket says 172.21.15.66:45022) 2023-07-21T22:32:25.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1942284977 learned_addr learned my addr 172.21.15.66:0/1942284977 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc3fff700 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018f1b0 0x55fe5018f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9dac0a12c0 0x7f9dac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc3fff700 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5018ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1942284977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018f1b0 msgr2=0x55fe5018f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018f1b0 0x55fe5018f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1942284977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 msgr2=0x55fe5018ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5018ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1942284977 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4ff09de0 con 0x55fe5018fac0 2023-07-21T22:32:25.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc3fff700 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5018ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:32:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 0x55fe5018a350 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9db8097a50 tx=0x7f9db8097410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a6ab03b9b0de4d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.881+0000 7f9da9ffb700 1 -- 172.21.15.66:0/1942284977 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80bf580 con 0x55fe5018fac0 2023-07-21T22:32:25.933 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9da9ffb700 1 -- 172.21.15.66:0/1942284977 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db809afe0 con 0x55fe5018fac0 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9da9ffb700 1 -- 172.21.15.66:0/1942284977 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8021610 con 0x55fe5018fac0 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1942284977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 msgr2=0x55fe5018a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9dac0a12c0 0x7f9dac0a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9db4011550 tx=0x7f9db4009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 0x55fe5018a350 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9db8097a50 tx=0x7f9db8097410 comp rx=0 tx=0).stop 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1942284977 shutdown_connections 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5018ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018f1b0 0x55fe5018f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.934 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1942284977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018fac0 0x55fe5018a350 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1942284977 >> 172.21.15.66:0/1942284977 conn(0x55fe5017fef0 msgr2=0x55fe50185830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1942284977 shutdown_connections 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.882+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1942284977 wait complete. 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.883+0000 7f9ddd524b80 1 -- start start 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.883+0000 7f9ddd524b80 1 Processor -- start 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 0x55fe50196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018f1b0 0x55fe50193dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50194360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5007bac0 con 0x55fe5018f1b0 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5007b7c0 con 0x55fe5018a930 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe4ff5f920 con 0x55fe5018fac0 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9ddd524b80 10 client.0 _ll_register_callbacks cb 0x55fe50079760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 0x55fe50196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 0x55fe50196bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45030/0 (socket says 172.21.15.66:45030) 2023-07-21T22:32:25.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 -- 172.21.15.66:0/2854965825 learned_addr learned my addr 172.21.15.66:0/2854965825 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018f1b0 0x55fe50193dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc3fff700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50194360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 -- 172.21.15.66:0/2854965825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 msgr2=0x55fe50194360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50194360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 -- 172.21.15.66:0/2854965825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018f1b0 msgr2=0x55fe50193dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018f1b0 0x55fe50193dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.884+0000 7f9dc37fe700 1 -- 172.21.15.66:0/2854965825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc419d360 con 0x55fe5018a930 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 0x55fe50196bb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9db400ed00 tx=0x7f9db40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a12cb6fa78b87f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9da2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9da2ffd700 1 -- 172.21.15.66:0/2854965825 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db400e3b0 con 0x55fe5018a930 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe500354f0 con 0x55fe5018a930 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9da2ffd700 1 -- 172.21.15.66:0/2854965825 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9db400fb60 con 0x55fe5018a930 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9da2ffd700 1 -- 172.21.15.66:0/2854965825 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db400fd20 con 0x55fe5018a930 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9ddd524b80 10 client.26927 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:25.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.885+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fe4fef2c60 con 0x55fe5018a930 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9da2ffd700 1 -- 172.21.15.66:0/2854965825 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe500354f0 con 0x55fe5018a930 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9da2ffd700 1 -- 172.21.15.66:0/2854965825 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9db4024020 con 0x55fe5018a930 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9ddd524b80 10 client.26927 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9ddd524b80 10 client.26927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9ddd524b80 20 client.26927 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501a40e0 0x55fe501a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9ddd524b80 4 client.26927 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:25.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe500d4eb0 con 0x55fe501a40e0 2023-07-21T22:32:25.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.886+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501a40e0 0x55fe501a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.887+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501a40e0 0x55fe501a64f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d591be91d0a937c3 server_cookie=b96a152ee186888 in_seq=0 out_seq=0 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.887+0000 7f9da2ffd700 10 client.26927 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.887+0000 7f9da2ffd700 1 -- 172.21.15.66:0/2854965825 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fe500d4eb0 con 0x55fe501a40e0 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.887+0000 7f9da2ffd700 10 client.26927 handle_command_reply: tid=0 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 client.26927 shutdown 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 msgr2=0x55fe50196bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 0x55fe50196bb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9db400ed00 tx=0x7f9db40047c0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 shutdown_connections 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018a930 0x55fe50196bb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501a40e0 0x55fe501a64f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50194360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/2854965825 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018f1b0 0x55fe50193dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9da2ffd700 0 client.26927 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:25.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 >> 172.21.15.66:0/2854965825 conn(0x55fe5017fef0 msgr2=0x55fe501853c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:25.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 shutdown_connections 2023-07-21T22:32:25.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 1 -- 172.21.15.66:0/2854965825 wait complete. 2023-07-21T22:32:25.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.888+0000 7f9ddd524b80 20 client.26927 trim_cache size 0 max 16384 2023-07-21T22:32:25.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 Processor -- start 2023-07-21T22:32:25.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 -- start start 2023-07-21T22:32:25.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5019b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50189fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe5018fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe501a0b70 con 0x55fe5018a930 2023-07-21T22:32:25.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5018b6d0 con 0x55fe5018f1b0 2023-07-21T22:32:25.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.908+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe50194e00 con 0x55fe5018fac0 2023-07-21T22:32:25.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe5018fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50189fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe5018fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45888/0 (socket says 172.21.15.66:45888) 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc3fff700 1 -- 172.21.15.66:0/1596171576 learned_addr learned my addr 172.21.15.66:0/1596171576 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1596171576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 msgr2=0x55fe5018fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe5018fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1596171576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 msgr2=0x55fe5019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1596171576 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe4ff30250 con 0x55fe5018f1b0 2023-07-21T22:32:25.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc3fff700 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe5018fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:25.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.909+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50189fb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9db80a3560 tx=0x7f9db809e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e076cd812d8422ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:25.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9daa7fc700 1 -- 172.21.15.66:0/1596171576 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80a33a0 con 0x55fe5018f1b0 2023-07-21T22:32:25.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9daa7fc700 1 -- 172.21.15.66:0/1596171576 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db8099120 con 0x55fe5018f1b0 2023-07-21T22:32:25.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9daa7fc700 1 -- 172.21.15.66:0/1596171576 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80bf3e0 con 0x55fe5018f1b0 2023-07-21T22:32:25.961 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1596171576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 msgr2=0x55fe50189fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50189fb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9db80a3560 tx=0x7f9db809e040 comp rx=0 tx=0).stop 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1596171576 shutdown_connections 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe5018fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe5019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1596171576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50189fb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1596171576 >> 172.21.15.66:0/1596171576 conn(0x55fe5017fef0 msgr2=0x55fe501898a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:25.962 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1596171576 shutdown_connections 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.910+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1596171576 wait complete. 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.911+0000 7f9ddd524b80 1 -- start start 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.911+0000 7f9ddd524b80 1 Processor -- start 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.911+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe50193960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.911+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50192570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9ddd524b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50192ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe50193f70 con 0x55fe5018a930 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe501940e0 con 0x55fe5018f1b0 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9ddd524b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe50192ff0 con 0x55fe5018fac0 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9ddd524b80 10 client.0 _ll_register_callbacks cb 0x7f9db400e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe50193960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50192570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe50193960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53068/0 (socket says 172.21.15.66:53068) 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50192ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc37fe700 1 -- 172.21.15.66:0/1348521237 learned_addr learned my addr 172.21.15.66:0/1348521237 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1348521237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 msgr2=0x55fe50192ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50192ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1348521237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 msgr2=0x55fe50193960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe50193960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 -- 172.21.15.66:0/1348521237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc007500 con 0x55fe5018f1b0 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe50193960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9dc2ffd700 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50192570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55fe50194dd0 tx=0x7f9db8097410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0221c3e7e53397d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:25.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.912+0000 7f9da2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9da2ffd700 1 -- 172.21.15.66:0/1348521237 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80cba50 con 0x55fe5018f1b0 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9db809afe0 con 0x55fe5018f1b0 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9da2ffd700 1 -- 172.21.15.66:0/1348521237 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9db80a33a0 con 0x55fe5018f1b0 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9da2ffd700 1 -- 172.21.15.66:0/1348521237 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db809ab20 con 0x55fe5018f1b0 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 10 client.26933 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fe4ff09de0 con 0x55fe5018f1b0 2023-07-21T22:32:25.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9da2ffd700 1 -- 172.21.15.66:0/1348521237 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9db809afe0 con 0x55fe5018f1b0 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9da2ffd700 1 -- 172.21.15.66:0/1348521237 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9db80cbd20 con 0x55fe5018f1b0 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 10 client.26933 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 10 client.26933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 20 client.26933 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501f8b70 0x55fe501faf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 4 client.26933 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.913+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe50097050 con 0x55fe501f8b70 2023-07-21T22:32:25.967 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.914+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501f8b70 0x55fe501faf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.914+0000 7f9da2ffd700 10 client.26933 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.914+0000 7f9dc37fe700 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501f8b70 0x55fe501faf80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0927a9abf18283d server_cookie=b284f3c1a32d3ebd in_seq=0 out_seq=0 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9da2ffd700 1 -- 172.21.15.66:0/1348521237 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe50097050 con 0x55fe501f8b70 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9da2ffd700 10 client.26933 handle_command_reply: tid=0 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 client.26933 shutdown 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 msgr2=0x55fe50192570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50192570 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x55fe50194dd0 tx=0x7f9db8097410 comp rx=0 tx=0).stop 2023-07-21T22:32:25.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 shutdown_connections 2023-07-21T22:32:25.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fe501f8b70 0x55fe501faf80 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fe5018fac0 0x55fe50192ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fe5018a930 0x55fe50193960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9da2ffd700 0 client.26933 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 --2- 172.21.15.66:0/1348521237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe5018f1b0 0x55fe50192570 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 >> 172.21.15.66:0/1348521237 conn(0x55fe5017fef0 msgr2=0x55fe501898a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 shutdown_connections 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 1 -- 172.21.15.66:0/1348521237 wait complete. 2023-07-21T22:32:25.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.915+0000 7f9ddd524b80 20 client.26933 trim_cache size 0 max 16384 2023-07-21T22:32:25.971 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:32:25.971 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:32:25.971 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:32:25.971 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:32:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9dac0a12c0 msgr2=0x7f9dac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9dac0a12c0 0x7f9dac0a3770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9db4011550 tx=0x7f9db4009260 comp rx=0 tx=0).stop 2023-07-21T22:32:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 msgr2=0x7f9dc4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 0x7f9dc4198360 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009990 tx=0x7f9db8009580 comp rx=0 tx=0).stop 2023-07-21T22:32:25.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 shutdown_connections 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9dc4105ae0 0x7f9dc419c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9dac0a12c0 0x7f9dac0a3770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9dc40ffad0 0x7f9dc4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 --2- 172.21.15.66:0/3138523427 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dc4100480 0x7f9dc4198360 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 >> 172.21.15.66:0/3138523427 conn(0x7f9dc40fb5f0 msgr2=0x7f9dc40fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 shutdown_connections 2023-07-21T22:32:25.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:25.916+0000 7f9dc9bff700 1 -- 172.21.15.66:0/3138523427 wait complete. 2023-07-21T22:32:25.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:32:25.989 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:32:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 -- 172.21.15.66:0/3425018349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 msgr2=0x7fdd50101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3425018349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50101070 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c009990 tx=0x7fdd4c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 -- 172.21.15.66:0/3425018349 shutdown_connections 2023-07-21T22:32:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3425018349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50101070 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3425018349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 0x7fdd5010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3425018349 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd501015b0 0x7fdd50109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 -- 172.21.15.66:0/3425018349 >> 172.21.15.66:0/3425018349 conn(0x7fdd500fa490 msgr2=0x7fdd500fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 -- 172.21.15.66:0/3425018349 shutdown_connections 2023-07-21T22:32:26.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.177+0000 7fdd577ff700 1 -- 172.21.15.66:0/3425018349 wait complete. 2023-07-21T22:32:26.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.178+0000 7fdd577ff700 1 Processor -- start 2023-07-21T22:32:26.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 -- start start 2023-07-21T22:32:26.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd501015b0 0x7fdd50198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 0x7fdd5019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5019cde0 con 0x7fdd501015b0 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5019cf50 con 0x7fdd50100ca0 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.179+0000 7fdd577ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5019d110 con 0x7fdd5010a070 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd5559b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd55d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 0x7fdd5019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd55d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 0x7fdd5019c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45918/0 (socket says 172.21.15.66:45918) 2023-07-21T22:32:26.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd55d9c700 1 -- 172.21.15.66:0/3761639542 learned_addr learned my addr 172.21.15.66:0/3761639542 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:26.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd54d9a700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd501015b0 0x7fdd50198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd5559b700 1 -- 172.21.15.66:0/3761639542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 msgr2=0x7fdd5019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd5559b700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 0x7fdd5019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd5559b700 1 -- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd501015b0 msgr2=0x7fdd50198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd5559b700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd501015b0 0x7fdd50198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.180+0000 7fdd5559b700 1 -- 172.21.15.66:0/3761639542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c0095f0 con 0x7fdd50100ca0 2023-07-21T22:32:26.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.181+0000 7fdd5559b700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50197e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.181+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c01d070 con 0x7fdd50100ca0 2023-07-21T22:32:26.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.181+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c021410 con 0x7fdd50100ca0 2023-07-21T22:32:26.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.181+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c017650 con 0x7fdd50100ca0 2023-07-21T22:32:26.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.181+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5019d390 con 0x7fdd50100ca0 2023-07-21T22:32:26.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.181+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5019d610 con 0x7fdd50100ca0 2023-07-21T22:32:26.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.182+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd50067300 con 0x7fdd50100ca0 2023-07-21T22:32:26.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.184+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdd4c0217f0 con 0x7fdd50100ca0 2023-07-21T22:32:26.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.186+0000 7fdd467fc700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdd3c0a1300 0x7fdd3c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.186+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdd4c0ce7d0 con 0x7fdd50100ca0 2023-07-21T22:32:26.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.186+0000 7fdd54d9a700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdd3c0a1300 0x7fdd3c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.187+0000 7fdd54d9a700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdd3c0a1300 0x7fdd3c0a37b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd50199410 tx=0x7fdd40009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.188+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdd4c097870 con 0x7fdd50100ca0 2023-07-21T22:32:26.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:25 smithi104 ceph-mon[129021]: pgmap v341: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-07-21T22:32:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:25 smithi066 ceph-mon[126379]: pgmap v341: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-07-21T22:32:26.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:25 smithi149 ceph-mon[129268]: pgmap v341: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 130 op/s 2023-07-21T22:32:26.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.464+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd50199150 con 0x7fdd50100ca0 2023-07-21T22:32:26.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.465+0000 7fdd467fc700 1 -- 172.21.15.66:0/3761639542 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10825 (secure 0 0 0) 0x7fdd4c021e60 con 0x7fdd50100ca0 2023-07-21T22:32:26.513 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:32:26.513 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:31:36.037177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:32:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdd3c0a1300 msgr2=0x7fdd3c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdd3c0a1300 0x7fdd3c0a37b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd50199410 tx=0x7fdd40009260 comp rx=0 tx=0).stop 2023-07-21T22:32:26.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 msgr2=0x7fdd50197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50197e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c000c00 tx=0x7fdd4c005740 comp rx=0 tx=0).stop 2023-07-21T22:32:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 shutdown_connections 2023-07-21T22:32:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd50100ca0 0x7fdd50197e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fdd3c0a1300 0x7fdd3c0a37b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd5010a070 0x7fdd5019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 --2- 172.21.15.66:0/3761639542 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fdd501015b0 0x7fdd50198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 >> 172.21.15.66:0/3761639542 conn(0x7fdd500fa490 msgr2=0x7fdd5010c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 shutdown_connections 2023-07-21T22:32:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.468+0000 7fdd577ff700 1 -- 172.21.15.66:0/3761639542 wait complete. 2023-07-21T22:32:26.517 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 41 2023-07-21T22:32:26.536 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:32:26.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 501385 5.22 MB/sec execute 473 sec latency 23.306 ms 2023-07-21T22:32:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 -- 172.21.15.66:0/4222644854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc0ffad0 msgr2=0x7fe9cc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/4222644854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc0ffad0 0x7fe9cc0fff40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009990 tx=0x7fe9c0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 -- 172.21.15.66:0/4222644854 shutdown_connections 2023-07-21T22:32:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/4222644854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc105ae0 0x7fe9cc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/4222644854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9cc100480 0x7fe9cc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/4222644854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc0ffad0 0x7fe9cc0fff40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 -- 172.21.15.66:0/4222644854 >> 172.21.15.66:0/4222644854 conn(0x7fe9cc0fb5f0 msgr2=0x7fe9cc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 -- 172.21.15.66:0/4222644854 shutdown_connections 2023-07-21T22:32:26.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.715+0000 7fe9d17ff700 1 -- 172.21.15.66:0/4222644854 wait complete. 2023-07-21T22:32:26.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 Processor -- start 2023-07-21T22:32:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 -- start start 2023-07-21T22:32:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9cc0ffad0 0x7fe9cc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 0x7fe9cc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc105ae0 0x7fe9cc1a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc073580 con 0x7fe9cc0ffad0 2023-07-21T22:32:26.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1a4310 con 0x7fe9cc100480 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9d17ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1a4450 con 0x7fe9cc105ae0 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 0x7fe9cc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9cc0ffad0 0x7fe9cc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 0x7fe9cc072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45088/0 (socket says 172.21.15.66:45088) 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.717+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/857807522 learned_addr learned my addr 172.21.15.66:0/857807522 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc105ae0 0x7fe9cc1a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/857807522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc105ae0 msgr2=0x7fe9cc1a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc105ae0 0x7fe9cc1a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9cc0ffad0 msgr2=0x7fe9cc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9cc0ffad0 0x7fe9cc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/857807522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00095f0 con 0x7fe9cc100480 2023-07-21T22:32:26.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 0x7fe9cc072d50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0005910 tx=0x7fe9c0005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.718+0000 7fe9b3fff700 1 -- 172.21.15.66:0/857807522 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c001d070 con 0x7fe9cc100480 2023-07-21T22:32:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.719+0000 7fe9b3fff700 1 -- 172.21.15.66:0/857807522 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c0022470 con 0x7fe9cc100480 2023-07-21T22:32:26.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.719+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc1a4650 con 0x7fe9cc100480 2023-07-21T22:32:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.719+0000 7fe9b3fff700 1 -- 172.21.15.66:0/857807522 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0017650 con 0x7fe9cc100480 2023-07-21T22:32:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.719+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1a4a70 con 0x7fe9cc100480 2023-07-21T22:32:26.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.723+0000 7fe9b3fff700 1 -- 172.21.15.66:0/857807522 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe9c00225e0 con 0x7fe9cc100480 2023-07-21T22:32:26.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.724+0000 7fe9b3fff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9b40a12c0 0x7fe9b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.724+0000 7fe9b3fff700 1 -- 172.21.15.66:0/857807522 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe9c00ce420 con 0x7fe9cc100480 2023-07-21T22:32:26.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.724+0000 7fe9e5098b80 1 Processor -- start 2023-07-21T22:32:26.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9caffd700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9b40a12c0 0x7fe9b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 -- start start 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68af1b0 0x5584c68af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 0x5584c68aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c67afcb0 con 0x5584c68afac0 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c6802a40 con 0x5584c68aa930 2023-07-21T22:32:26.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c67fbed0 con 0x5584c68af1b0 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9caffd700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9b40a12c0 0x7fe9b40a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009be0 tx=0x7fe9bc009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 0x5584c68aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9cb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68af1b0 0x5584c68af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68af1b0 0x5584c68af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45950/0 (socket says 172.21.15.66:45950) 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9caffd700 1 -- 172.21.15.66:0/751835264 learned_addr learned my addr 172.21.15.66:0/751835264 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:26.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.725+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 0x5584c68aa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53118/0 (socket says 172.21.15.66:53118) 2023-07-21T22:32:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9cb7fe700 1 -- 172.21.15.66:0/751835264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68af1b0 msgr2=0x5584c68af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68af1b0 0x5584c68af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9cb7fe700 1 -- 172.21.15.66:0/751835264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 msgr2=0x5584c68aa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 0x5584c68aa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9cb7fe700 1 -- 172.21.15.66:0/751835264 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584c6629de0 con 0x5584c68aa930 2023-07-21T22:32:26.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 0x5584c68aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68aada0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4011550 tx=0x7fe9c4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbb38ab9e1e6b99d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9b17fa700 1 -- 172.21.15.66:0/751835264 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400e040 con 0x5584c68aa930 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9b17fa700 1 -- 172.21.15.66:0/751835264 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c400f040 con 0x5584c68aa930 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.726+0000 7fe9b17fa700 1 -- 172.21.15.66:0/751835264 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4004370 con 0x5584c68aa930 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 -- 172.21.15.66:0/751835264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 msgr2=0x5584c68aada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68aada0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4011550 tx=0x7fe9c4009920 comp rx=0 tx=0).stop 2023-07-21T22:32:26.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 -- 172.21.15.66:0/751835264 shutdown_connections 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68aada0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68af1b0 0x5584c68af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/751835264 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68afac0 0x5584c68aa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 -- 172.21.15.66:0/751835264 >> 172.21.15.66:0/751835264 conn(0x5584c689fef0 msgr2=0x5584c68a5830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 -- 172.21.15.66:0/751835264 shutdown_connections 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.727+0000 7fe9e5098b80 1 -- 172.21.15.66:0/751835264 wait complete. 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.728+0000 7fe9e5098b80 1 -- start start 2023-07-21T22:32:26.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.728+0000 7fe9e5098b80 1 Processor -- start 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c68b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c679bac0 con 0x5584c68af1b0 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c679b7c0 con 0x5584c68aa930 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c667f920 con 0x5584c68afac0 2023-07-21T22:32:26.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9e5098b80 10 client.0 _ll_register_callbacks cb 0x5584c6799760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c68b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45094/0 (socket says 172.21.15.66:45094) 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9caffd700 1 -- 172.21.15.66:0/4254059239 learned_addr learned my addr 172.21.15.66:0/4254059239 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.729+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9caffd700 1 -- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 msgr2=0x5584c68b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9caffd700 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9caffd700 1 -- 172.21.15.66:0/4254059239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 msgr2=0x5584c68b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9caffd700 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c68b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9caffd700 1 -- 172.21.15.66:0/4254059239 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc1a4a70 con 0x5584c68aa930 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9caffd700 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68b6b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc01f8f0 tx=0x7fe9bc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8da5042663712e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9aeff5700 1 -- 172.21.15.66:0/4254059239 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc01d740 con 0x5584c68aa930 2023-07-21T22:32:26.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584c67554f0 con 0x5584c68aa930 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.730+0000 7fe9aeff5700 1 -- 172.21.15.66:0/4254059239 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe9bc01d8a0 con 0x5584c68aa930 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9aeff5700 1 -- 172.21.15.66:0/4254059239 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc01da60 con 0x5584c68aa930 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 10 client.26951 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fe9c4011230 con 0x5584c68aa930 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9aeff5700 1 -- 172.21.15.66:0/4254059239 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584c67554f0 con 0x5584c68aa930 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9aeff5700 1 -- 172.21.15.66:0/4254059239 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9bc02f070 con 0x5584c68aa930 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 10 client.26951 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:26.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 10 client.26951 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 20 client.26951 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c68c4450 0x5584c68c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 4 client.26951 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.731+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584c67f4eb0 con 0x5584c68c4450 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.732+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c68c4450 0x5584c68c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.732+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c68c4450 0x5584c68c6860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2343962bc7a0a32 server_cookie=5530cf509a364ef in_seq=0 out_seq=0 2023-07-21T22:32:26.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.732+0000 7fe9aeff5700 10 client.26951 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.733+0000 7fe9aeff5700 1 -- 172.21.15.66:0/4254059239 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5584c67f4eb0 con 0x5584c68c4450 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.733+0000 7fe9aeff5700 10 client.26951 handle_command_reply: tid=0 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.733+0000 7fe9e5098b80 1 client.26951 shutdown 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 msgr2=0x5584c68b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68b6b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc01f8f0 tx=0x7fe9bc0047c0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 shutdown_connections 2023-07-21T22:32:26.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c68c4450 0x5584c68c6860 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c68b6b80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/4254059239 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c68b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 >> 172.21.15.66:0/4254059239 conn(0x5584c689fef0 msgr2=0x5584c68a53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9aeff5700 0 client.26951 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 shutdown_connections 2023-07-21T22:32:26.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 1 -- 172.21.15.66:0/4254059239 wait complete. 2023-07-21T22:32:26.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.734+0000 7fe9e5098b80 20 client.26951 trim_cache size 0 max 16384 2023-07-21T22:32:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.749+0000 7fe9e5098b80 1 Processor -- start 2023-07-21T22:32:26.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.749+0000 7fe9e5098b80 1 -- start start 2023-07-21T22:32:26.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68aa930 0x5584c68b6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68af1b0 0x5584c68b3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c67554f0 con 0x5584c68aa930 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c68ab6d0 con 0x5584c68af1b0 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c68a2040 con 0x5584c68afac0 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9cb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9ca7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68af1b0 0x5584c68b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9cb7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:45962/0 (socket says 172.21.15.66:45962) 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9cb7fe700 1 -- 172.21.15.66:0/3933476455 learned_addr learned my addr 172.21.15.66:0/3933476455 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9caffd700 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68aa930 0x5584c68b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/3933476455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 msgr2=0x5584c68b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/3933476455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68aa930 msgr2=0x5584c68b6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68aa930 0x5584c68b6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9ca7fc700 1 -- 172.21.15.66:0/3933476455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584c68aafd0 con 0x5584c68af1b0 2023-07-21T22:32:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.750+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:26.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.751+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68af1b0 0x5584c68b3610 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00cb770 tx=0x7fe9c001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffbc9d11f0e975a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.751+0000 7fe9b1ffb700 1 -- 172.21.15.66:0/3933476455 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c009aae0 con 0x5584c68af1b0 2023-07-21T22:32:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.751+0000 7fe9b1ffb700 1 -- 172.21.15.66:0/3933476455 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c009b180 con 0x5584c68af1b0 2023-07-21T22:32:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.751+0000 7fe9b1ffb700 1 -- 172.21.15.66:0/3933476455 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00cb9f0 con 0x5584c68af1b0 2023-07-21T22:32:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.751+0000 7fe9e5098b80 1 -- 172.21.15.66:0/3933476455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68af1b0 msgr2=0x5584c68b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.751+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68af1b0 0x5584c68b3610 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00cb770 tx=0x7fe9c001a040 comp rx=0 tx=0).stop 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 -- 172.21.15.66:0/3933476455 shutdown_connections 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68aa930 0x5584c68b6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/3933476455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68af1b0 0x5584c68b3610 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 -- 172.21.15.66:0/3933476455 >> 172.21.15.66:0/3933476455 conn(0x5584c689fef0 msgr2=0x5584c68a8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 -- 172.21.15.66:0/3933476455 shutdown_connections 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 -- 172.21.15.66:0/3933476455 wait complete. 2023-07-21T22:32:26.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.752+0000 7fe9e5098b80 1 -- start start 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 Processor -- start 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c6911500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c6911a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c68b9ad0 con 0x5584c68af1b0 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c68b9c40 con 0x5584c68aa930 2023-07-21T22:32:26.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584c68b9e00 con 0x5584c68afac0 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9e5098b80 10 client.0 _ll_register_callbacks cb 0x7fe9bc01f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.753+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c6911500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c6911500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45110/0 (socket says 172.21.15.66:45110) 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 -- 172.21.15.66:0/127964800 learned_addr learned my addr 172.21.15.66:0/127964800 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c6911a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9cb7fe700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 -- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 msgr2=0x5584c68b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 -- 172.21.15.66:0/127964800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 msgr2=0x5584c6911a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c6911a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 -- 172.21.15.66:0/127964800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584c6629de0 con 0x5584c68aa930 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9caffd700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c6911500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc000c00 tx=0x7fe9bc0093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9469a7c0f78d6ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.754+0000 7fe9aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9aeff5700 1 -- 172.21.15.66:0/127964800 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc01d740 con 0x5584c68aa930 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9aeff5700 1 -- 172.21.15.66:0/127964800 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe9bc01d8a0 con 0x5584c68aa930 2023-07-21T22:32:26.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9c400f040 con 0x5584c68aa930 2023-07-21T22:32:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9aeff5700 1 -- 172.21.15.66:0/127964800 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc01daa0 con 0x5584c68aa930 2023-07-21T22:32:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9e5098b80 10 client.26957 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:26.807 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fe9cc1a4650 con 0x5584c68aa930 2023-07-21T22:32:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9aeff5700 1 -- 172.21.15.66:0/127964800 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9c400f040 con 0x5584c68aa930 2023-07-21T22:32:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9aeff5700 1 -- 172.21.15.66:0/127964800 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9bc01f920 con 0x5584c68aa930 2023-07-21T22:32:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9e5098b80 10 client.26957 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:26.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9e5098b80 10 client.26957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.755+0000 7fe9e5098b80 20 client.26957 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c6919270 0x5584c691b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9e5098b80 4 client.26957 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584c67b7050 con 0x5584c6919270 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c6919270 0x5584c691b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9ca7fc700 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c6919270 0x5584c691b680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21acb1ac9ad74734 server_cookie=279f5bbd80032938 in_seq=0 out_seq=0 2023-07-21T22:32:26.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9aeff5700 10 client.26957 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.756+0000 7fe9aeff5700 1 -- 172.21.15.66:0/127964800 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584c67b7050 con 0x5584c6919270 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9aeff5700 10 client.26957 handle_command_reply: tid=0 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 client.26957 shutdown 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 msgr2=0x5584c6911500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c6911500 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc000c00 tx=0x7fe9bc0093d0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 shutdown_connections 2023-07-21T22:32:26.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5584c6919270 0x5584c691b680 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584c68aa930 0x5584c6911500 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5584c68afac0 0x5584c68b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 --2- 172.21.15.66:0/127964800 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5584c68af1b0 0x5584c6911a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9aeff5700 0 client.26957 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.757+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 >> 172.21.15.66:0/127964800 conn(0x5584c689fef0 msgr2=0x5584c68a9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.758+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 shutdown_connections 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.758+0000 7fe9e5098b80 1 -- 172.21.15.66:0/127964800 wait complete. 2023-07-21T22:32:26.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.758+0000 7fe9e5098b80 20 client.26957 trim_cache size 0 max 16384 2023-07-21T22:32:26.812 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:32:26.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9b40a12c0 msgr2=0x7fe9b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9b40a12c0 0x7fe9b40a3770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009be0 tx=0x7fe9bc009260 comp rx=0 tx=0).stop 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 msgr2=0x7fe9cc072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 0x7fe9cc072d50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0005910 tx=0x7fe9c0005720 comp rx=0 tx=0).stop 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 shutdown_connections 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe9b40a12c0 0x7fe9b40a3770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9cc105ae0 0x7fe9cc1a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe9cc0ffad0 0x7fe9cc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 --2- 172.21.15.66:0/857807522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9cc100480 0x7fe9cc072d50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 >> 172.21.15.66:0/857807522 conn(0x7fe9cc0fb5f0 msgr2=0x7fe9cc0fbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 shutdown_connections 2023-07-21T22:32:26.814 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:26.759+0000 7fe9d17ff700 1 -- 172.21.15.66:0/857807522 wait complete. 2023-07-21T22:32:26.825 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:32:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.046+0000 7fe8e31ff700 1 -- 172.21.15.66:0/2115538927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc0ffad0 msgr2=0x7fe8dc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.046+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/2115538927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc0ffad0 0x7fe8dc0fff40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009990 tx=0x7fe8c8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 -- 172.21.15.66:0/2115538927 shutdown_connections 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/2115538927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc105ae0 0x7fe8dc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/2115538927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8dc100480 0x7fe8dc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/2115538927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc0ffad0 0x7fe8dc0fff40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 -- 172.21.15.66:0/2115538927 >> 172.21.15.66:0/2115538927 conn(0x7fe8dc0fb5f0 msgr2=0x7fe8dc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 -- 172.21.15.66:0/2115538927 shutdown_connections 2023-07-21T22:32:27.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.047+0000 7fe8e31ff700 1 -- 172.21.15.66:0/2115538927 wait complete. 2023-07-21T22:32:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.048+0000 7fe8e31ff700 1 Processor -- start 2023-07-21T22:32:27.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 -- start start 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8dc0ffad0 0x7fe8dc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 0x7fe8dc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc105ae0 0x7fe8dc19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc19ce10 con 0x7fe8dc0ffad0 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc19cf80 con 0x7fe8dc100480 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e31ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc19d140 con 0x7fe8dc105ae0 2023-07-21T22:32:27.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8dbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 0x7fe8dc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8dbfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 0x7fe8dc198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45126/0 (socket says 172.21.15.66:45126) 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8dbfff700 1 -- 172.21.15.66:0/516064673 learned_addr learned my addr 172.21.15.66:0/516064673 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8dc0ffad0 0x7fe8dc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.049+0000 7fe8e179c700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc105ae0 0x7fe8dc19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8dbfff700 1 -- 172.21.15.66:0/516064673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc105ae0 msgr2=0x7fe8dc19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc105ae0 0x7fe8dc19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.099 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8dbfff700 1 -- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8dc0ffad0 msgr2=0x7fe8dc197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8dc0ffad0 0x7fe8dc197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8dbfff700 1 -- 172.21.15.66:0/516064673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d0009520 con 0x7fe8dc100480 2023-07-21T22:32:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 0x7fe8dc198370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8005910 tx=0x7fe8c8005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8d9ffb700 1 -- 172.21.15.66:0/516064673 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801d070 con 0x7fe8dc100480 2023-07-21T22:32:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c80095f0 con 0x7fe8dc100480 2023-07-21T22:32:27.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8d9ffb700 1 -- 172.21.15.66:0/516064673 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8c8022470 con 0x7fe8dc100480 2023-07-21T22:32:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8d9ffb700 1 -- 172.21.15.66:0/516064673 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8017650 con 0x7fe8dc100480 2023-07-21T22:32:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.050+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc1a4a80 con 0x7fe8dc100480 2023-07-21T22:32:27.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.054+0000 7fe8d9ffb700 1 -- 172.21.15.66:0/516064673 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe8c80225e0 con 0x7fe8dc100480 2023-07-21T22:32:27.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.055+0000 7fe8f6bffb80 1 Processor -- start 2023-07-21T22:32:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 -- start start 2023-07-21T22:32:27.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394da1b0 0x5615394da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394d5930 0x5615394d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615393dacb0 con 0x5615394d5930 2023-07-21T22:32:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56153942da40 con 0x5615394da1b0 2023-07-21T22:32:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561539426ed0 con 0x5615394daac0 2023-07-21T22:32:27.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8d9ffb700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe8cc0a1340 0x7fe8cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46006/0 (socket says 172.21.15.66:46006) 2023-07-21T22:32:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/2488462177 learned_addr learned my addr 172.21.15.66:0/2488462177 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8d9ffb700 1 -- 172.21.15.66:0/516064673 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe8c80ce380 con 0x7fe8dc100480 2023-07-21T22:32:27.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.056+0000 7fe8e179c700 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394da1b0 0x5615394da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/2488462177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394da1b0 msgr2=0x5615394da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394da1b0 0x5615394da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/2488462177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394d5930 msgr2=0x5615394d5da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394d5930 0x5615394d5da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/2488462177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561539254de0 con 0x5615394daac0 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394d5350 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009e90 tx=0x7fe8d0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802f6e13530bd985 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e179c700 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394da1b0 0x5615394da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:32:27.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8e179c700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe8cc0a1340 0x7fe8cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8c6ffd700 1 -- 172.21.15.66:0/2488462177 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000e670 con 0x5615394daac0 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8c6ffd700 1 -- 172.21.15.66:0/2488462177 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8d000ec90 con 0x5615394daac0 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.057+0000 7fe8c6ffd700 1 -- 172.21.15.66:0/2488462177 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0019d70 con 0x5615394daac0 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.058+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/2488462177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 msgr2=0x5615394d5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.058+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394d5350 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009e90 tx=0x7fe8d0011720 comp rx=0 tx=0).stop 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.058+0000 7fe8e179c700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe8cc0a1340 0x7fe8cc0a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009ec0 tx=0x7fe8d4004ad0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/2488462177 shutdown_connections 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394da1b0 0x5615394da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394d5930 0x5615394d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/2488462177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394d5350 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/2488462177 >> 172.21.15.66:0/2488462177 conn(0x5615394caef0 msgr2=0x5615394d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/2488462177 shutdown_connections 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/2488462177 wait complete. 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.059+0000 7fe8f6bffb80 1 -- start start 2023-07-21T22:32:27.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 Processor -- start 2023-07-21T22:32:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 0x5615394e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394df460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615393c6ac0 con 0x5615394da1b0 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615393c67c0 con 0x5615394d5930 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615392aa920 con 0x5615394daac0 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.060+0000 7fe8f6bffb80 10 client.0 _ll_register_callbacks cb 0x5615393c4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 0x5615394e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e179c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394df460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8dbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 0x5615394e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45142/0 (socket says 172.21.15.66:45142) 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/3196758917 learned_addr learned my addr 172.21.15.66:0/3196758917 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/3196758917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 msgr2=0x5615394df460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394df460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/3196758917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 msgr2=0x5615394dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/3196758917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d0009af0 con 0x5615394d5930 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 0x5615394e1b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d0019620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f6567f81785b032 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.061+0000 7fe8bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8bbfff700 1 -- 172.21.15.66:0/3196758917 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0019430 con 0x5615394d5930 2023-07-21T22:32:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8bbfff700 1 -- 172.21.15.66:0/3196758917 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe8d0004250 con 0x5615394d5930 2023-07-21T22:32:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615393804f0 con 0x5615394d5930 2023-07-21T22:32:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8bbfff700 1 -- 172.21.15.66:0/3196758917 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8d0004410 con 0x5615394d5930 2023-07-21T22:32:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 10 client.26966 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:27.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56153923dc60 con 0x5615394d5930 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8bbfff700 1 -- 172.21.15.66:0/3196758917 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615393804f0 con 0x5615394d5930 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8bbfff700 1 -- 172.21.15.66:0/3196758917 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe8d00052a0 con 0x5615394d5930 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 10 client.26966 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 10 client.26966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 20 client.26966 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.062+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5615394ef320 0x5615394f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.063+0000 7fe8f6bffb80 4 client.26966 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.063+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56153941feb0 con 0x5615394ef320 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.063+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5615394ef320 0x5615394f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.063+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5615394ef320 0x5615394f1730 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e788d9c3f52521 server_cookie=4e81aceb8cf9d5b3 in_seq=0 out_seq=0 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.063+0000 7fe8bbfff700 10 client.26966 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8bbfff700 1 -- 172.21.15.66:0/3196758917 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56153941feb0 con 0x5615394ef320 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8bbfff700 10 client.26966 handle_command_reply: tid=0 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 client.26966 shutdown 2023-07-21T22:32:27.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 msgr2=0x5615394e1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 0x5615394e1b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d0019620 comp rx=0 tx=0).stop 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 shutdown_connections 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x5615394ef320 0x5615394f1730 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394d5930 0x5615394e1b80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394daac0 0x5615394df460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/3196758917 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 >> 172.21.15.66:0/3196758917 conn(0x5615394caef0 msgr2=0x5615394d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8bbfff700 0 client.26966 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:32:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.064+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 shutdown_connections 2023-07-21T22:32:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.065+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/3196758917 wait complete. 2023-07-21T22:32:27.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.065+0000 7fe8f6bffb80 20 client.26966 trim_cache size 0 max 16384 2023-07-21T22:32:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 Processor -- start 2023-07-21T22:32:27.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 -- start start 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394e6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615393804f0 con 0x5615394da1b0 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615394cd010 con 0x5615394daac0 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615394e03e0 con 0x5615394d5930 2023-07-21T22:32:27.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394e6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8e179c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8dbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394e6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46022/0 (socket says 172.21.15.66:46022) 2023-07-21T22:32:27.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.082+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/365916677 learned_addr learned my addr 172.21.15.66:0/365916677 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8e179c700 1 -- 172.21.15.66:0/365916677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 msgr2=0x5615394e6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8e179c700 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394e6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8e179c700 1 -- 172.21.15.66:0/365916677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 msgr2=0x5615394e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8e179c700 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8e179c700 1 -- 172.21.15.66:0/365916677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56153927b250 con 0x5615394daac0 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8e179c700 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394d3a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007c20 tx=0x7fe8d4015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a09e59d176e063 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8c77fe700 1 -- 172.21.15.66:0/365916677 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400e030 con 0x5615394daac0 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8c77fe700 1 -- 172.21.15.66:0/365916677 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8d400f040 con 0x5615394daac0 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.083+0000 7fe8c77fe700 1 -- 172.21.15.66:0/365916677 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4023660 con 0x5615394daac0 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/365916677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 msgr2=0x5615394d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394d3a60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4007c20 tx=0x7fe8d4015000 comp rx=0 tx=0).stop 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/365916677 shutdown_connections 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394d3a60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394e6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/365916677 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/365916677 >> 172.21.15.66:0/365916677 conn(0x5615394caef0 msgr2=0x5615394f1330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/365916677 shutdown_connections 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.084+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/365916677 wait complete. 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 -- start start 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 Processor -- start 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394de8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615394ddaf0 con 0x5615394da1b0 2023-07-21T22:32:27.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615394ddc60 con 0x5615394daac0 2023-07-21T22:32:27.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.085+0000 7fe8f6bffb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615394dde20 con 0x5615394d5930 2023-07-21T22:32:27.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394de8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8f6bffb80 10 client.0 _ll_register_callbacks cb 0x7fe8d0009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:27.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394de8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46036/0 (socket says 172.21.15.66:46036) 2023-07-21T22:32:27.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/1762381864 learned_addr learned my addr 172.21.15.66:0/1762381864 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e179c700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 msgr2=0x5615394dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/1762381864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 msgr2=0x5615394dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 -- 172.21.15.66:0/1762381864 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d4016690 con 0x5615394d5930 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e179c700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8e0f9b700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394de8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d00194c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e855f3e4a4aa2e01 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.086+0000 7fe8bbfff700 1 -- 172.21.15.66:0/1762381864 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0004450 con 0x5615394d5930 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8d000ec90 con 0x5615394d5930 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8bbfff700 1 -- 172.21.15.66:0/1762381864 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe8d0000e40 con 0x5615394d5930 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8bbfff700 1 -- 172.21.15.66:0/1762381864 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8d000eec0 con 0x5615394d5930 2023-07-21T22:32:27.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 10 client.26950 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5615392532b0 con 0x5615394d5930 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8bbfff700 1 -- 172.21.15.66:0/1762381864 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8d000ec90 con 0x5615394d5930 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8bbfff700 1 -- 172.21.15.66:0/1762381864 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe8d00223a0 con 0x5615394d5930 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 10 client.26950 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 10 client.26950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 20 client.26950 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:27.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561539543f20 0x561539546330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 4 client.26950 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.087+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5615393e2050 con 0x561539543f20 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.088+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561539543f20 0x561539546330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.088+0000 7fe8dbfff700 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561539543f20 0x561539546330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc056b719f90399a server_cookie=e9c079238fa3f0ab in_seq=0 out_seq=0 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.088+0000 7fe8bbfff700 10 client.26950 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.088+0000 7fe8bbfff700 1 -- 172.21.15.66:0/1762381864 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5615393e2050 con 0x561539543f20 2023-07-21T22:32:27.142 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.088+0000 7fe8bbfff700 10 client.26950 handle_command_reply: tid=0 2023-07-21T22:32:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.088+0000 7fe8f6bffb80 1 client.26950 shutdown 2023-07-21T22:32:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 msgr2=0x5615394de8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394de8c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0000c00 tx=0x7fe8d00194c0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 shutdown_connections 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x561539543f20 0x561539546330 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615394daac0 0x5615394dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8bbfff700 0 client.26950 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5615394d5930 0x5615394de8c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 --2- 172.21.15.66:0/1762381864 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5615394da1b0 0x5615394dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 >> 172.21.15.66:0/1762381864 conn(0x5615394caef0 msgr2=0x5615394d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 shutdown_connections 2023-07-21T22:32:27.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 1 -- 172.21.15.66:0/1762381864 wait complete. 2023-07-21T22:32:27.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8f6bffb80 20 client.26950 trim_cache size 0 max 16384 2023-07-21T22:32:27.145 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:32:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe8cc0a1340 msgr2=0x7fe8cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.089+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe8cc0a1340 0x7fe8cc0a37f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009ec0 tx=0x7fe8d4004ad0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 msgr2=0x7fe8dc198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.146 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 0x7fe8dc198370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8005910 tx=0x7fe8c8005720 comp rx=0 tx=0).stop 2023-07-21T22:32:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 shutdown_connections 2023-07-21T22:32:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe8cc0a1340 0x7fe8cc0a37f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8dc105ae0 0x7fe8dc19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8dc0ffad0 0x7fe8dc197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 --2- 172.21.15.66:0/516064673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8dc100480 0x7fe8dc198370 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 >> 172.21.15.66:0/516064673 conn(0x7fe8dc0fb5f0 msgr2=0x7fe8dc10ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 shutdown_connections 2023-07-21T22:32:27.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.090+0000 7fe8e31ff700 1 -- 172.21.15.66:0/516064673 wait complete. 2023-07-21T22:32:27.155 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:26 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3761639542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:32:27.158 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:32:27.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:26 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3761639542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:32:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.335+0000 7efcbebff700 1 -- 172.21.15.66:0/408914738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb8100ca0 msgr2=0x7efcb8101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.335+0000 7efcbebff700 1 --2- 172.21.15.66:0/408914738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb8100ca0 0x7efcb8101070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efcac0098f0 tx=0x7efcac009c00 comp rx=0 tx=0).stop 2023-07-21T22:32:27.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 -- 172.21.15.66:0/408914738 shutdown_connections 2023-07-21T22:32:27.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 --2- 172.21.15.66:0/408914738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb810a070 0x7efcb810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 --2- 172.21.15.66:0/408914738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb8100ca0 0x7efcb8101070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 --2- 172.21.15.66:0/408914738 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb81015b0 0x7efcb8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 -- 172.21.15.66:0/408914738 >> 172.21.15.66:0/408914738 conn(0x7efcb80fa4b0 msgr2=0x7efcb80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 -- 172.21.15.66:0/408914738 shutdown_connections 2023-07-21T22:32:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.336+0000 7efcbebff700 1 -- 172.21.15.66:0/408914738 wait complete. 2023-07-21T22:32:27.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.337+0000 7efcbebff700 1 Processor -- start 2023-07-21T22:32:27.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 -- start start 2023-07-21T22:32:27.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 0x7efcb80fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb81015b0 0x7efcb80fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb810a070 0x7efcb81a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80ff660 con 0x7efcb81015b0 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb81a4400 con 0x7efcb8100ca0 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbebff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb81a4540 con 0x7efcb810a070 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbc99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 0x7efcb80fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbc99b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 0x7efcb80fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45178/0 (socket says 172.21.15.66:45178) 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbc99b700 1 -- 172.21.15.66:0/2953663805 learned_addr learned my addr 172.21.15.66:0/2953663805 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcbd19c700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb810a070 0x7efcb81a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.338+0000 7efcb7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb81015b0 0x7efcb80fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbc99b700 1 -- 172.21.15.66:0/2953663805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb810a070 msgr2=0x7efcb81a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbc99b700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb810a070 0x7efcb81a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbc99b700 1 -- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb81015b0 msgr2=0x7efcb80fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbc99b700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb81015b0 0x7efcb80fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbc99b700 1 -- 172.21.15.66:0/2953663805 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009520 con 0x7efcb8100ca0 2023-07-21T22:32:27.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbc99b700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 0x7efcb80fe8f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcac009810 tx=0x7efcac00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcb5ffb700 1 -- 172.21.15.66:0/2953663805 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac01c070 con 0x7efcb8100ca0 2023-07-21T22:32:27.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb81a4740 con 0x7efcb8100ca0 2023-07-21T22:32:27.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcb5ffb700 1 -- 172.21.15.66:0/2953663805 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcac00bb20 con 0x7efcb8100ca0 2023-07-21T22:32:27.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcb5ffb700 1 -- 172.21.15.66:0/2953663805 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac004930 con 0x7efcb8100ca0 2023-07-21T22:32:27.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.339+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb81a4c10 con 0x7efcb8100ca0 2023-07-21T22:32:27.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.343+0000 7efcb5ffb700 1 -- 172.21.15.66:0/2953663805 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efcac020070 con 0x7efcb8100ca0 2023-07-21T22:32:27.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.344+0000 7efcd25f4b80 1 Processor -- start 2023-07-21T22:32:27.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.344+0000 7efcd25f4b80 1 -- start start 2023-07-21T22:32:27.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.344+0000 7efcb5ffb700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efca80a12c0 0x7efca80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb5ffb700 1 -- 172.21.15.66:0/2953663805 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efcac0cddd0 con 0x7efcb8100ca0 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c93580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c8e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c8eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9b93cb0 con 0x556cd9c8e930 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9be6a40 con 0x556cd9c931b0 2023-07-21T22:32:27.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9bdfed0 con 0x556cd9c93ac0 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcbd19c700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efca80a12c0 0x7efca80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c8e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c8e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46046/0 (socket says 172.21.15.66:46046) 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 -- 172.21.15.66:0/555410331 learned_addr learned my addr 172.21.15.66:0/555410331 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcbd19c700 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c8eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcbc99b700 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c93580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 -- 172.21.15.66:0/555410331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 msgr2=0x556cd9c93580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c93580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 -- 172.21.15.66:0/555410331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 msgr2=0x556cd9c8eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c8eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcb7fff700 1 -- 172.21.15.66:0/555410331 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd9a0dde0 con 0x556cd9c93ac0 2023-07-21T22:32:27.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.345+0000 7efcbc99b700 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:32:27.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.346+0000 7efcb7fff700 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c8e350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efca4009d20 tx=0x7efca4011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec9031482b13250 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.346+0000 7efcbd19c700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efca80a12c0 0x7efca80a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcb8100a00 tx=0x7efcb000fc00 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.346+0000 7efca2ffd700 1 -- 172.21.15.66:0/555410331 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4019630 con 0x556cd9c93ac0 2023-07-21T22:32:27.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.346+0000 7efca2ffd700 1 -- 172.21.15.66:0/555410331 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efca4019c50 con 0x556cd9c93ac0 2023-07-21T22:32:27.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.346+0000 7efca2ffd700 1 -- 172.21.15.66:0/555410331 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400eb50 con 0x556cd9c93ac0 2023-07-21T22:32:27.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 -- 172.21.15.66:0/555410331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 msgr2=0x556cd9c8e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c8e350 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efca4009d20 tx=0x7efca4011720 comp rx=0 tx=0).stop 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 -- 172.21.15.66:0/555410331 shutdown_connections 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c8eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/555410331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c8e350 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 -- 172.21.15.66:0/555410331 >> 172.21.15.66:0/555410331 conn(0x556cd9c83ef0 msgr2=0x556cd9c89830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 -- 172.21.15.66:0/555410331 shutdown_connections 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.347+0000 7efcd25f4b80 1 -- 172.21.15.66:0/555410331 wait complete. 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 -- start start 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 Processor -- start 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c97e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9b7fac0 con 0x556cd9c931b0 2023-07-21T22:32:27.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9b7f7c0 con 0x556cd9c93ac0 2023-07-21T22:32:27.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9a63920 con 0x556cd9c8e930 2023-07-21T22:32:27.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.348+0000 7efcd25f4b80 10 client.0 _ll_register_callbacks cb 0x556cd9b7d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:27.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46062/0 (socket says 172.21.15.66:46062) 2023-07-21T22:32:27.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbd19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 -- 172.21.15.66:0/468811084 learned_addr learned my addr 172.21.15.66:0/468811084 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcb7fff700 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c97e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 -- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 msgr2=0x556cd9c984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 -- 172.21.15.66:0/468811084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 msgr2=0x556cd9c97e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c97e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 -- 172.21.15.66:0/468811084 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca4009980 con 0x556cd9c8e930 2023-07-21T22:32:27.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcbc99b700 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9aba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efcac09aac0 tx=0x7efcac09a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791f2767f42d608c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efc97fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efc97fff700 1 -- 172.21.15.66:0/468811084 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac09eb60 con 0x556cd9c8e930 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efc97fff700 1 -- 172.21.15.66:0/468811084 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efcac0a2ef0 con 0x556cd9c8e930 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.349+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556cd9b394f0 con 0x556cd9c8e930 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efc97fff700 1 -- 172.21.15.66:0/468811084 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcac09a020 con 0x556cd9c8e930 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efcd25f4b80 10 client.26962 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556cd99f6c60 con 0x556cd9c8e930 2023-07-21T22:32:27.403 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efc97fff700 1 -- 172.21.15.66:0/468811084 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556cd9b394f0 con 0x556cd9c8e930 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efc97fff700 1 -- 172.21.15.66:0/468811084 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efcac00b5e0 con 0x556cd9c8e930 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efcd25f4b80 10 client.26962 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efcd25f4b80 10 client.26962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.350+0000 7efcd25f4b80 20 client.26962 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.351+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9ca7fa0 0x556cd9caa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.351+0000 7efcd25f4b80 4 client.26962 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:27.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.351+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cd9bd8eb0 con 0x556cd9ca7fa0 2023-07-21T22:32:27.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.351+0000 7efcb7fff700 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9ca7fa0 0x556cd9caa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.351+0000 7efcb7fff700 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9ca7fa0 0x556cd9caa3b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd241d162755bb35 server_cookie=bf8db65ee34c2623 in_seq=0 out_seq=0 2023-07-21T22:32:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.351+0000 7efc97fff700 10 client.26962 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efc97fff700 1 -- 172.21.15.66:0/468811084 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556cd9bd8eb0 con 0x556cd9ca7fa0 2023-07-21T22:32:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efc97fff700 10 client.26962 handle_command_reply: tid=0 2023-07-21T22:32:27.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efcd25f4b80 1 client.26962 shutdown 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 msgr2=0x556cd9c9aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9aba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efcac09aac0 tx=0x7efcac09a490 comp rx=0 tx=0).stop 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 shutdown_connections 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.352+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9ca7fa0 0x556cd9caa3b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9aba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/468811084 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c97e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efc97fff700 0 client.26962 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 >> 172.21.15.66:0/468811084 conn(0x556cd9c83ef0 msgr2=0x556cd9c89420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 shutdown_connections 2023-07-21T22:32:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efcd25f4b80 1 -- 172.21.15.66:0/468811084 wait complete. 2023-07-21T22:32:27.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.353+0000 7efcd25f4b80 20 client.26962 trim_cache size 0 max 16384 2023-07-21T22:32:27.409 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:26 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3761639542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:32:27.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.366+0000 7efcd25f4b80 1 Processor -- start 2023-07-21T22:32:27.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 -- start start 2023-07-21T22:32:27.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.421 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c892d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9c86010 con 0x556cd9c931b0 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9c81050 con 0x556cd9c93ac0 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9c98db0 con 0x556cd9c8e930 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcbc99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcb7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.422 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcbc99b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46076/0 (socket says 172.21.15.66:46076) 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcbc99b700 1 -- 172.21.15.66:0/1279789187 learned_addr learned my addr 172.21.15.66:0/1279789187 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.367+0000 7efcbd19c700 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c892d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcbc99b700 1 -- 172.21.15.66:0/1279789187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 msgr2=0x556cd9c892d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcbc99b700 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c892d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcbc99b700 1 -- 172.21.15.66:0/1279789187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 msgr2=0x556cd9c9a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcbc99b700 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c9a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.423 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcbc99b700 1 -- 172.21.15.66:0/1279789187 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd9a34250 con 0x556cd9c8e930 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcb7fff700 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c9a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efcbc99b700 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9f570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efcac0b0040 tx=0x7efcac00bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ebc94ab42bf4fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efca37fe700 1 -- 172.21.15.66:0/1279789187 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac0b3250 con 0x556cd9c8e930 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efca37fe700 1 -- 172.21.15.66:0/1279789187 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcac0a2ef0 con 0x556cd9c8e930 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.368+0000 7efca37fe700 1 -- 172.21.15.66:0/1279789187 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac09a020 con 0x556cd9c8e930 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 -- 172.21.15.66:0/1279789187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 msgr2=0x556cd9c9f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9f570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efcac0b0040 tx=0x7efcac00bab0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.424 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 -- 172.21.15.66:0/1279789187 shutdown_connections 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c93ac0 0x556cd9c892d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c8e930 0x556cd9c9f570 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/1279789187 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c931b0 0x556cd9c9a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 -- 172.21.15.66:0/1279789187 >> 172.21.15.66:0/1279789187 conn(0x556cd9c83ef0 msgr2=0x556cd9c8ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 -- 172.21.15.66:0/1279789187 shutdown_connections 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.369+0000 7efcd25f4b80 1 -- 172.21.15.66:0/1279789187 wait complete. 2023-07-21T22:32:27.425 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.370+0000 7efcd25f4b80 1 -- start start 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.370+0000 7efcd25f4b80 1 Processor -- start 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c9f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c96bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9c97130 con 0x556cd9c8e930 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9c972a0 con 0x556cd9c931b0 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd9c97460 con 0x556cd9c93ac0 2023-07-21T22:32:27.426 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcd25f4b80 10 client.0 _ll_register_callbacks cb 0x7efcac09a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c96670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45216/0 (socket says 172.21.15.66:45216) 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 -- 172.21.15.66:0/3068327671 learned_addr learned my addr 172.21.15.66:0/3068327671 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcbd19c700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c96bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcbc99b700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c9f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 -- 172.21.15.66:0/3068327671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 msgr2=0x556cd9c96bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c96bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 -- 172.21.15.66:0/3068327671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 msgr2=0x556cd9c9f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c9f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.371+0000 7efcb7fff700 1 -- 172.21.15.66:0/3068327671 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cd9a0dde0 con 0x556cd9c931b0 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efcbd19c700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c96bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efcbc99b700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c9f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efcb7fff700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c96670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efca400a010 tx=0x7efca4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96e975e916fd7cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:27.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efc97fff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efc97fff700 1 -- 172.21.15.66:0/3068327671 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca402a040 con 0x556cd9c931b0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efca4019c50 con 0x556cd9c931b0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efc97fff700 1 -- 172.21.15.66:0/3068327671 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efca40194b0 con 0x556cd9c931b0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efc97fff700 1 -- 172.21.15.66:0/3068327671 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca4019690 con 0x556cd9c931b0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efcd25f4b80 10 client.26987 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.372+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556cd99f6c60 con 0x556cd9c931b0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efc97fff700 1 -- 172.21.15.66:0/3068327671 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efca4019c50 con 0x556cd9c931b0 2023-07-21T22:32:27.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efc97fff700 1 -- 172.21.15.66:0/3068327671 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efca4005040 con 0x556cd9c931b0 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcd25f4b80 10 client.26987 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcd25f4b80 10 client.26987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcd25f4b80 20 client.26987 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9d05cb0 0x556cd9d080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcd25f4b80 4 client.26987 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556cd9b9b050 con 0x556cd9d05cb0 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.373+0000 7efcbc99b700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9d05cb0 0x556cd9d080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:27.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.374+0000 7efcbc99b700 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9d05cb0 0x556cd9d080c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec26ab400cfb47b server_cookie=e72c45f4fa9c5715 in_seq=0 out_seq=0 2023-07-21T22:32:27.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.374+0000 7efc97fff700 10 client.26987 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:27.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.375+0000 7efc97fff700 1 -- 172.21.15.66:0/3068327671 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556cd9b9b050 con 0x556cd9d05cb0 2023-07-21T22:32:27.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.375+0000 7efc97fff700 10 client.26987 handle_command_reply: tid=0 2023-07-21T22:32:27.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.375+0000 7efcd25f4b80 1 client.26987 shutdown 2023-07-21T22:32:27.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 msgr2=0x556cd9c96670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c96670 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7efca400a010 tx=0x7efca4000bc0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 shutdown_connections 2023-07-21T22:32:27.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x556cd9d05cb0 0x556cd9d080c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd9c93ac0 0x556cd9c96bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x556cd9c8e930 0x556cd9c9f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efc97fff700 0 client.26987 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 --2- 172.21.15.66:0/3068327671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cd9c931b0 0x556cd9c96670 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 >> 172.21.15.66:0/3068327671 conn(0x556cd9c83ef0 msgr2=0x556cd9c892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 shutdown_connections 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 1 -- 172.21.15.66:0/3068327671 wait complete. 2023-07-21T22:32:27.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.376+0000 7efcd25f4b80 20 client.26987 trim_cache size 0 max 16384 2023-07-21T22:32:27.434 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:32:27.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efca80a12c0 msgr2=0x7efca80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efca80a12c0 0x7efca80a3770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcb8100a00 tx=0x7efcb000fc00 comp rx=0 tx=0).stop 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 msgr2=0x7efcb80fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 0x7efcb80fe8f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efcac009810 tx=0x7efcac00fa00 comp rx=0 tx=0).stop 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 shutdown_connections 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb8100ca0 0x7efcb80fe8f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efca80a12c0 0x7efca80a3770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efcb810a070 0x7efcb81a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 --2- 172.21.15.66:0/2953663805 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efcb81015b0 0x7efcb80fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 >> 172.21.15.66:0/2953663805 conn(0x7efcb80fa4b0 msgr2=0x7efcb810c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 shutdown_connections 2023-07-21T22:32:27.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:27.377+0000 7efcbebff700 1 -- 172.21.15.66:0/2953663805 wait complete. 2023-07-21T22:32:27.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 502550 5.22 MB/sec execute 474 sec latency 17.428 ms 2023-07-21T22:32:28.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:27 smithi104 ceph-mon[129021]: pgmap v342: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 917 KiB/s wr, 91 op/s 2023-07-21T22:32:28.445 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:32:28.446 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:32:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:27 smithi066 ceph-mon[126379]: pgmap v342: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 917 KiB/s wr, 91 op/s 2023-07-21T22:32:28.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:27 smithi149 ceph-mon[129268]: pgmap v342: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 917 KiB/s wr, 91 op/s 2023-07-21T22:32:28.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 504318 5.23 MB/sec execute 475 sec latency 16.897 ms 2023-07-21T22:32:28.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.632+0000 7f895a1ff700 1 -- 172.21.15.66:0/3924798682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954105af0 msgr2=0x7f8954105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.632+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3924798682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954105af0 0x7f8954105ec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8944009990 tx=0x7f8944009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 -- 172.21.15.66:0/3924798682 shutdown_connections 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3924798682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954100490 0x7f895410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3924798682 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954105af0 0x7f8954105ec0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3924798682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f89540fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 -- 172.21.15.66:0/3924798682 >> 172.21.15.66:0/3924798682 conn(0x7f89540fb5e0 msgr2=0x7f89540fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.681 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 -- 172.21.15.66:0/3924798682 shutdown_connections 2023-07-21T22:32:28.682 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.633+0000 7f895a1ff700 1 -- 172.21.15.66:0/3924798682 wait complete. 2023-07-21T22:32:28.683 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 Processor -- start 2023-07-21T22:32:28.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 -- start start 2023-07-21T22:32:28.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f895419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954100490 0x7f895419bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954105af0 0x7f8954195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954196080 con 0x7f8954100490 2023-07-21T22:32:28.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541961f0 con 0x7f8954105af0 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.634+0000 7f895a1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541963b0 con 0x7f89540ffae0 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f895419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f895419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46096/0 (socket says 172.21.15.66:46096) 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 -- 172.21.15.66:0/3523656605 learned_addr learned my addr 172.21.15.66:0/3523656605 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f8952ffd700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954100490 0x7f895419bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f8953fff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954105af0 0x7f8954195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 -- 172.21.15.66:0/3523656605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954105af0 msgr2=0x7f8954195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954105af0 0x7f8954195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 -- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954100490 msgr2=0x7f895419bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954100490 0x7f895419bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 -- 172.21.15.66:0/3523656605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89440095f0 con 0x7f89540ffae0 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.635+0000 7f89537fe700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f895419b9a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8944005910 tx=0x7f8944005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.636+0000 7f8950ff9700 1 -- 172.21.15.66:0/3523656605 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894401d070 con 0x7f89540ffae0 2023-07-21T22:32:28.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.636+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8954196630 con 0x7f89540ffae0 2023-07-21T22:32:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.636+0000 7f8950ff9700 1 -- 172.21.15.66:0/3523656605 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8944022470 con 0x7f89540ffae0 2023-07-21T22:32:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.636+0000 7f8950ff9700 1 -- 172.21.15.66:0/3523656605 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944017650 con 0x7f89540ffae0 2023-07-21T22:32:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.636+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541a2990 con 0x7f89540ffae0 2023-07-21T22:32:28.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.640+0000 7f8950ff9700 1 -- 172.21.15.66:0/3523656605 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f89440225e0 con 0x7f89540ffae0 2023-07-21T22:32:28.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.641+0000 7f896db0eb80 1 Processor -- start 2023-07-21T22:32:28.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.641+0000 7f896db0eb80 1 -- start start 2023-07-21T22:32:28.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615dd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f896db0eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7614ddd30 con 0x55d7615d89b0 2023-07-21T22:32:28.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f896db0eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761530ac0 con 0x55d7615ddb40 2023-07-21T22:32:28.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f896db0eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761529f50 con 0x55d7615dd230 2023-07-21T22:32:28.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f8953fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f89537fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f8953fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615dd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46108/0 (socket says 172.21.15.66:46108) 2023-07-21T22:32:28.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f8953fff700 1 -- 172.21.15.66:0/1882833732 learned_addr learned my addr 172.21.15.66:0/1882833732 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:28.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f8952ffd700 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f8953fff700 1 -- 172.21.15.66:0/1882833732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 msgr2=0x55d7615d83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.642+0000 7f8953fff700 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615d83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8953fff700 1 -- 172.21.15.66:0/1882833732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 msgr2=0x55d7615d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8953fff700 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8953fff700 1 -- 172.21.15.66:0/1882833732 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761357e60 con 0x55d7615dd230 2023-07-21T22:32:28.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8950ff9700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f893c0a12c0 0x7f893c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f89537fe700 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:28.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8950ff9700 1 -- 172.21.15.66:0/3523656605 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f89440ce420 con 0x7f89540ffae0 2023-07-21T22:32:28.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8953fff700 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615dd600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f894c011600 tx=0x7f894c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b86cc6d8f94d6b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f89537fe700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f893c0a12c0 0x7f893c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8939ffb700 1 -- 172.21.15.66:0/1882833732 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00e040 con 0x55d7615dd230 2023-07-21T22:32:28.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8939ffb700 1 -- 172.21.15.66:0/1882833732 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894c00f040 con 0x55d7615dd230 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.643+0000 7f8939ffb700 1 -- 172.21.15.66:0/1882833732 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0046c0 con 0x55d7615dd230 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 -- 172.21.15.66:0/1882833732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 msgr2=0x55d7615dd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615dd600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f894c011600 tx=0x7f894c009920 comp rx=0 tx=0).stop 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f89537fe700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f893c0a12c0 0x7f893c0a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55d7615d4ef0 tx=0x7f89440d1e90 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 -- 172.21.15.66:0/1882833732 shutdown_connections 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615dd600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1882833732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 -- 172.21.15.66:0/1882833732 >> 172.21.15.66:0/1882833732 conn(0x55d7615cdf70 msgr2=0x55d7615d38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 -- 172.21.15.66:0/1882833732 shutdown_connections 2023-07-21T22:32:28.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.644+0000 7f896db0eb80 1 -- 172.21.15.66:0/1882833732 wait complete. 2023-07-21T22:32:28.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 -- start start 2023-07-21T22:32:28.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 Processor -- start 2023-07-21T22:32:28.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d7615e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7615e1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7614c9b40 con 0x55d7615dd230 2023-07-21T22:32:28.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7614c9840 con 0x55d7615ddb40 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7613ad9a0 con 0x55d7615d89b0 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.645+0000 7f896db0eb80 10 client.0 _ll_register_callbacks cb 0x55d7614c77e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7615e1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7615e1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53284/0 (socket says 172.21.15.66:53284) 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 -- 172.21.15.66:0/946440882 learned_addr learned my addr 172.21.15.66:0/946440882 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8953fff700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d7615e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8953fff700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 -- 172.21.15.66:0/946440882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 msgr2=0x55d7615e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d7615e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 -- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 msgr2=0x55d7615e2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 -- 172.21.15.66:0/946440882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89541a2990 con 0x55d7615dd230 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8953fff700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:32:28.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8952ffd700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7615e1ef0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f89480117c0 tx=0x7f8948011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bc704e5df204d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.646+0000 7f8932ffd700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f8932ffd700 1 -- 172.21.15.66:0/946440882 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948016070 con 0x55d7615dd230 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d761483570 con 0x55d7615dd230 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f8932ffd700 1 -- 172.21.15.66:0/946440882 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f894800f910 con 0x55d7615dd230 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f8932ffd700 1 -- 172.21.15.66:0/946440882 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894800fad0 con 0x55d7615dd230 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f896db0eb80 10 client.16941 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f894c0112b0 con 0x55d7615dd230 2023-07-21T22:32:28.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.647+0000 7f8932ffd700 1 -- 172.21.15.66:0/946440882 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d761483570 con 0x55d7615dd230 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f8932ffd700 1 -- 172.21.15.66:0/946440882 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f894801ad00 con 0x55d7615dd230 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f896db0eb80 10 client.16941 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f896db0eb80 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f896db0eb80 20 client.16941 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f896db0eb80 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d7615f6a50 0x55d7615f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f896db0eb80 4 client.16941 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:28.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d761522f30 con 0x55d7615f6a50 2023-07-21T22:32:28.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f8953fff700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d7615f6a50 0x55d7615f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f8932ffd700 10 client.16941 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:28.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.648+0000 7f8953fff700 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d7615f6a50 0x55d7615f8e60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5151413b95dfa7b5 server_cookie=d84a0d446a3551a6 in_seq=0 out_seq=0 2023-07-21T22:32:28.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f8932ffd700 1 -- 172.21.15.66:0/946440882 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d761522f30 con 0x55d7615f6a50 2023-07-21T22:32:28.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f8932ffd700 10 client.16941 handle_command_reply: tid=0 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 client.16941 shutdown 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 msgr2=0x55d7615e1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7615e1ef0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f89480117c0 tx=0x7f8948011b80 comp rx=0 tx=0).stop 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 shutdown_connections 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d7615f6a50 0x55d7615f8e60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d7615e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f8932ffd700 0 client.16941 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:28.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 --2- 172.21.15.66:0/946440882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7615e1ef0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.649+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 >> 172.21.15.66:0/946440882 conn(0x55d7615cdf70 msgr2=0x55d7615d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.650+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 shutdown_connections 2023-07-21T22:32:28.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.650+0000 7f896db0eb80 1 -- 172.21.15.66:0/946440882 wait complete. 2023-07-21T22:32:28.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.650+0000 7f896db0eb80 20 client.16941 trim_cache size 0 max 16384 2023-07-21T22:32:28.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 Processor -- start 2023-07-21T22:32:28.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 -- start start 2023-07-21T22:32:28.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7615d9520 con 0x55d7615d89b0 2023-07-21T22:32:28.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7615e3000 con 0x55d7615ddb40 2023-07-21T22:32:28.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f896db0eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7615e2c10 con 0x55d7615dd230 2023-07-21T22:32:28.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f8952ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.669+0000 7f8953fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615e47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46116/0 (socket says 172.21.15.66:46116) 2023-07-21T22:32:28.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 -- 172.21.15.66:0/2776533519 learned_addr learned my addr 172.21.15.66:0/2776533519 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:28.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8953fff700 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 -- 172.21.15.66:0/2776533519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 msgr2=0x55d7615e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 -- 172.21.15.66:0/2776533519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 msgr2=0x55d7615d8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 -- 172.21.15.66:0/2776533519 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761340ce0 con 0x55d7615dd230 2023-07-21T22:32:28.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f8952ffd700 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615e47a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8948019410 tx=0x7f8948019630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566552a9529800b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f893a7fc700 1 -- 172.21.15.66:0/2776533519 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948016070 con 0x55d7615dd230 2023-07-21T22:32:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f893a7fc700 1 -- 172.21.15.66:0/2776533519 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894800fdd0 con 0x55d7615dd230 2023-07-21T22:32:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.670+0000 7f893a7fc700 1 -- 172.21.15.66:0/2776533519 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948005330 con 0x55d7615dd230 2023-07-21T22:32:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 -- 172.21.15.66:0/2776533519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 msgr2=0x55d7615e47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615e47a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8948019410 tx=0x7f8948019630 comp rx=0 tx=0).stop 2023-07-21T22:32:28.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 -- 172.21.15.66:0/2776533519 shutdown_connections 2023-07-21T22:32:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615d89b0 0x55d7615d8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 --2- 172.21.15.66:0/2776533519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615dd230 0x55d7615e47a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 -- 172.21.15.66:0/2776533519 >> 172.21.15.66:0/2776533519 conn(0x55d7615cdf70 msgr2=0x55d7615d7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 -- 172.21.15.66:0/2776533519 shutdown_connections 2023-07-21T22:32:28.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.671+0000 7f896db0eb80 1 -- 172.21.15.66:0/2776533519 wait complete. 2023-07-21T22:32:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.672+0000 7f896db0eb80 1 -- start start 2023-07-21T22:32:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.672+0000 7f896db0eb80 1 Processor -- start 2023-07-21T22:32:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d761643bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7616440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7615ecf80 con 0x55d7615dd230 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7615ed0f0 con 0x55d7615ddb40 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7615ed2b0 con 0x55d7615d89b0 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f896db0eb80 10 client.0 _ll_register_callbacks cb 0x7f8948009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d761643bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d761643bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46118/0 (socket says 172.21.15.66:46118) 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8952ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7616440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 -- 172.21.15.66:0/1927679834 learned_addr learned my addr 172.21.15.66:0/1927679834 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 -- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 msgr2=0x55d7615ec9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615ec9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 -- 172.21.15.66:0/1927679834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 msgr2=0x55d7616440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7616440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8953fff700 1 -- 172.21.15.66:0/1927679834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c0112b0 con 0x55d7615d89b0 2023-07-21T22:32:28.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.673+0000 7f8952ffd700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7616440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f8953fff700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d761643bb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f894c011bf0 tx=0x7f894c022040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a4be61990b03e8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f8932ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f8932ffd700 1 -- 172.21.15.66:0/1927679834 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c025040 con 0x55d7615d89b0 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f894c00f040 con 0x55d7615d89b0 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f8932ffd700 1 -- 172.21.15.66:0/1927679834 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f894c004a90 con 0x55d7615d89b0 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f8932ffd700 1 -- 172.21.15.66:0/1927679834 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f894c00e040 con 0x55d7615d89b0 2023-07-21T22:32:28.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f896db0eb80 10 client.26986 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d76137e2d0 con 0x55d7615d89b0 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.674+0000 7f8932ffd700 1 -- 172.21.15.66:0/1927679834 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894c00f040 con 0x55d7615d89b0 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f8932ffd700 1 -- 172.21.15.66:0/1927679834 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f894c004680 con 0x55d7615d89b0 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f896db0eb80 10 client.26986 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f896db0eb80 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f896db0eb80 20 client.26986 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d76164cb60 0x55d76164ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f896db0eb80 4 client.26986 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d7614e50d0 con 0x55d76164cb60 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f8952ffd700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d76164cb60 0x55d76164ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f8952ffd700 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d76164cb60 0x55d76164ef70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7e6b78cd8c15fc server_cookie=8967234cfee2f3be in_seq=0 out_seq=0 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.675+0000 7f8932ffd700 10 client.26986 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.676+0000 7f8932ffd700 1 -- 172.21.15.66:0/1927679834 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d7614e50d0 con 0x55d76164cb60 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.676+0000 7f8932ffd700 10 client.26986 handle_command_reply: tid=0 2023-07-21T22:32:28.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.676+0000 7f896db0eb80 1 client.26986 shutdown 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 msgr2=0x55d761643bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d761643bb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f894c011bf0 tx=0x7f894c022040 comp rx=0 tx=0).stop 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 shutdown_connections 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7615ddb40 0x55d7615ec9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d76164cb60 0x55d76164ef70 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d7615d89b0 0x55d761643bb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 --2- 172.21.15.66:0/1927679834 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d7615dd230 0x55d7616440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 >> 172.21.15.66:0/1927679834 conn(0x55d7615cdf70 msgr2=0x55d7615dbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f8932ffd700 0 client.26986 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 shutdown_connections 2023-07-21T22:32:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 1 -- 172.21.15.66:0/1927679834 wait complete. 2023-07-21T22:32:28.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.677+0000 7f896db0eb80 20 client.26986 trim_cache size 0 max 16384 2023-07-21T22:32:28.732 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:32:28.733 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:32:28.733 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "b4aae4b9-0076-4ccb-86ed-a7bb8866cd2f", 2023-07-21T22:32:28.733 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:32:28.733 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:32:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f893c0a12c0 msgr2=0x7f893c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f893c0a12c0 0x7f893c0a3770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55d7615d4ef0 tx=0x7f89440d1e90 comp rx=0 tx=0).stop 2023-07-21T22:32:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 msgr2=0x7f895419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f895419b9a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8944005910 tx=0x7f8944005720 comp rx=0 tx=0).stop 2023-07-21T22:32:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 shutdown_connections 2023-07-21T22:32:28.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8954105af0 0x7f8954195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f893c0a12c0 0x7f893c0a3770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89540ffae0 0x7f895419b9a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 --2- 172.21.15.66:0/3523656605 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8954100490 0x7f895419bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 >> 172.21.15.66:0/3523656605 conn(0x7f89540fb5e0 msgr2=0x7f89541035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 shutdown_connections 2023-07-21T22:32:28.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.678+0000 7f895a1ff700 1 -- 172.21.15.66:0/3523656605 wait complete. 2023-07-21T22:32:28.750 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:32:28.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.933+0000 7f3e253ff700 1 -- 172.21.15.66:0/496474970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 msgr2=0x7f3e200fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.933+0000 7f3e253ff700 1 --2- 172.21.15.66:0/496474970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e200fff50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009990 tx=0x7f3e08009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.936+0000 7f3e253ff700 1 -- 172.21.15.66:0/496474970 shutdown_connections 2023-07-21T22:32:28.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.936+0000 7f3e253ff700 1 --2- 172.21.15.66:0/496474970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e20100490 0x7f3e2010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.936+0000 7f3e253ff700 1 --2- 172.21.15.66:0/496474970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e20105af0 0x7f3e20105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.936+0000 7f3e253ff700 1 --2- 172.21.15.66:0/496474970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e200fff50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.936+0000 7f3e253ff700 1 -- 172.21.15.66:0/496474970 >> 172.21.15.66:0/496474970 conn(0x7f3e200fb5e0 msgr2=0x7f3e200fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:28.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.938+0000 7f3e253ff700 1 -- 172.21.15.66:0/496474970 shutdown_connections 2023-07-21T22:32:28.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.938+0000 7f3e253ff700 1 -- 172.21.15.66:0/496474970 wait complete. 2023-07-21T22:32:28.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.940+0000 7f3e253ff700 1 Processor -- start 2023-07-21T22:32:28.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 -- start start 2023-07-21T22:32:28.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e2006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e20100490 0x7f3e2006afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e20105af0 0x7f3e201aa130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:28.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201aa730 con 0x7f3e20105af0 2023-07-21T22:32:28.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201aa8a0 con 0x7f3e200ffae0 2023-07-21T22:32:28.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e253ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e201aaa60 con 0x7f3e20100490 2023-07-21T22:32:28.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e1effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e2006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e1effd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e2006aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45320/0 (socket says 172.21.15.66:45320) 2023-07-21T22:32:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e1effd700 1 -- 172.21.15.66:0/241734155 learned_addr learned my addr 172.21.15.66:0/241734155 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.941+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e20105af0 0x7f3e201aa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.943+0000 7f3e1effd700 1 -- 172.21.15.66:0/241734155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e20100490 msgr2=0x7f3e2006afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.943+0000 7f3e1effd700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e20100490 0x7f3e2006afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.943+0000 7f3e1effd700 1 -- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e20105af0 msgr2=0x7f3e201aa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:28.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.943+0000 7f3e1effd700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e20105af0 0x7f3e201aa130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:28.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.943+0000 7f3e1effd700 1 -- 172.21.15.66:0/241734155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080095f0 con 0x7f3e200ffae0 2023-07-21T22:32:28.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.943+0000 7f3e1effd700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e2006aa70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000ef10 tx=0x7f3e1000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:28.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.944+0000 7f3e1cff9700 1 -- 172.21.15.66:0/241734155 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1000ca70 con 0x7f3e200ffae0 2023-07-21T22:32:28.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.944+0000 7f3e1cff9700 1 -- 172.21.15.66:0/241734155 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e1000f040 con 0x7f3e200ffae0 2023-07-21T22:32:28.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.944+0000 7f3e1cff9700 1 -- 172.21.15.66:0/241734155 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e100146e0 con 0x7f3e200ffae0 2023-07-21T22:32:28.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.945+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e201aace0 con 0x7f3e200ffae0 2023-07-21T22:32:28.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.945+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e201ab1a0 con 0x7f3e200ffae0 2023-07-21T22:32:28.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.952+0000 7f3e1cff9700 1 -- 172.21.15.66:0/241734155 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f3e10007500 con 0x7f3e200ffae0 2023-07-21T22:32:29.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.954+0000 7f3e38c59b80 1 Processor -- start 2023-07-21T22:32:29.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.955+0000 7f3e38c59b80 1 -- start start 2023-07-21T22:32:29.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.956+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb604580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.956+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb5ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.956+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.956+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb504cb0 con 0x5642fb604ac0 2023-07-21T22:32:29.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.956+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb557a40 con 0x5642fb5ff930 2023-07-21T22:32:29.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.956+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb550ed0 con 0x5642fb6041b0 2023-07-21T22:32:29.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.957+0000 7f3e1effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb5ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.957+0000 7f3e1effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb5ff350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:53310/0 (socket says 172.21.15.66:53310) 2023-07-21T22:32:29.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.957+0000 7f3e1effd700 1 -- 172.21.15.66:0/3463895465 learned_addr learned my addr 172.21.15.66:0/3463895465 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:29.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.958+0000 7f3e1effd700 1 -- 172.21.15.66:0/3463895465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 msgr2=0x5642fb604580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.005 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.958+0000 7f3e1effd700 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb604580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.958+0000 7f3e1effd700 1 -- 172.21.15.66:0/3463895465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 msgr2=0x5642fb5ffda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:32:29.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.958+0000 7f3e1effd700 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ffda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.958+0000 7f3e1effd700 1 -- 172.21.15.66:0/3463895465 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642fb37ede0 con 0x5642fb604ac0 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.959+0000 7f3e1cff9700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f3e0c0a1340 0x7f3e0c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.959+0000 7f3e1effd700 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb5ff350 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3e100ca500 tx=0x7f3e100d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdcd4e818ca4d08 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.960+0000 7f3e157fa700 1 -- 172.21.15.66:0/3463895465 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10096250 con 0x5642fb604ac0 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.960+0000 7f3e157fa700 1 -- 172.21.15.66:0/3463895465 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e10099ce0 con 0x5642fb604ac0 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.960+0000 7f3e157fa700 1 -- 172.21.15.66:0/3463895465 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e10099f10 con 0x5642fb604ac0 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.960+0000 7f3e1cff9700 1 -- 172.21.15.66:0/241734155 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3e100cd1e0 con 0x7f3e200ffae0 2023-07-21T22:32:29.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.960+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f3e0c0a1340 0x7f3e0c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3463895465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 msgr2=0x5642fb5ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:29.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb5ff350 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3e100ca500 tx=0x7f3e100d5550 comp rx=0 tx=0).stop 2023-07-21T22:32:29.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f3e0c0a1340 0x7f3e0c0a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800b620 tx=0x7f3e1800b5b0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:29.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3463895465 shutdown_connections 2023-07-21T22:32:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb604580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3463895465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb5ff350 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3463895465 >> 172.21.15.66:0/3463895465 conn(0x5642fb5f4ef0 msgr2=0x5642fb5fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:29.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3463895465 shutdown_connections 2023-07-21T22:32:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.961+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3463895465 wait complete. 2023-07-21T22:32:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.962+0000 7f3e38c59b80 1 -- start start 2023-07-21T22:32:29.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.962+0000 7f3e38c59b80 1 Processor -- start 2023-07-21T22:32:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb5ff930 0x5642fb60f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb6041b0 0x5642fb60f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb60fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb4f0ac0 con 0x5642fb6041b0 2023-07-21T22:32:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb4f07c0 con 0x5642fb604ac0 2023-07-21T22:32:29.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb3d4920 con 0x5642fb5ff930 2023-07-21T22:32:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.963+0000 7f3e38c59b80 10 client.0 _ll_register_callbacks cb 0x5642fb4ee760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:29.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb60fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb60fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:45342/0 (socket says 172.21.15.66:45342) 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 -- 172.21.15.66:0/3418798667 learned_addr learned my addr 172.21.15.66:0/3418798667 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1effd700 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb5ff930 0x5642fb60f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 -- 172.21.15.66:0/3418798667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb5ff930 msgr2=0x5642fb60f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb5ff930 0x5642fb60f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 -- 172.21.15.66:0/3418798667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb6041b0 msgr2=0x5642fb60f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb6041b0 0x5642fb60f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.964+0000 7f3e1f7fe700 1 -- 172.21.15.66:0/3418798667 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e201ab1a0 con 0x5642fb604ac0 2023-07-21T22:32:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.965+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb60fcd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800bfd0 tx=0x7f3e1800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504c7c3d59b0bb3e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.965+0000 7f3e067fc700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.966+0000 7f3e067fc700 1 -- 172.21.15.66:0/3418798667 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18011ce0 con 0x5642fb604ac0 2023-07-21T22:32:29.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.966+0000 7f3e067fc700 1 -- 172.21.15.66:0/3418798667 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3e18017070 con 0x5642fb604ac0 2023-07-21T22:32:29.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.966+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642fb4aa4f0 con 0x5642fb604ac0 2023-07-21T22:32:29.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.968+0000 7f3e067fc700 1 -- 172.21.15.66:0/3418798667 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e18011e40 con 0x5642fb604ac0 2023-07-21T22:32:29.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.968+0000 7f3e38c59b80 10 client.27011 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:29.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.968+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5642fb367c60 con 0x5642fb604ac0 2023-07-21T22:32:29.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e067fc700 1 -- 172.21.15.66:0/3418798667 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642fb4aa4f0 con 0x5642fb604ac0 2023-07-21T22:32:29.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e067fc700 1 -- 172.21.15.66:0/3418798667 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3e1801e6d0 con 0x5642fb604ac0 2023-07-21T22:32:29.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e38c59b80 10 client.27011 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:29.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e38c59b80 10 client.27011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e38c59b80 20 client.27011 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb622000 0x5642fb624410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e38c59b80 4 client.27011 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.969+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642fb549eb0 con 0x5642fb622000 2023-07-21T22:32:29.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.970+0000 7f3e1effd700 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb622000 0x5642fb624410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.972+0000 7f3e067fc700 10 client.27011 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:29.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.972+0000 7f3e1effd700 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb622000 0x5642fb624410 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68942b358cf2917b server_cookie=5a850ab6bef22159 in_seq=0 out_seq=0 2023-07-21T22:32:29.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.974+0000 7f3e067fc700 1 -- 172.21.15.66:0/3418798667 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5642fb549eb0 con 0x5642fb622000 2023-07-21T22:32:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.974+0000 7f3e067fc700 10 client.27011 handle_command_reply: tid=0 2023-07-21T22:32:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.974+0000 7f3e38c59b80 1 client.27011 shutdown 2023-07-21T22:32:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 msgr2=0x5642fb60fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:29.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb60fcd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800bfd0 tx=0x7f3e1800b390 comp rx=0 tx=0).stop 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 shutdown_connections 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb60fcd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb622000 0x5642fb624410 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb5ff930 0x5642fb60f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/3418798667 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb6041b0 0x5642fb60f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 >> 172.21.15.66:0/3418798667 conn(0x5642fb5f4ef0 msgr2=0x5642fb5fa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:29.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.975+0000 7f3e067fc700 0 client.27011 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:29.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.976+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 shutdown_connections 2023-07-21T22:32:29.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.976+0000 7f3e38c59b80 1 -- 172.21.15.66:0/3418798667 wait complete. 2023-07-21T22:32:29.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:28.978+0000 7f3e38c59b80 20 client.27011 trim_cache size 0 max 16384 2023-07-21T22:32:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 Processor -- start 2023-07-21T22:32:29.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 -- start start 2023-07-21T22:32:29.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb60f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb6085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb60be70 con 0x5642fb604ac0 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb5f49b0 con 0x5642fb5ff930 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.003+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb5f2050 con 0x5642fb6041b0 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb60f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb60f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46164/0 (socket says 172.21.15.66:46164) 2023-07-21T22:32:29.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 -- 172.21.15.66:0/19859632 learned_addr learned my addr 172.21.15.66:0/19859632 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb6085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e1effd700 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 -- 172.21.15.66:0/19859632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 msgr2=0x5642fb5ff150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ff150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 -- 172.21.15.66:0/19859632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 msgr2=0x5642fb6085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb6085e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 -- 172.21.15.66:0/19859632 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642fb3a5250 con 0x5642fb6041b0 2023-07-21T22:32:29.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.004+0000 7f3e17fff700 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb60f920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e080098e0 tx=0x7f3e08005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e74df84dc2b7567 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.005+0000 7f3e15ffb700 1 -- 172.21.15.66:0/19859632 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0801d070 con 0x5642fb6041b0 2023-07-21T22:32:29.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.005+0000 7f3e15ffb700 1 -- 172.21.15.66:0/19859632 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e0800bd10 con 0x5642fb6041b0 2023-07-21T22:32:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.005+0000 7f3e15ffb700 1 -- 172.21.15.66:0/19859632 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08005050 con 0x5642fb6041b0 2023-07-21T22:32:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.005+0000 7f3e38c59b80 1 -- 172.21.15.66:0/19859632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 msgr2=0x5642fb60f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.005+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb60f920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e080098e0 tx=0x7f3e08005c00 comp rx=0 tx=0).stop 2023-07-21T22:32:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 -- 172.21.15.66:0/19859632 shutdown_connections 2023-07-21T22:32:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb5ff930 0x5642fb5ff150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb604ac0 0x5642fb6085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/19859632 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb60f920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 -- 172.21.15.66:0/19859632 >> 172.21.15.66:0/19859632 conn(0x5642fb5f4ef0 msgr2=0x5642fb60f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 -- 172.21.15.66:0/19859632 shutdown_connections 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 -- 172.21.15.66:0/19859632 wait complete. 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 -- start start 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.006+0000 7f3e38c59b80 1 Processor -- start 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb5ff930 0x5642fb61e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb618ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb619400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb619b10 con 0x5642fb5ff930 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb619c80 con 0x5642fb604ac0 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642fb675eb0 con 0x5642fb6041b0 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e38c59b80 10 client.0 _ll_register_callbacks cb 0x7f3e180037f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e17fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb618ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e1effd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb5ff930 0x5642fb61e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e17fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb618ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:46172/0 (socket says 172.21.15.66:46172) 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e17fff700 1 -- 172.21.15.66:0/1746259850 learned_addr learned my addr 172.21.15.66:0/1746259850 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e1f7fe700 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb619400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e17fff700 1 -- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 msgr2=0x5642fb619400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.007+0000 7f3e17fff700 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb619400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e17fff700 1 -- 172.21.15.66:0/1746259850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb5ff930 msgr2=0x5642fb61e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e17fff700 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb5ff930 0x5642fb61e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e17fff700 1 -- 172.21.15.66:0/1746259850 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2010d220 con 0x5642fb6041b0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e17fff700 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb618ec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3e08000c00 tx=0x7f3e08005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c855b5d3c4f0980 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e067fc700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e067fc700 1 -- 172.21.15.66:0/1746259850 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0801d070 con 0x5642fb6041b0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e067fc700 1 -- 172.21.15.66:0/1746259850 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3e08005050 con 0x5642fb6041b0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e10099ce0 con 0x5642fb6041b0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e067fc700 1 -- 172.21.15.66:0/1746259850 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e08005210 con 0x5642fb6041b0 2023-07-21T22:32:29.060 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e38c59b80 10 client.26995 fetch_fsmap learned FSMap version 41 2023-07-21T22:32:29.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.008+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5642fb37d2b0 con 0x5642fb6041b0 2023-07-21T22:32:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e067fc700 1 -- 172.21.15.66:0/1746259850 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e10099ce0 con 0x5642fb6041b0 2023-07-21T22:32:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e067fc700 1 -- 172.21.15.66:0/1746259850 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 41) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3e0800fb40 con 0x5642fb6041b0 2023-07-21T22:32:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e38c59b80 10 client.26995 fetch_fsmap finished waiting for FSMap version 41 2023-07-21T22:32:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e38c59b80 10 client.26995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:29.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e38c59b80 20 client.26995 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb6770c0 0x5642fb6794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e38c59b80 4 client.26995 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5642fb50c050 con 0x5642fb6770c0 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.009+0000 7f3e1effd700 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb6770c0 0x5642fb6794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e067fc700 10 client.26995 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e1effd700 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb6770c0 0x5642fb6794a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a88e17e9ef5261ba server_cookie=ec9f0f46cacf42ba in_seq=0 out_seq=0 2023-07-21T22:32:29.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e067fc700 1 -- 172.21.15.66:0/1746259850 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5642fb50c050 con 0x5642fb6770c0 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e067fc700 10 client.26995 handle_command_reply: tid=0 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 client.26995 shutdown 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 msgr2=0x5642fb618ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb618ec0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3e08000c00 tx=0x7f3e08005ea0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 shutdown_connections 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb604ac0 0x5642fb619400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5642fb6770c0 0x5642fb6794a0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.064 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5642fb5ff930 0x5642fb61e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 --2- 172.21.15.66:0/1746259850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5642fb6041b0 0x5642fb618ec0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 >> 172.21.15.66:0/1746259850 conn(0x5642fb5f4ef0 msgr2=0x5642fb60f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:29.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.010+0000 7f3e067fc700 0 client.26995 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:29.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.011+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 shutdown_connections 2023-07-21T22:32:29.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.011+0000 7f3e38c59b80 1 -- 172.21.15.66:0/1746259850 wait complete. 2023-07-21T22:32:29.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.011+0000 7f3e38c59b80 20 client.26995 trim_cache size 0 max 16384 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout: "b4aae4b9-0076-4ccb-86ed-a7bb8866cd2f": { 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "b4aae4b9-0076-4ccb-86ed-a7bb8866cd2f", 2023-07-21T22:32:29.066 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:32:29.067 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:32:29.067 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:32:29.067 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:32:29.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f3e0c0a1340 msgr2=0x7f3e0c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:29.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f3e0c0a1340 0x7f3e0c0a37f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800b620 tx=0x7f3e1800b5b0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 msgr2=0x7f3e2006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:29.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e2006aa70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000ef10 tx=0x7f3e1000c3a0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 shutdown_connections 2023-07-21T22:32:29.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e200ffae0 0x7f3e2006aa70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f3e0c0a1340 0x7f3e0c0a37f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f3e20105af0 0x7f3e201aa130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 --2- 172.21.15.66:0/241734155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e20100490 0x7f3e2006afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 >> 172.21.15.66:0/241734155 conn(0x7f3e200fb5e0 msgr2=0x7f3e200fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 shutdown_connections 2023-07-21T22:32:29.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:29.012+0000 7f3e253ff700 1 -- 172.21.15.66:0/241734155 wait complete. 2023-07-21T22:32:29.075 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:28 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:32:29.075 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:28 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.075 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:28 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:32:29.075 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:28 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.078 INFO:tasks.cephfs.filesystem:scrub status for tag:b4aae4b9-0076-4ccb-86ed-a7bb8866cd2f - {'path': '/', 'tag': 'b4aae4b9-0076-4ccb-86ed-a7bb8866cd2f', 'options': 'recursive,force'} 2023-07-21T22:32:29.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:28 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:32:29.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:28 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:28 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:32:29.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:28 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:28 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:32:29.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:28 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:28 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:32:29.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:28 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:29.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 505202 5.23 MB/sec execute 476 sec latency 27.004 ms 2023-07-21T22:32:30.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:32:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:29 smithi104 ceph-mon[129021]: pgmap v343: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 917 KiB/s wr, 91 op/s 2023-07-21T22:32:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:32:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:32:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:32:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:32:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:29 smithi066 ceph-mon[126379]: pgmap v343: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 917 KiB/s wr, 91 op/s 2023-07-21T22:32:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:32:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:32:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:32:30.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:32:30.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:32:30.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:29 smithi149 ceph-mon[129268]: pgmap v343: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 917 KiB/s wr, 91 op/s 2023-07-21T22:32:30.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 506675 5.23 MB/sec execute 477 sec latency 16.786 ms 2023-07-21T22:32:31.567 INFO:tasks.workunit.client.0.smithi066.stdout: 1 508243 5.24 MB/sec execute 478 sec latency 83.735 ms 2023-07-21T22:32:32.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:31 smithi104 ceph-mon[129021]: pgmap v344: 129 pgs: 129 active+clean; 473 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 174 op/s 2023-07-21T22:32:32.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:31 smithi066 ceph-mon[126379]: pgmap v344: 129 pgs: 129 active+clean; 473 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 174 op/s 2023-07-21T22:32:32.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:31 smithi149 ceph-mon[129268]: pgmap v344: 129 pgs: 129 active+clean; 473 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 174 op/s 2023-07-21T22:32:32.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 509203 5.24 MB/sec execute 479 sec latency 15.521 ms 2023-07-21T22:32:33.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 510911 5.25 MB/sec execute 480 sec latency 19.032 ms 2023-07-21T22:32:34.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:34 smithi104 ceph-mon[129021]: pgmap v345: 129 pgs: 129 active+clean; 473 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2023-07-21T22:32:34.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:34 smithi066 ceph-mon[126379]: pgmap v345: 129 pgs: 129 active+clean; 473 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2023-07-21T22:32:34.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:34 smithi149 ceph-mon[129268]: pgmap v345: 129 pgs: 129 active+clean; 473 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2023-07-21T22:32:34.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 512050 5.25 MB/sec execute 481 sec latency 13.794 ms 2023-07-21T22:32:35.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 513114 5.25 MB/sec execute 482 sec latency 23.372 ms 2023-07-21T22:32:36.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:36 smithi104 ceph-mon[129021]: pgmap v346: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2023-07-21T22:32:36.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:36 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:32:36.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:36 smithi066 ceph-mon[126379]: pgmap v346: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2023-07-21T22:32:36.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:36 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:32:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:36 smithi149 ceph-mon[129268]: pgmap v346: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.5 MiB/s wr, 252 op/s 2023-07-21T22:32:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:36 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:32:36.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 514943 5.26 MB/sec execute 483 sec latency 19.308 ms 2023-07-21T22:32:37.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:37 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:32:37.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:37 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:32:37.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:37 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:32:37.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:37 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:32:37.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:37 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:32:37.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:37 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:32:37.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 515810 5.26 MB/sec execute 484 sec latency 65.966 ms 2023-07-21T22:32:38.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:38 smithi104 ceph-mon[129021]: pgmap v347: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-21T22:32:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:38 smithi066 ceph-mon[126379]: pgmap v347: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-21T22:32:38.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:38 smithi149 ceph-mon[129268]: pgmap v347: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-21T22:32:38.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 516995 5.26 MB/sec execute 485 sec latency 70.471 ms 2023-07-21T22:32:39.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:39 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:32:39.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:39 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:32:39.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:39 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:32:39.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 518472 5.27 MB/sec execute 486 sec latency 183.142 ms 2023-07-21T22:32:40.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:40 smithi104 ceph-mon[129021]: pgmap v348: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-21T22:32:40.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:40 smithi066 ceph-mon[126379]: pgmap v348: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-21T22:32:40.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:40 smithi149 ceph-mon[129268]: pgmap v348: 129 pgs: 129 active+clean; 482 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2023-07-21T22:32:40.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 518919 5.26 MB/sec execute 487 sec latency 804.466 ms 2023-07-21T22:32:41.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 519860 5.26 MB/sec execute 488 sec latency 18.238 ms 2023-07-21T22:32:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:42 smithi104 ceph-mon[129021]: pgmap v349: 129 pgs: 129 active+clean; 490 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2023-07-21T22:32:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:42 smithi066 ceph-mon[126379]: pgmap v349: 129 pgs: 129 active+clean; 490 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2023-07-21T22:32:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:42 smithi149 ceph-mon[129268]: pgmap v349: 129 pgs: 129 active+clean; 490 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 248 op/s 2023-07-21T22:32:42.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 521227 5.26 MB/sec execute 489 sec latency 21.874 ms 2023-07-21T22:32:43.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 522625 5.27 MB/sec execute 490 sec latency 18.999 ms 2023-07-21T22:32:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:44 smithi104 ceph-mon[129021]: pgmap v350: 129 pgs: 129 active+clean; 490 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 165 op/s 2023-07-21T22:32:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:44 smithi066 ceph-mon[126379]: pgmap v350: 129 pgs: 129 active+clean; 490 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 165 op/s 2023-07-21T22:32:44.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:44 smithi149 ceph-mon[129268]: pgmap v350: 129 pgs: 129 active+clean; 490 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.7 MiB/s wr, 165 op/s 2023-07-21T22:32:44.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 523732 5.27 MB/sec execute 491 sec latency 47.368 ms 2023-07-21T22:32:45.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 525586 5.28 MB/sec execute 492 sec latency 17.837 ms 2023-07-21T22:32:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:46 smithi104 ceph-mon[129021]: pgmap v351: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 246 op/s 2023-07-21T22:32:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:46 smithi066 ceph-mon[126379]: pgmap v351: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 246 op/s 2023-07-21T22:32:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:46 smithi149 ceph-mon[129268]: pgmap v351: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.6 MiB/s wr, 246 op/s 2023-07-21T22:32:46.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 526632 5.28 MB/sec execute 493 sec latency 14.540 ms 2023-07-21T22:32:47.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 528099 5.28 MB/sec execute 494 sec latency 33.057 ms 2023-07-21T22:32:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:48 smithi104 ceph-mon[129021]: pgmap v352: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:32:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:48 smithi066 ceph-mon[126379]: pgmap v352: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:32:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:48 smithi149 ceph-mon[129268]: pgmap v352: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:32:48.568 INFO:tasks.workunit.client.0.smithi066.stdout: 1 529601 5.29 MB/sec execute 495 sec latency 23.690 ms 2023-07-21T22:32:49.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 530692 5.29 MB/sec execute 496 sec latency 14.282 ms 2023-07-21T22:32:50.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:50 smithi104 ceph-mon[129021]: pgmap v353: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:32:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:50 smithi066 ceph-mon[126379]: pgmap v353: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:32:50.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:50 smithi149 ceph-mon[129268]: pgmap v353: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 158 op/s 2023-07-21T22:32:50.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 532213 5.30 MB/sec execute 497 sec latency 18.232 ms 2023-07-21T22:32:51.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 533440 5.30 MB/sec execute 498 sec latency 12.295 ms 2023-07-21T22:32:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:52 smithi104 ceph-mon[129021]: pgmap v354: 129 pgs: 129 active+clean; 509 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2023-07-21T22:32:52.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:52 smithi066 ceph-mon[126379]: pgmap v354: 129 pgs: 129 active+clean; 509 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2023-07-21T22:32:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:52 smithi149 ceph-mon[129268]: pgmap v354: 129 pgs: 129 active+clean; 509 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2023-07-21T22:32:52.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 534582 5.30 MB/sec execute 499 sec latency 16.177 ms 2023-07-21T22:32:53.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:53 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:53.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:53 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:53.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:53 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:32:53.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 536263 5.31 MB/sec execute 500 sec latency 46.331 ms 2023-07-21T22:32:54.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:54 smithi104 ceph-mon[129021]: pgmap v355: 129 pgs: 129 active+clean; 509 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 167 op/s 2023-07-21T22:32:54.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:54 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:54.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:54 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:54 smithi066 ceph-mon[126379]: pgmap v355: 129 pgs: 129 active+clean; 509 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 167 op/s 2023-07-21T22:32:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:54 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:54.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:54 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:54 smithi149 ceph-mon[129268]: pgmap v355: 129 pgs: 129 active+clean; 509 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 167 op/s 2023-07-21T22:32:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:54 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:54.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:54 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:54.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 537152 5.31 MB/sec execute 501 sec latency 30.696 ms 2023-07-21T22:32:55.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:55 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:55.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:55 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:55.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:55 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:55.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:55 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:55.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:55 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:55.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:55 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:55.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:55 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:32:55.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:55 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:32:55.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:55 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:32:55.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 538832 5.31 MB/sec execute 502 sec latency 17.713 ms 2023-07-21T22:32:56.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:56 smithi104 ceph-mon[129021]: pgmap v356: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2023-07-21T22:32:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:56 smithi066 ceph-mon[126379]: pgmap v356: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2023-07-21T22:32:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:56 smithi149 ceph-mon[129268]: pgmap v356: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.1 MiB/s wr, 264 op/s 2023-07-21T22:32:56.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 540270 5.32 MB/sec execute 503 sec latency 42.512 ms 2023-07-21T22:32:57.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 541382 5.32 MB/sec execute 504 sec latency 17.361 ms 2023-07-21T22:32:58.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:32:58 smithi104 ceph-mon[129021]: pgmap v357: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-07-21T22:32:58.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:32:58 smithi066 ceph-mon[126379]: pgmap v357: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-07-21T22:32:58.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:32:58 smithi149 ceph-mon[129268]: pgmap v357: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-07-21T22:32:58.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 542898 5.33 MB/sec execute 505 sec latency 18.030 ms 2023-07-21T22:32:59.079 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:32:59.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 -- 172.21.15.66:0/112571070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc8100480 msgr2=0x7fecc810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 --2- 172.21.15.66:0/112571070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc8100480 0x7fecc810c910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fecc000b0a0 tx=0x7fecc000b3b0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 -- 172.21.15.66:0/112571070 shutdown_connections 2023-07-21T22:32:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 --2- 172.21.15.66:0/112571070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc8105ae0 0x7fecc8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 --2- 172.21.15.66:0/112571070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc8100480 0x7fecc810c910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 --2- 172.21.15.66:0/112571070 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc80ffad0 0x7fecc80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 -- 172.21.15.66:0/112571070 >> 172.21.15.66:0/112571070 conn(0x7fecc80fb5f0 msgr2=0x7fecc80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 -- 172.21.15.66:0/112571070 shutdown_connections 2023-07-21T22:32:59.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.263+0000 7fecce9ff700 1 -- 172.21.15.66:0/112571070 wait complete. 2023-07-21T22:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 Processor -- start 2023-07-21T22:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 -- start start 2023-07-21T22:32:59.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 0x7fecc8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc8100480 0x7fecc8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc8105ae0 0x7fecc81a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc8073580 con 0x7fecc8105ae0 2023-07-21T22:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81a4400 con 0x7fecc8100480 2023-07-21T22:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.264+0000 7fecce9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81a4540 con 0x7fecc80ffad0 2023-07-21T22:32:59.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 0x7fecc8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 0x7fecc8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37484/0 (socket says 172.21.15.66:37484) 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 -- 172.21.15.66:0/3997721882 learned_addr learned my addr 172.21.15.66:0/3997721882 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecccf9c700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc8105ae0 0x7fecc81a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc77fe700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc8100480 0x7fecc8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 -- 172.21.15.66:0/3997721882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc8100480 msgr2=0x7fecc8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc8100480 0x7fecc8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 -- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc8105ae0 msgr2=0x7fecc81a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc8105ae0 0x7fecc81a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 -- 172.21.15.66:0/3997721882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0009d00 con 0x7fecc80ffad0 2023-07-21T22:32:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc7fff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 0x7fecc8072810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fecb800e9a0 tx=0x7fecb800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.265+0000 7fecc57fa700 1 -- 172.21.15.66:0/3997721882 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8009790 con 0x7fecc80ffad0 2023-07-21T22:32:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.266+0000 7fecc57fa700 1 -- 172.21.15.66:0/3997721882 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecb8010430 con 0x7fecc80ffad0 2023-07-21T22:32:59.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.266+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc81a47a0 con 0x7fecc80ffad0 2023-07-21T22:32:59.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.266+0000 7fecc57fa700 1 -- 172.21.15.66:0/3997721882 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8018680 con 0x7fecc80ffad0 2023-07-21T22:32:59.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.266+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc81a4cd0 con 0x7fecc80ffad0 2023-07-21T22:32:59.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.269+0000 7fecc57fa700 1 -- 172.21.15.66:0/3997721882 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fecb80105a0 con 0x7fecc80ffad0 2023-07-21T22:32:59.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.271+0000 7fecc57fa700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fecb00a1340 0x7fecb00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.271+0000 7fece22e1b80 1 Processor -- start 2023-07-21T22:32:59.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.271+0000 7fecc57fa700 1 -- 172.21.15.66:0/3997721882 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fecb8014070 con 0x7fecc80ffad0 2023-07-21T22:32:59.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.271+0000 7fecc77fe700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fecb00a1340 0x7fecb00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.271+0000 7fece22e1b80 1 -- start start 2023-07-21T22:32:59.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc77fe700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fecb00a1340 0x7fecb00a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fecc8074000 tx=0x7fecbc009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e35350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e35da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fece22e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98d3acb0 con 0x55cc98e3aac0 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fece22e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98d8da40 con 0x55cc98e3a1b0 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fece22e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98d86ed0 con 0x55cc98e35930 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42362/0 (socket says 172.21.15.66:42362) 2023-07-21T22:32:59.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc7fff700 1 -- 172.21.15.66:0/2407798930 learned_addr learned my addr 172.21.15.66:0/2407798930 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecccf9c700 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e35da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc77fe700 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e35350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc7fff700 1 -- 172.21.15.66:0/2407798930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 msgr2=0x55cc98e35da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.272+0000 7fecc7fff700 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e35da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecc7fff700 1 -- 172.21.15.66:0/2407798930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 msgr2=0x55cc98e35350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecc7fff700 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e35350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecc7fff700 1 -- 172.21.15.66:0/2407798930 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc98bb4de0 con 0x55cc98e3a1b0 2023-07-21T22:32:59.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecc7fff700 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3a580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fecb8096980 tx=0x7fecb80998f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f961b43c157c9f48 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecae7fc700 1 -- 172.21.15.66:0/2407798930 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80a2360 con 0x55cc98e3a1b0 2023-07-21T22:32:59.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecae7fc700 1 -- 172.21.15.66:0/2407798930 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecb8010880 con 0x55cc98e3a1b0 2023-07-21T22:32:59.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.273+0000 7fecae7fc700 1 -- 172.21.15.66:0/2407798930 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8099660 con 0x55cc98e3a1b0 2023-07-21T22:32:59.325 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 -- 172.21.15.66:0/2407798930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 msgr2=0x55cc98e3a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3a580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fecb8096980 tx=0x7fecb80998f0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 -- 172.21.15.66:0/2407798930 shutdown_connections 2023-07-21T22:32:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3a580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e35da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2407798930 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e35350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 -- 172.21.15.66:0/2407798930 >> 172.21.15.66:0/2407798930 conn(0x55cc98e2aef0 msgr2=0x55cc98e30830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 -- 172.21.15.66:0/2407798930 shutdown_connections 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 -- 172.21.15.66:0/2407798930 wait complete. 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.274+0000 7fece22e1b80 1 -- start start 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 Processor -- start 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 0x55cc98e41bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e3f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98d26ac0 con 0x55cc98e3a1b0 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98d267c0 con 0x55cc98e35930 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98c0a920 con 0x55cc98e3aac0 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.275+0000 7fece22e1b80 10 client.0 _ll_register_callbacks cb 0x55cc98d24760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 0x55cc98e41bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 0x55cc98e41bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42378/0 (socket says 172.21.15.66:42378) 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 -- 172.21.15.66:0/2087027075 learned_addr learned my addr 172.21.15.66:0/2087027075 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:59.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecccf9c700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e3f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc77fe700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 -- 172.21.15.66:0/2087027075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 msgr2=0x55cc98e3f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e3f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 -- 172.21.15.66:0/2087027075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3a1b0 msgr2=0x55cc98e3eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 -- 172.21.15.66:0/2087027075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc81a4cd0 con 0x55cc98e35930 2023-07-21T22:32:59.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc77fe700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7fecc7fff700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 0x55cc98e41bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cab10 tx=0x7fecb809a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1f8546864fd57fc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.276+0000 7feca77fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7feca77fe700 1 -- 172.21.15.66:0/2087027075 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80c8070 con 0x55cc98e35930 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc98ce04f0 con 0x55cc98e35930 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7feca77fe700 1 -- 172.21.15.66:0/2087027075 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fecb80a2360 con 0x55cc98e35930 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7feca77fe700 1 -- 172.21.15.66:0/2087027075 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb80982b0 con 0x55cc98e35930 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 10 client.27029 fetch_fsmap learned FSMap version 42 2023-07-21T22:32:59.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55cc98b9dc60 con 0x55cc98e35930 2023-07-21T22:32:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7feca77fe700 1 -- 172.21.15.66:0/2087027075 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc98ce04f0 con 0x55cc98e35930 2023-07-21T22:32:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7feca77fe700 1 -- 172.21.15.66:0/2087027075 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fecb8098e70 con 0x55cc98e35930 2023-07-21T22:32:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 10 client.27029 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:32:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 10 client.27029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 20 client.27029 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:59.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98e4f1e0 0x55cc98e515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 4 client.27029 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:32:59.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.277+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc98d7feb0 con 0x55cc98e4f1e0 2023-07-21T22:32:59.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.278+0000 7fecc77fe700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98e4f1e0 0x55cc98e515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.278+0000 7fecc77fe700 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98e4f1e0 0x55cc98e515f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6951b2c3404bae7 server_cookie=c04ecb17e8b7110a in_seq=0 out_seq=0 2023-07-21T22:32:59.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.278+0000 7feca77fe700 10 client.27029 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7feca77fe700 1 -- 172.21.15.66:0/2087027075 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc98d7feb0 con 0x55cc98e4f1e0 2023-07-21T22:32:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7feca77fe700 10 client.27029 handle_command_reply: tid=0 2023-07-21T22:32:59.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 client.27029 shutdown 2023-07-21T22:32:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 msgr2=0x55cc98e41bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 0x55cc98e41bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fecb80cab10 tx=0x7fecb809a420 comp rx=0 tx=0).stop 2023-07-21T22:32:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 shutdown_connections 2023-07-21T22:32:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e35930 0x55cc98e41bb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98e4f1e0 0x55cc98e515f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e3f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 --2- 172.21.15.66:0/2087027075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3a1b0 0x55cc98e3eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 >> 172.21.15.66:0/2087027075 conn(0x55cc98e2aef0 msgr2=0x55cc98e30420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7feca77fe700 0 client.27029 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 shutdown_connections 2023-07-21T22:32:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.279+0000 7fece22e1b80 1 -- 172.21.15.66:0/2087027075 wait complete. 2023-07-21T22:32:59.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.280+0000 7fece22e1b80 20 client.27029 trim_cache size 0 max 16384 2023-07-21T22:32:59.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.298+0000 7fece22e1b80 1 Processor -- start 2023-07-21T22:32:59.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.298+0000 7fece22e1b80 1 -- start start 2023-07-21T22:32:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e35930 0x55cc98e46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e41750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e33b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fece22e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98e27e50 con 0x55cc98e35930 2023-07-21T22:32:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fece22e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98e2f5e0 con 0x55cc98e3a1b0 2023-07-21T22:32:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fece22e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98e28030 con 0x55cc98e3aac0 2023-07-21T22:32:59.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecccf9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e33b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e35930 0x55cc98e46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecc77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e41750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecccf9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e33b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37504/0 (socket says 172.21.15.66:37504) 2023-07-21T22:32:59.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecccf9c700 1 -- 172.21.15.66:0/313308253 learned_addr learned my addr 172.21.15.66:0/313308253 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:59.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecc7fff700 1 -- 172.21.15.66:0/313308253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 msgr2=0x55cc98e33b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecc7fff700 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e33b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecc7fff700 1 -- 172.21.15.66:0/313308253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 msgr2=0x55cc98e41750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.299+0000 7fecc7fff700 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e41750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecc7fff700 1 -- 172.21.15.66:0/313308253 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc98bdb250 con 0x55cc98e35930 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecccf9c700 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e33b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecc77fe700 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e41750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:32:59.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecc7fff700 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e35930 0x55cc98e46580 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fecb8099ea0 tx=0x7fecb80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b64ec3a5047e6a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecaeffd700 1 -- 172.21.15.66:0/313308253 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80a2360 con 0x55cc98e35930 2023-07-21T22:32:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecaeffd700 1 -- 172.21.15.66:0/313308253 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecb80ca2b0 con 0x55cc98e35930 2023-07-21T22:32:59.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.300+0000 7fecaeffd700 1 -- 172.21.15.66:0/313308253 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80a2130 con 0x55cc98e35930 2023-07-21T22:32:59.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 -- 172.21.15.66:0/313308253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e35930 msgr2=0x55cc98e46580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e35930 0x55cc98e46580 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fecb8099ea0 tx=0x7fecb80092d0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 -- 172.21.15.66:0/313308253 shutdown_connections 2023-07-21T22:32:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e3aac0 0x55cc98e33b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e35930 0x55cc98e46580 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 --2- 172.21.15.66:0/313308253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e41750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 -- 172.21.15.66:0/313308253 >> 172.21.15.66:0/313308253 conn(0x55cc98e2aef0 msgr2=0x55cc98e4ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 -- 172.21.15.66:0/313308253 shutdown_connections 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 -- 172.21.15.66:0/313308253 wait complete. 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 -- start start 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.301+0000 7fece22e1b80 1 Processor -- start 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e462b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e44ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98e46bc0 con 0x55cc98e3aac0 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98e45050 con 0x55cc98e3a1b0 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc98e45210 con 0x55cc98e35930 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fece22e1b80 10 client.0 _ll_register_callbacks cb 0x7fecbc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e462b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fecccf9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e44ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e462b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37512/0 (socket says 172.21.15.66:37512) 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fecc7fff700 1 -- 172.21.15.66:0/3551895190 learned_addr learned my addr 172.21.15.66:0/3551895190 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.302+0000 7fecc77fe700 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fecc7fff700 1 -- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 msgr2=0x55cc98e445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fecc7fff700 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fecc7fff700 1 -- 172.21.15.66:0/3551895190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 msgr2=0x55cc98e44ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fecc7fff700 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e44ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fecc7fff700 1 -- 172.21.15.66:0/3551895190 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc017070 con 0x55cc98e35930 2023-07-21T22:32:59.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fecc7fff700 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e462b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fecb8099c10 tx=0x7fecb80984c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976258331592c39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7feca77fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7feca77fe700 1 -- 172.21.15.66:0/3551895190 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb80109f0 con 0x55cc98e35930 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecb8010880 con 0x55cc98e35930 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7feca77fe700 1 -- 172.21.15.66:0/3551895190 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fecb80a2360 con 0x55cc98e35930 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7feca77fe700 1 -- 172.21.15.66:0/3551895190 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecb80a20d0 con 0x55cc98e35930 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fece22e1b80 10 client.27013 fetch_fsmap learned FSMap version 42 2023-07-21T22:32:59.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.303+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fecc810d210 con 0x55cc98e35930 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7feca77fe700 1 -- 172.21.15.66:0/3551895190 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecb8010880 con 0x55cc98e35930 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7feca77fe700 1 -- 172.21.15.66:0/3551895190 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fecb800cae0 con 0x55cc98e35930 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fece22e1b80 10 client.27013 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fece22e1b80 10 client.27013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fece22e1b80 20 client.27013 populate_metadata read hostname 'smithi066' 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fece22e1b80 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98ea3f70 0x55cc98ea6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fece22e1b80 4 client.27013 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:32:59.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc98d42050 con 0x55cc98ea3f70 2023-07-21T22:32:59.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.304+0000 7fecc77fe700 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98ea3f70 0x55cc98ea6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7feca77fe700 10 client.27013 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:59.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fecc77fe700 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98ea3f70 0x55cc98ea6350 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656c937a60dbbf8b server_cookie=65efddd235da4582 in_seq=0 out_seq=0 2023-07-21T22:32:59.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7feca77fe700 1 -- 172.21.15.66:0/3551895190 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cc98d42050 con 0x55cc98ea3f70 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7feca77fe700 10 client.27013 handle_command_reply: tid=0 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 client.27013 shutdown 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 msgr2=0x55cc98e462b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e462b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fecb8099c10 tx=0x7fecb80984c0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 shutdown_connections 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55cc98ea3f70 0x55cc98ea6350 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55cc98e3aac0 0x55cc98e44ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55cc98e35930 0x55cc98e462b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7feca77fe700 0 client.27013 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:32:59.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 --2- 172.21.15.66:0/3551895190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cc98e3a1b0 0x55cc98e445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 >> 172.21.15.66:0/3551895190 conn(0x55cc98e2aef0 msgr2=0x55cc98e33b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.305+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 shutdown_connections 2023-07-21T22:32:59.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fece22e1b80 1 -- 172.21.15.66:0/3551895190 wait complete. 2023-07-21T22:32:59.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fece22e1b80 20 client.27013 trim_cache size 0 max 16384 2023-07-21T22:32:59.364 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:32:59.364 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:32:59.364 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:32:59.364 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:32:59.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fecb00a1340 msgr2=0x7fecb00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fecb00a1340 0x7fecb00a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fecc8074000 tx=0x7fecbc009260 comp rx=0 tx=0).stop 2023-07-21T22:32:59.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 msgr2=0x7fecc8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.365 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 0x7fecc8072810 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fecb800e9a0 tx=0x7fecb800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 shutdown_connections 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fecb00a1340 0x7fecb00a37f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fecc8105ae0 0x7fecc81a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecc80ffad0 0x7fecc8072810 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 --2- 172.21.15.66:0/3997721882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc8100480 0x7fecc8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.306+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 >> 172.21.15.66:0/3997721882 conn(0x7fecc80fb5f0 msgr2=0x7fecc80fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.307+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 shutdown_connections 2023-07-21T22:32:59.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.307+0000 7fecce9ff700 1 -- 172.21.15.66:0/3997721882 wait complete. 2023-07-21T22:32:59.372 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:32:59.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:32:59.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 543993 5.33 MB/sec execute 506 sec latency 64.987 ms 2023-07-21T22:32:59.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 -- 172.21.15.66:0/3636878747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c0ffaf0 msgr2=0x7f323c0fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 --2- 172.21.15.66:0/3636878747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c0ffaf0 0x7f323c0fff60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f322c009990 tx=0x7f322c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 -- 172.21.15.66:0/3636878747 shutdown_connections 2023-07-21T22:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 --2- 172.21.15.66:0/3636878747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c1004a0 0x7f323c10c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 --2- 172.21.15.66:0/3636878747 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f323c105b00 0x7f323c105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 --2- 172.21.15.66:0/3636878747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c0ffaf0 0x7f323c0fff60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 -- 172.21.15.66:0/3636878747 >> 172.21.15.66:0/3636878747 conn(0x7f323c0fb630 msgr2=0x7f323c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 -- 172.21.15.66:0/3636878747 shutdown_connections 2023-07-21T22:32:59.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.547+0000 7f32443ff700 1 -- 172.21.15.66:0/3636878747 wait complete. 2023-07-21T22:32:59.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.548+0000 7f32443ff700 1 Processor -- start 2023-07-21T22:32:59.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.548+0000 7f32443ff700 1 -- start start 2023-07-21T22:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f32443ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 0x7f323c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f32443ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c1004a0 0x7f323c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f32443ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f323c105b00 0x7f323c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f32443ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c19cdc0 con 0x7f323c105b00 2023-07-21T22:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f32443ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c19cf30 con 0x7f323c1004a0 2023-07-21T22:32:59.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f32443ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c19d0f0 con 0x7f323c0ffaf0 2023-07-21T22:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f324219b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 0x7f323c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f324219b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 0x7f323c197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37542/0 (socket says 172.21.15.66:37542) 2023-07-21T22:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f324219b700 1 -- 172.21.15.66:0/2457554030 learned_addr learned my addr 172.21.15.66:0/2457554030 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f324199a700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c1004a0 0x7f323c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.549+0000 7f324299c700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f323c105b00 0x7f323c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f324219b700 1 -- 172.21.15.66:0/2457554030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c1004a0 msgr2=0x7f323c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f324219b700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c1004a0 0x7f323c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f324219b700 1 -- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f323c105b00 msgr2=0x7f323c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f324219b700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f323c105b00 0x7f323c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f324219b700 1 -- 172.21.15.66:0/2457554030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0095f0 con 0x7f323c0ffaf0 2023-07-21T22:32:59.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f324219b700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 0x7f323c197de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f323800e9a0 tx=0x7f323800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238009790 con 0x7f323c0ffaf0 2023-07-21T22:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c19d3d0 con 0x7f323c0ffaf0 2023-07-21T22:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3238010430 con 0x7f323c0ffaf0 2023-07-21T22:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238018680 con 0x7f323c0ffaf0 2023-07-21T22:32:59.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.550+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c1a4c10 con 0x7f323c0ffaf0 2023-07-21T22:32:59.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.551+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c067300 con 0x7f323c0ffaf0 2023-07-21T22:32:59.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.553+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f32380105a0 con 0x7f323c0ffaf0 2023-07-21T22:32:59.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.555+0000 7f32337fe700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f32280a1300 0x7f32280a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:32:59.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.555+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3238014070 con 0x7f323c0ffaf0 2023-07-21T22:32:59.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.555+0000 7f324199a700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f32280a1300 0x7f32280a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:32:59.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.556+0000 7f324199a700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f32280a1300 0x7f32280a37b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f323c1993f0 tx=0x7f322c005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:32:59.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.559+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3238051770 con 0x7f323c0ffaf0 2023-07-21T22:32:59.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.874+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f323c199130 con 0x7f323c0ffaf0 2023-07-21T22:32:59.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.875+0000 7f32337fe700 1 -- 172.21.15.66:0/2457554030 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10827 (secure 0 0 0) 0x7f3238096260 con 0x7f323c0ffaf0 2023-07-21T22:32:59.923 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:32:59.923 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:32:38.011518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:32:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f32280a1300 msgr2=0x7f32280a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f32280a1300 0x7f32280a37b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f323c1993f0 tx=0x7f322c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 msgr2=0x7f323c197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 0x7f323c197de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f323800e9a0 tx=0x7f323800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 shutdown_connections 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f32280a1300 0x7f32280a37b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f323c105b00 0x7f323c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f323c0ffaf0 0x7f323c197de0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 --2- 172.21.15.66:0/2457554030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f323c1004a0 0x7f323c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:32:59.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 >> 172.21.15.66:0/2457554030 conn(0x7f323c0fb630 msgr2=0x7f323c10ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:32:59.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.878+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 shutdown_connections 2023-07-21T22:32:59.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:32:59.879+0000 7f32443ff700 1 -- 172.21.15.66:0/2457554030 wait complete. 2023-07-21T22:32:59.927 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 42 2023-07-21T22:32:59.947 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:33:00.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:00 smithi066 ceph-mon[126379]: pgmap v358: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-07-21T22:33:00.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2457554030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:33:00.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 -- 172.21.15.66:0/3777096862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c8105ae0 msgr2=0x7f76c8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 --2- 172.21.15.66:0/3777096862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c8105ae0 0x7f76c8105eb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009990 tx=0x7f76c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 -- 172.21.15.66:0/3777096862 shutdown_connections 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 --2- 172.21.15.66:0/3777096862 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c8100480 0x7f76c810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 --2- 172.21.15.66:0/3777096862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c8105ae0 0x7f76c8105eb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 --2- 172.21.15.66:0/3777096862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c80ffad0 0x7f76c80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 -- 172.21.15.66:0/3777096862 >> 172.21.15.66:0/3777096862 conn(0x7f76c80fb5f0 msgr2=0x7f76c80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 -- 172.21.15.66:0/3777096862 shutdown_connections 2023-07-21T22:33:00.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.167+0000 7f76cffff700 1 -- 172.21.15.66:0/3777096862 wait complete. 2023-07-21T22:33:00.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.168+0000 7f76cffff700 1 Processor -- start 2023-07-21T22:33:00.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.168+0000 7f76cffff700 1 -- start start 2023-07-21T22:33:00.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 0x7f76c8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c8100480 0x7f76c8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8105ae0 0x7f76c819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cffff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c819cda0 con 0x7f76c8100480 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c819cf10 con 0x7f76c8105ae0 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cffff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c819d0d0 con 0x7f76c80ffad0 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 0x7f76c8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 0x7f76c8197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37566/0 (socket says 172.21.15.66:37566) 2023-07-21T22:33:00.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cd59a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c8100480 0x7f76c8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 -- 172.21.15.66:0/132500624 learned_addr learned my addr 172.21.15.66:0/132500624 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76ce59c700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8105ae0 0x7f76c819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 -- 172.21.15.66:0/132500624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8105ae0 msgr2=0x7f76c819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8105ae0 0x7f76c819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 -- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c8100480 msgr2=0x7f76c8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c8100480 0x7f76c8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.169+0000 7f76cdd9b700 1 -- 172.21.15.66:0/132500624 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40095f0 con 0x7f76c80ffad0 2023-07-21T22:33:00.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.170+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 0x7f76c8197dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005910 tx=0x7f76c4005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.170+0000 7f76beffd700 1 -- 172.21.15.66:0/132500624 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c401d070 con 0x7f76c80ffad0 2023-07-21T22:33:00.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.170+0000 7f76beffd700 1 -- 172.21.15.66:0/132500624 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c4022470 con 0x7f76c80ffad0 2023-07-21T22:33:00.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.170+0000 7f76beffd700 1 -- 172.21.15.66:0/132500624 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4017650 con 0x7f76c80ffad0 2023-07-21T22:33:00.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.170+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c819d350 con 0x7f76c80ffad0 2023-07-21T22:33:00.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.170+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c819d5d0 con 0x7f76c80ffad0 2023-07-21T22:33:00.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.173+0000 7f76beffd700 1 -- 172.21.15.66:0/132500624 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f76c40225e0 con 0x7f76c80ffad0 2023-07-21T22:33:00.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76e39bbb80 1 Processor -- start 2023-07-21T22:33:00.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76e39bbb80 1 -- start start 2023-07-21T22:33:00.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76beffd700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76b40a1340 0x7f76b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76beffd700 1 -- 172.21.15.66:0/132500624 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f76c40ce190 con 0x7f76c80ffad0 2023-07-21T22:33:00.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.175+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2ac930 0x563f5d2acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d1b1cb0 con 0x563f5d2b1ac0 2023-07-21T22:33:00.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d204a40 con 0x563f5d2b11b0 2023-07-21T22:33:00.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d1fded0 con 0x563f5d2ac930 2023-07-21T22:33:00.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76ce59c700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76b40a1340 0x7f76b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42436/0 (socket says 172.21.15.66:42436) 2023-07-21T22:33:00.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2131345109 learned_addr learned my addr 172.21.15.66:0/2131345109 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cd59a700 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76ce59c700 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2ac930 0x563f5d2acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2131345109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2ac930 msgr2=0x563f5d2acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2ac930 0x563f5d2acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2131345109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 msgr2=0x563f5d2ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2131345109 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f5d02bde0 con 0x563f5d2b11b0 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cd59a700 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:00.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.176+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2b1580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76c40979c0 tx=0x7f76c4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d15bd35083d8cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.177+0000 7f76b3fff700 1 -- 172.21.15.66:0/2131345109 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c409a8b0 con 0x563f5d2b11b0 2023-07-21T22:33:00.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.177+0000 7f76ce59c700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76b40a1340 0x7f76b40a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009ec0 tx=0x7f76c0009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.177+0000 7f76b3fff700 1 -- 172.21.15.66:0/2131345109 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c409af50 con 0x563f5d2b11b0 2023-07-21T22:33:00.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.177+0000 7f76b3fff700 1 -- 172.21.15.66:0/2131345109 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c409b410 con 0x563f5d2b11b0 2023-07-21T22:33:00.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.177+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2131345109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 msgr2=0x563f5d2b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.177+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2b1580 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f76c40979c0 tx=0x7f76c4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2131345109 shutdown_connections 2023-07-21T22:33:00.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2b1580 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.230 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2ac930 0x563f5d2acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2131345109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2131345109 >> 172.21.15.66:0/2131345109 conn(0x563f5d2a1ef0 msgr2=0x563f5d2a7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2131345109 shutdown_connections 2023-07-21T22:33:00.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2131345109 wait complete. 2023-07-21T22:33:00.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 -- start start 2023-07-21T22:33:00.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.178+0000 7f76e39bbb80 1 Processor -- start 2023-07-21T22:33:00.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b11b0 0x563f5d2b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d19dac0 con 0x563f5d2b11b0 2023-07-21T22:33:00.232 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d19d7c0 con 0x563f5d2ac930 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d081920 con 0x563f5d2b1ac0 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76e39bbb80 10 client.0 _ll_register_callbacks cb 0x563f5d19b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42450/0 (socket says 172.21.15.66:42450) 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76cdd9b700 1 -- 172.21.15.66:0/3496821096 learned_addr learned my addr 172.21.15.66:0/3496821096 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76ce59c700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.179+0000 7f76cd59a700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b11b0 0x563f5d2b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76cdd9b700 1 -- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 msgr2=0x563f5d2b63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76cdd9b700 1 -- 172.21.15.66:0/3496821096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b11b0 msgr2=0x563f5d2b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b11b0 0x563f5d2b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76cdd9b700 1 -- 172.21.15.66:0/3496821096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c819d350 con 0x563f5d2ac930 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f76c409ac50 tx=0x7f76c40cb4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce185f2e8bc3b2d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:00.234 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76b17fa700 1 -- 172.21.15.66:0/3496821096 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c40a3310 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76b17fa700 1 -- 172.21.15.66:0/3496821096 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f76c409ef80 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f5d1574f0 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76b17fa700 1 -- 172.21.15.66:0/3496821096 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76c40228c0 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76e39bbb80 10 client.27053 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.180+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x563f5d014c60 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76b17fa700 1 -- 172.21.15.66:0/3496821096 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c407d020 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76b17fa700 1 -- 172.21.15.66:0/3496821096 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f76c40a2e80 con 0x563f5d2ac930 2023-07-21T22:33:00.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76e39bbb80 10 client.27053 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76e39bbb80 10 client.27053 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76e39bbb80 20 client.27053 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d2c5d40 0x563f5d2c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76e39bbb80 4 client.27053 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f5d1f6eb0 con 0x563f5d2c5d40 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.181+0000 7f76cd59a700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d2c5d40 0x563f5d2c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.182+0000 7f76cd59a700 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d2c5d40 0x563f5d2c8120 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1500c5e977ba811 server_cookie=ad6a29f7742ed60 in_seq=0 out_seq=0 2023-07-21T22:33:00.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.182+0000 7f76b17fa700 10 client.27053 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:00.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76b17fa700 1 -- 172.21.15.66:0/3496821096 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f5d1f6eb0 con 0x563f5d2c5d40 2023-07-21T22:33:00.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76b17fa700 10 client.27053 handle_command_reply: tid=0 2023-07-21T22:33:00.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 client.27053 shutdown 2023-07-21T22:33:00.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 msgr2=0x563f5d2b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2b8b90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f76c409ac50 tx=0x7f76c40cb4e0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 shutdown_connections 2023-07-21T22:33:00.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d2c5d40 0x563f5d2c8120 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2b8b90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3496821096 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b11b0 0x563f5d2b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 >> 172.21.15.66:0/3496821096 conn(0x563f5d2a1ef0 msgr2=0x563f5d2a73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76b17fa700 0 client.27053 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 shutdown_connections 2023-07-21T22:33:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3496821096 wait complete. 2023-07-21T22:33:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.183+0000 7f76e39bbb80 20 client.27053 trim_cache size 0 max 16384 2023-07-21T22:33:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 Processor -- start 2023-07-21T22:33:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 -- start start 2023-07-21T22:33:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2ac930 0x563f5d2bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d29f050 con 0x563f5d2ac930 2023-07-21T22:33:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d2c21e0 con 0x563f5d2b11b0 2023-07-21T22:33:00.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.198+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d2a7080 con 0x563f5d2b1ac0 2023-07-21T22:33:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2a72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42466/0 (socket says 172.21.15.66:42466) 2023-07-21T22:33:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 -- 172.21.15.66:0/3462241780 learned_addr learned my addr 172.21.15.66:0/3462241780 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2ac930 0x563f5d2bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76ce59c700 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 -- 172.21.15.66:0/3462241780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 msgr2=0x563f5d2b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 -- 172.21.15.66:0/3462241780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2ac930 msgr2=0x563f5d2bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2ac930 0x563f5d2bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 -- 172.21.15.66:0/3462241780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f5d052250 con 0x563f5d2b11b0 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.199+0000 7f76cd59a700 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2a72d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x563f5d2ad5b0 tx=0x7f76b8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a444c2542b53ddf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76bcff9700 1 -- 172.21.15.66:0/3462241780 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8017040 con 0x563f5d2b11b0 2023-07-21T22:33:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76bcff9700 1 -- 172.21.15.66:0/3462241780 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76b8023070 con 0x563f5d2b11b0 2023-07-21T22:33:00.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76bcff9700 1 -- 172.21.15.66:0/3462241780 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b80053a0 con 0x563f5d2b11b0 2023-07-21T22:33:00.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3462241780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 msgr2=0x563f5d2a72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2a72d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x563f5d2ad5b0 tx=0x7f76b8009260 comp rx=0 tx=0).stop 2023-07-21T22:33:00.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3462241780 shutdown_connections 2023-07-21T22:33:00.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b1ac0 0x563f5d2b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2ac930 0x563f5d2bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/3462241780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2b11b0 0x563f5d2a72d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3462241780 >> 172.21.15.66:0/3462241780 conn(0x563f5d2a1ef0 msgr2=0x563f5d2c1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3462241780 shutdown_connections 2023-07-21T22:33:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.200+0000 7f76e39bbb80 1 -- 172.21.15.66:0/3462241780 wait complete. 2023-07-21T22:33:00.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.201+0000 7f76e39bbb80 1 -- start start 2023-07-21T22:33:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.201+0000 7f76e39bbb80 1 Processor -- start 2023-07-21T22:33:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.201+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2bd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.201+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b11b0 0x563f5d2bb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.201+0000 7f76e39bbb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2bbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d2bc470 con 0x563f5d2b1ac0 2023-07-21T22:33:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d2bc5e0 con 0x563f5d2ac930 2023-07-21T22:33:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76e39bbb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5d319be0 con 0x563f5d2b11b0 2023-07-21T22:33:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76e39bbb80 10 client.0 _ll_register_callbacks cb 0x7f76c4096c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2bd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2bd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42482/0 (socket says 172.21.15.66:42482) 2023-07-21T22:33:00.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2968567161 learned_addr learned my addr 172.21.15.66:0/2968567161 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cd59a700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b11b0 0x563f5d2bb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76ce59c700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2bbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b11b0 msgr2=0x563f5d2bb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b11b0 0x563f5d2bb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2968567161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 msgr2=0x563f5d2bbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2bbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 -- 172.21.15.66:0/2968567161 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b8009d70 con 0x563f5d2ac930 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76ce59c700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76cdd9b700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2bd470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f76c400f740 tx=0x7f76c4022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60bdcd4c96ed2917 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.202+0000 7f76b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76b17fa700 1 -- 172.21.15.66:0/2968567161 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c40aec60 con 0x563f5d2ac930 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76b17fa700 1 -- 172.21.15.66:0/2968567161 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f76c407e080 con 0x563f5d2ac930 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76c409af50 con 0x563f5d2ac930 2023-07-21T22:33:00.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76b17fa700 1 -- 172.21.15.66:0/2968567161 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76c40cb330 con 0x563f5d2ac930 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 10 client.27059 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x563f5d02a2b0 con 0x563f5d2ac930 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76b17fa700 1 -- 172.21.15.66:0/2968567161 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c409af50 con 0x563f5d2ac930 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76b17fa700 1 -- 172.21.15.66:0/2968567161 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f76c409b1b0 con 0x563f5d2ac930 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 10 client.27059 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 10 client.27059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 20 client.27059 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:00.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d31ae30 0x563f5d31d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 4 client.27059 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:33:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.203+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f5d1b9050 con 0x563f5d31ae30 2023-07-21T22:33:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.204+0000 7f76cd59a700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d31ae30 0x563f5d31d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.204+0000 7f76cd59a700 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d31ae30 0x563f5d31d210 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf4a0db4b06e66ba server_cookie=2dc3e115f6ce049a in_seq=0 out_seq=0 2023-07-21T22:33:00.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.204+0000 7f76b17fa700 10 client.27059 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.204+0000 7f76b17fa700 1 -- 172.21.15.66:0/2968567161 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f5d1b9050 con 0x563f5d31ae30 2023-07-21T22:33:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.204+0000 7f76b17fa700 10 client.27059 handle_command_reply: tid=0 2023-07-21T22:33:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.204+0000 7f76e39bbb80 1 client.27059 shutdown 2023-07-21T22:33:00.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 msgr2=0x563f5d2bd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2bd470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f76c400f740 tx=0x7f76c4022c90 comp rx=0 tx=0).stop 2023-07-21T22:33:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 shutdown_connections 2023-07-21T22:33:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x563f5d31ae30 0x563f5d31d210 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563f5d2ac930 0x563f5d2bd470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x563f5d2b1ac0 0x563f5d2bbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 --2- 172.21.15.66:0/2968567161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563f5d2b11b0 0x563f5d2bb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 >> 172.21.15.66:0/2968567161 conn(0x563f5d2a1ef0 msgr2=0x563f5d2c2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76b17fa700 0 client.27059 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 shutdown_connections 2023-07-21T22:33:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 1 -- 172.21.15.66:0/2968567161 wait complete. 2023-07-21T22:33:00.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.205+0000 7f76e39bbb80 20 client.27059 trim_cache size 0 max 16384 2023-07-21T22:33:00.261 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:33:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76b40a1340 msgr2=0x7f76b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76b40a1340 0x7f76b40a37f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f76c0009ec0 tx=0x7f76c0009920 comp rx=0 tx=0).stop 2023-07-21T22:33:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 msgr2=0x7f76c8197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 0x7f76c8197dc0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76c4005910 tx=0x7f76c4005720 comp rx=0 tx=0).stop 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 shutdown_connections 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c8105ae0 0x7f76c819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f76b40a1340 0x7f76b40a37f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76c80ffad0 0x7f76c8197dc0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 --2- 172.21.15.66:0/132500624 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f76c8100480 0x7f76c8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 >> 172.21.15.66:0/132500624 conn(0x7f76c80fb5f0 msgr2=0x7f76c810ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 shutdown_connections 2023-07-21T22:33:00.263 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.206+0000 7f76cffff700 1 -- 172.21.15.66:0/132500624 wait complete. 2023-07-21T22:33:00.272 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:33:00.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:00 smithi104 ceph-mon[129021]: pgmap v358: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-07-21T22:33:00.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2457554030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:33:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:00 smithi149 ceph-mon[129268]: pgmap v358: 129 pgs: 129 active+clean; 518 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2023-07-21T22:33:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2457554030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:33:00.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 -- 172.21.15.66:0/2387564903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b94100ca0 msgr2=0x7f7b94101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 --2- 172.21.15.66:0/2387564903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b94100ca0 0x7f7b94101070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b88009990 tx=0x7f7b88009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 -- 172.21.15.66:0/2387564903 shutdown_connections 2023-07-21T22:33:00.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 --2- 172.21.15.66:0/2387564903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b94100ca0 0x7f7b94101070 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 --2- 172.21.15.66:0/2387564903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b9410a070 0x7f7b9410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 --2- 172.21.15.66:0/2387564903 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b941015b0 0x7f7b94109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 -- 172.21.15.66:0/2387564903 >> 172.21.15.66:0/2387564903 conn(0x7f7b940fa4b0 msgr2=0x7f7b940fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 -- 172.21.15.66:0/2387564903 shutdown_connections 2023-07-21T22:33:00.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.447+0000 7f7b9afff700 1 -- 172.21.15.66:0/2387564903 wait complete. 2023-07-21T22:33:00.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.448+0000 7f7b9afff700 1 Processor -- start 2023-07-21T22:33:00.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.448+0000 7f7b9afff700 1 -- start start 2023-07-21T22:33:00.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9afff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 0x7f7b94197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9afff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 0x7f7b94198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9afff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9410a070 0x7f7b9419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9afff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419ce10 con 0x7f7b94100ca0 2023-07-21T22:33:00.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9afff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419cf80 con 0x7f7b9410a070 2023-07-21T22:33:00.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9afff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9419d140 con 0x7f7b941015b0 2023-07-21T22:33:00.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 0x7f7b94197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 0x7f7b94198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 0x7f7b94197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51340/0 (socket says 172.21.15.66:51340) 2023-07-21T22:33:00.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 0x7f7b94198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37618/0 (socket says 172.21.15.66:37618) 2023-07-21T22:33:00.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b98d9b700 1 -- 172.21.15.66:0/3229022984 learned_addr learned my addr 172.21.15.66:0/3229022984 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b9959c700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9410a070 0x7f7b9419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 -- 172.21.15.66:0/3229022984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9410a070 msgr2=0x7f7b9419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9410a070 0x7f7b9419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 -- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 msgr2=0x7f7b94197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 0x7f7b94197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b93fff700 1 -- 172.21.15.66:0/3229022984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880095f0 con 0x7f7b941015b0 2023-07-21T22:33:00.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.449+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 0x7f7b94197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.450+0000 7f7b93fff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 0x7f7b94198370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9f0 tx=0x7f7b8000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.450+0000 7f7b91ffb700 1 -- 172.21.15.66:0/3229022984 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8000c9c0 con 0x7f7b941015b0 2023-07-21T22:33:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.450+0000 7f7b91ffb700 1 -- 172.21.15.66:0/3229022984 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b80010430 con 0x7f7b941015b0 2023-07-21T22:33:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.450+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9419d420 con 0x7f7b941015b0 2023-07-21T22:33:00.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.450+0000 7f7b91ffb700 1 -- 172.21.15.66:0/3229022984 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80018680 con 0x7f7b941015b0 2023-07-21T22:33:00.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.450+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b941a4c60 con 0x7f7b941015b0 2023-07-21T22:33:00.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.453+0000 7f7b91ffb700 1 -- 172.21.15.66:0/3229022984 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7b80010860 con 0x7f7b941015b0 2023-07-21T22:33:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.455+0000 7f7bae8bab80 1 Processor -- start 2023-07-21T22:33:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.455+0000 7f7b91ffb700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7b840a1340 0x7f7b840a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.455+0000 7f7bae8bab80 1 -- start start 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.455+0000 7f7b91ffb700 1 -- 172.21.15.66:0/3229022984 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7b80014070 con 0x7f7b941015b0 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b9959c700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7b840a1340 0x7f7b840a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc7120580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc711b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc711b930 0x55bfc711bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7020cb0 con 0x55bfc71201b0 2023-07-21T22:33:00.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7073a40 con 0x55bfc7120ac0 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc706ced0 con 0x55bfc711b930 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc7120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc711b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc711b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42508/0 (socket says 172.21.15.66:42508) 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc7120580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51342/0 (socket says 172.21.15.66:51342) 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 -- 172.21.15.66:0/3835595763 learned_addr learned my addr 172.21.15.66:0/3835595763 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b9959c700 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc711b930 0x55bfc711bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.518 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 -- 172.21.15.66:0/3835595763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc711b930 msgr2=0x55bfc711bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc711b930 0x55bfc711bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 -- 172.21.15.66:0/3835595763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 msgr2=0x55bfc7120580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc7120580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b93fff700 1 -- 172.21.15.66:0/3835595763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc6e9ade0 con 0x55bfc7120ac0 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.456+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc7120580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.457+0000 7f7b9959c700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7b840a1340 0x7f7b840a37f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009c10 tx=0x7f7b8c004040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.457+0000 7f7b93fff700 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc711b350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b80096910 tx=0x7f7b80099900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d19428e0e5c25b81 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.457+0000 7f7b7effd700 1 -- 172.21.15.66:0/3835595763 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800a22f0 con 0x55bfc7120ac0 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.457+0000 7f7b7effd700 1 -- 172.21.15.66:0/3835595763 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b800105a0 con 0x55bfc7120ac0 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.457+0000 7f7b7effd700 1 -- 172.21.15.66:0/3835595763 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800995d0 con 0x55bfc7120ac0 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3835595763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 msgr2=0x55bfc711b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc711b350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b80096910 tx=0x7f7b80099900 comp rx=0 tx=0).stop 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3835595763 shutdown_connections 2023-07-21T22:33:00.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc711b930 0x55bfc711bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc7120580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3835595763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc711b350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3835595763 >> 172.21.15.66:0/3835595763 conn(0x55bfc7110ef0 msgr2=0x55bfc7116830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3835595763 shutdown_connections 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.458+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3835595763 wait complete. 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 -- start start 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 Processor -- start 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc7124ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 0x55bfc7125430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc7125970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc700cac0 con 0x55bfc711b930 2023-07-21T22:33:00.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc700c7c0 con 0x55bfc71201b0 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6ef0920 con 0x55bfc7120ac0 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.459+0000 7f7bae8bab80 10 client.0 _ll_register_callbacks cb 0x55bfc700a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 0x55bfc7125430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 0x55bfc7125430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42514/0 (socket says 172.21.15.66:42514) 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 -- 172.21.15.66:0/95999705 learned_addr learned my addr 172.21.15.66:0/95999705 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc7124ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b9959c700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc7125970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 -- 172.21.15.66:0/95999705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 msgr2=0x55bfc7125970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc7125970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 -- 172.21.15.66:0/95999705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 msgr2=0x55bfc7124ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc7124ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 -- 172.21.15.66:0/95999705 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9410d170 con 0x55bfc71201b0 2023-07-21T22:33:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc7124ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:00.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b93fff700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 0x55bfc7125430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7b80099e30 tx=0x7f7b80009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1db71c534aa7c2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.460+0000 7f7b73fff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7b73fff700 1 -- 172.21.15.66:0/95999705 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800b82c0 con 0x55bfc71201b0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7b73fff700 1 -- 172.21.15.66:0/95999705 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7b800a22f0 con 0x55bfc71201b0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfc6fc64f0 con 0x55bfc71201b0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7b73fff700 1 -- 172.21.15.66:0/95999705 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b800a2070 con 0x55bfc71201b0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7bae8bab80 10 client.27071 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f7b941a4c60 con 0x55bfc71201b0 2023-07-21T22:33:00.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.461+0000 7f7b73fff700 1 -- 172.21.15.66:0/95999705 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bfc6fc64f0 con 0x55bfc71201b0 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7b73fff700 1 -- 172.21.15.66:0/95999705 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7b80095c90 con 0x55bfc71201b0 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7bae8bab80 10 client.27071 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7bae8bab80 10 client.27071 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7bae8bab80 20 client.27071 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc71350f0 0x55bfc71374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7bae8bab80 4 client.27071 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:00.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfc7065eb0 con 0x55bfc71350f0 2023-07-21T22:33:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.462+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc71350f0 0x55bfc71374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.463+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc71350f0 0x55bfc71374d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=773902dfa2f4f5a4 server_cookie=a27d077072f52a3c in_seq=0 out_seq=0 2023-07-21T22:33:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.463+0000 7f7b73fff700 10 client.27071 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.463+0000 7f7b73fff700 1 -- 172.21.15.66:0/95999705 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bfc7065eb0 con 0x55bfc71350f0 2023-07-21T22:33:00.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.463+0000 7f7b73fff700 10 client.27071 handle_command_reply: tid=0 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.463+0000 7f7bae8bab80 1 client.27071 shutdown 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 msgr2=0x55bfc7125430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 0x55bfc7125430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7b80099e30 tx=0x7f7b80009790 comp rx=0 tx=0).stop 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 shutdown_connections 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc7125970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc7124ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.528 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc71350f0 0x55bfc71374d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/95999705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc71201b0 0x55bfc7125430 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 >> 172.21.15.66:0/95999705 conn(0x55bfc7110ef0 msgr2=0x55bfc71163c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7b73fff700 0 client.27071 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 shutdown_connections 2023-07-21T22:33:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 1 -- 172.21.15.66:0/95999705 wait complete. 2023-07-21T22:33:00.529 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.464+0000 7f7bae8bab80 20 client.27071 trim_cache size 0 max 16384 2023-07-21T22:33:00.530 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 Processor -- start 2023-07-21T22:33:00.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 -- start start 2023-07-21T22:33:00.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 0x55bfc712c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc71257f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc711aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7128360 con 0x55bfc71201b0 2023-07-21T22:33:00.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc711c6f0 con 0x55bfc711b930 2023-07-21T22:33:00.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7128160 con 0x55bfc7120ac0 2023-07-21T22:33:00.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 0x55bfc712c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc71257f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 0x55bfc712c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42518/0 (socket says 172.21.15.66:42518) 2023-07-21T22:33:00.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.483+0000 7f7b98d9b700 1 -- 172.21.15.66:0/3635847252 learned_addr learned my addr 172.21.15.66:0/3635847252 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b9959c700 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc711aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b98d9b700 1 -- 172.21.15.66:0/3635847252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 msgr2=0x55bfc711aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc711aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b98d9b700 1 -- 172.21.15.66:0/3635847252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 msgr2=0x55bfc71257f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc71257f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b98d9b700 1 -- 172.21.15.66:0/3635847252 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc713c100 con 0x55bfc711b930 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b9959c700 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc711aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.484+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 0x55bfc712c5a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7b88006010 tx=0x7f7b88009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cc4717756da58cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.485+0000 7f7b7f7fe700 1 -- 172.21.15.66:0/3635847252 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8801d070 con 0x55bfc711b930 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.485+0000 7f7b7f7fe700 1 -- 172.21.15.66:0/3635847252 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8801e040 con 0x55bfc711b930 2023-07-21T22:33:00.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.485+0000 7f7b7f7fe700 1 -- 172.21.15.66:0/3635847252 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88004760 con 0x55bfc711b930 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.485+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3635847252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 msgr2=0x55bfc712c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.485+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 0x55bfc712c5a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7b88006010 tx=0x7f7b88009310 comp rx=0 tx=0).stop 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3635847252 shutdown_connections 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc711b930 0x55bfc712c5a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc7120ac0 0x55bfc711aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/3635847252 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc71201b0 0x55bfc71257f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3635847252 >> 172.21.15.66:0/3635847252 conn(0x55bfc7110ef0 msgr2=0x55bfc7119aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3635847252 shutdown_connections 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 -- 172.21.15.66:0/3635847252 wait complete. 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 -- start start 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.486+0000 7f7bae8bab80 1 Processor -- start 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc712a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc71201b0 0x55bfc712aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc712b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7190910 con 0x55bfc711b930 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7190a80 con 0x55bfc7120ac0 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc7190c40 con 0x55bfc71201b0 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7bae8bab80 10 client.0 _ll_register_callbacks cb 0x7f7b80005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc712a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7b9959c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc712b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc71201b0 0x55bfc712aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7b98d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc712a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51374/0 (socket says 172.21.15.66:51374) 2023-07-21T22:33:00.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.487+0000 7f7b98d9b700 1 -- 172.21.15.66:0/2169278275 learned_addr learned my addr 172.21.15.66:0/2169278275 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b93fff700 1 -- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 msgr2=0x55bfc712b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b93fff700 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc712b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b93fff700 1 -- 172.21.15.66:0/2169278275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 msgr2=0x55bfc712a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b93fff700 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc712a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b93fff700 1 -- 172.21.15.66:0/2169278275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8800be60 con 0x55bfc71201b0 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc712a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b93fff700 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc71201b0 0x55bfc712aad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7b800981f0 tx=0x7f7b80094ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d565d798cf0a97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b73fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b73fff700 1 -- 172.21.15.66:0/2169278275 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800b82c0 con 0x55bfc71201b0 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b800105a0 con 0x55bfc71201b0 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b73fff700 1 -- 172.21.15.66:0/2169278275 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7b800a22f0 con 0x55bfc71201b0 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7b73fff700 1 -- 172.21.15.66:0/2169278275 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b800adba0 con 0x55bfc71201b0 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7bae8bab80 10 client.27052 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:00.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.488+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bfc6e83c60 con 0x55bfc71201b0 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7b73fff700 1 -- 172.21.15.66:0/2169278275 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b800105a0 con 0x55bfc71201b0 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7b73fff700 1 -- 172.21.15.66:0/2169278275 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7b80094bd0 con 0x55bfc71201b0 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7bae8bab80 10 client.27052 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7bae8bab80 10 client.27052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7bae8bab80 20 client.27052 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc7191e90 0x55bfc7194270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7bae8bab80 4 client.27052 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:33:00.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfc7028050 con 0x55bfc7191e90 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.489+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc7191e90 0x55bfc7194270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7b98d9b700 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc7191e90 0x55bfc7194270 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=448f56f9aaf21d8b server_cookie=8bb031138a0d84f7 in_seq=0 out_seq=0 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7b73fff700 10 client.27052 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7b73fff700 1 -- 172.21.15.66:0/2169278275 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfc7028050 con 0x55bfc7191e90 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7b73fff700 10 client.27052 handle_command_reply: tid=0 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 client.27052 shutdown 2023-07-21T22:33:00.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc71201b0 msgr2=0x55bfc712aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc71201b0 0x55bfc712aad0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7b800981f0 tx=0x7f7b80094ab0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 shutdown_connections 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55bfc7191e90 0x55bfc7194270 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bfc7120ac0 0x55bfc712b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bfc711b930 0x55bfc712a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7b73fff700 0 client.27052 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 --2- 172.21.15.66:0/2169278275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bfc71201b0 0x55bfc712aad0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 >> 172.21.15.66:0/2169278275 conn(0x55bfc7110ef0 msgr2=0x55bfc711aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 shutdown_connections 2023-07-21T22:33:00.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 1 -- 172.21.15.66:0/2169278275 wait complete. 2023-07-21T22:33:00.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.490+0000 7f7bae8bab80 20 client.27052 trim_cache size 0 max 16384 2023-07-21T22:33:00.546 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:33:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7b840a1340 msgr2=0x7f7b840a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7b840a1340 0x7f7b840a37f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009c10 tx=0x7f7b8c004040 comp rx=0 tx=0).stop 2023-07-21T22:33:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 msgr2=0x7f7b94198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 0x7f7b94198370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000e9f0 tx=0x7f7b8000ed00 comp rx=0 tx=0).stop 2023-07-21T22:33:00.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 shutdown_connections 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b9410a070 0x7f7b9419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f7b840a1340 0x7f7b840a37f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7b94100ca0 0x7f7b94197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 --2- 172.21.15.66:0/3229022984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7b941015b0 0x7f7b94198370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 >> 172.21.15.66:0/3229022984 conn(0x7f7b940fa4b0 msgr2=0x7f7b9410c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 shutdown_connections 2023-07-21T22:33:00.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.491+0000 7f7b9afff700 1 -- 172.21.15.66:0/3229022984 wait complete. 2023-07-21T22:33:00.557 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:33:00.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 545160 5.33 MB/sec execute 507 sec latency 44.513 ms 2023-07-21T22:33:00.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 -- 172.21.15.66:0/3721370095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0101470 msgr2=0x7f19b0109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3721370095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0101470 0x7f19b0109b40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f19a40098f0 tx=0x7f19a4009c00 comp rx=0 tx=0).stop 2023-07-21T22:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 -- 172.21.15.66:0/3721370095 shutdown_connections 2023-07-21T22:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3721370095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0100b60 0x7f19b0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3721370095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b010a080 0x7f19b010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3721370095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0101470 0x7f19b0109b40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 -- 172.21.15.66:0/3721370095 >> 172.21.15.66:0/3721370095 conn(0x7f19b00fa4b0 msgr2=0x7f19b00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 -- 172.21.15.66:0/3721370095 shutdown_connections 2023-07-21T22:33:00.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.730+0000 7f19b57ff700 1 -- 172.21.15.66:0/3721370095 wait complete. 2023-07-21T22:33:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.731+0000 7f19b57ff700 1 Processor -- start 2023-07-21T22:33:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.731+0000 7f19b57ff700 1 -- start start 2023-07-21T22:33:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19b57ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 0x7f19b019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19b57ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0101470 0x7f19b019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19b57ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b010a080 0x7f19b0195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19b57ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0196290 con 0x7f19b0100b60 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19b57ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0196400 con 0x7f19b0101470 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19b57ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01965c0 con 0x7f19b010a080 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 0x7f19b019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 0x7f19b019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51400/0 (socket says 172.21.15.66:51400) 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19aeffd700 1 -- 172.21.15.66:0/3376214837 learned_addr learned my addr 172.21.15.66:0/3376214837 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0101470 0x7f19b019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.732+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b010a080 0x7f19b0195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19aeffd700 1 -- 172.21.15.66:0/3376214837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b010a080 msgr2=0x7f19b0195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19aeffd700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b010a080 0x7f19b0195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19aeffd700 1 -- 172.21.15.66:0/3376214837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0101470 msgr2=0x7f19b019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19aeffd700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0101470 0x7f19b019c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19aeffd700 1 -- 172.21.15.66:0/3376214837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00095f0 con 0x7f19b0100b60 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0101470 0x7f19b019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19aeffd700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 0x7f19b019bac0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ec10 tx=0x7f19a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f1997fff700 1 -- 172.21.15.66:0/3376214837 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000cc90 con 0x7f19b0100b60 2023-07-21T22:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f1997fff700 1 -- 172.21.15.66:0/3376214837 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a00044e0 con 0x7f19b0100b60 2023-07-21T22:33:00.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f1997fff700 1 -- 172.21.15.66:0/3376214837 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0010430 con 0x7f19b0100b60 2023-07-21T22:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a4009520 con 0x7f19b0100b60 2023-07-21T22:33:00.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.733+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b01a28d0 con 0x7f19b0100b60 2023-07-21T22:33:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.737+0000 7f1997fff700 1 -- 172.21.15.66:0/3376214837 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f19a0003fb0 con 0x7f19b0100b60 2023-07-21T22:33:00.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.738+0000 7f19c9123b80 1 Processor -- start 2023-07-21T22:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f1997fff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19980a1340 0x7f19980a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 -- start start 2023-07-21T22:33:00.792 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f1997fff700 1 -- 172.21.15.66:0/3376214837 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19a0014070 con 0x7f19b0100b60 2023-07-21T22:33:00.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 0x559a9f776580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19980a1340 0x7f19980a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f776ac0 0x559a9f771350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f771930 0x559a9f771da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f676cb0 con 0x559a9f771930 2023-07-21T22:33:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f6c9a40 con 0x559a9f776ac0 2023-07-21T22:33:00.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19c9123b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f6c2ed0 con 0x559a9f7761b0 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 0x559a9f776580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 0x559a9f776580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37688/0 (socket says 172.21.15.66:37688) 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19ae7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f776ac0 0x559a9f771350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.739+0000 7f19aeffd700 1 -- 172.21.15.66:0/986642079 learned_addr learned my addr 172.21.15.66:0/986642079 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19af7fe700 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f771930 0x559a9f771da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19aeffd700 1 -- 172.21.15.66:0/986642079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f776ac0 msgr2=0x559a9f771350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.795 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19aeffd700 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f776ac0 0x559a9f771350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19aeffd700 1 -- 172.21.15.66:0/986642079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f771930 msgr2=0x559a9f771da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19aeffd700 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f771930 0x559a9f771da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19aeffd700 1 -- 172.21.15.66:0/986642079 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9f4f0de0 con 0x559a9f7761b0 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19980a1340 0x7f19980a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f19b019c7c0 tx=0x7f19a8009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19aeffd700 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 0x559a9f776580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f19a00a21a0 tx=0x7f19a0099dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e333402ce9eaf2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19957fa700 1 -- 172.21.15.66:0/986642079 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a009e2b0 con 0x559a9f7761b0 2023-07-21T22:33:00.796 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19957fa700 1 -- 172.21.15.66:0/986642079 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a0094e90 con 0x559a9f7761b0 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.740+0000 7f19957fa700 1 -- 172.21.15.66:0/986642079 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a00cd190 con 0x559a9f7761b0 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 -- 172.21.15.66:0/986642079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 msgr2=0x559a9f776580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 0x559a9f776580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f19a00a21a0 tx=0x7f19a0099dc0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 -- 172.21.15.66:0/986642079 shutdown_connections 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f771930 0x559a9f771da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f7761b0 0x559a9f776580 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 --2- 172.21.15.66:0/986642079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f776ac0 0x559a9f771350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.797 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 -- 172.21.15.66:0/986642079 >> 172.21.15.66:0/986642079 conn(0x559a9f766ef0 msgr2=0x559a9f76c830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 -- 172.21.15.66:0/986642079 shutdown_connections 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.741+0000 7f19c9123b80 1 -- 172.21.15.66:0/986642079 wait complete. 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.742+0000 7f19c9123b80 1 -- start start 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.742+0000 7f19c9123b80 1 Processor -- start 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 0x559a9f77dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f7761b0 0x559a9f77aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f776ac0 0x559a9f77b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.798 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f662ac0 con 0x559a9f7761b0 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f6627c0 con 0x559a9f771930 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f546920 con 0x559a9f776ac0 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19c9123b80 10 client.0 _ll_register_callbacks cb 0x559a9f660760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 0x559a9f77dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 0x559a9f77dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42560/0 (socket says 172.21.15.66:42560) 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19aeffd700 1 -- 172.21.15.66:0/3087907989 learned_addr learned my addr 172.21.15.66:0/3087907989 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f776ac0 0x559a9f77b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.799 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.743+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f7761b0 0x559a9f77aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19af7fe700 1 -- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 msgr2=0x559a9f77dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 0x559a9f77dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19af7fe700 1 -- 172.21.15.66:0/3087907989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f7761b0 msgr2=0x559a9f77aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f7761b0 0x559a9f77aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19af7fe700 1 -- 172.21.15.66:0/3087907989 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b01a28d0 con 0x559a9f776ac0 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19aeffd700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 0x559a9f77dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.800 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19af7fe700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f776ac0 0x559a9f77b600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f19a8019e30 tx=0x7f19a8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f1564d6ceca82de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f1992ff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f1992ff5700 1 -- 172.21.15.66:0/3087907989 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800e040 con 0x559a9f776ac0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a9f61c4f0 con 0x559a9f776ac0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.744+0000 7f1992ff5700 1 -- 172.21.15.66:0/3087907989 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19a8009020 con 0x559a9f776ac0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f1992ff5700 1 -- 172.21.15.66:0/3087907989 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a8019e60 con 0x559a9f776ac0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 10 client.27064 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559a9f4d9c60 con 0x559a9f776ac0 2023-07-21T22:33:00.801 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f1992ff5700 1 -- 172.21.15.66:0/3087907989 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a9f61c4f0 con 0x559a9f776ac0 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f1992ff5700 1 -- 172.21.15.66:0/3087907989 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f19a8022400 con 0x559a9f776ac0 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 10 client.27064 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 20 client.27064 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f78b4e0 0x559a9f78d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 4 client.27064 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:00.802 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a9f6bbeb0 con 0x559a9f78b4e0 2023-07-21T22:33:00.803 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.745+0000 7f19aeffd700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f78b4e0 0x559a9f78d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.746+0000 7f1992ff5700 10 client.27064 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.746+0000 7f19aeffd700 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f78b4e0 0x559a9f78d8f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e30e44c3f33d4d server_cookie=614b7c13c051842e in_seq=0 out_seq=0 2023-07-21T22:33:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f1992ff5700 1 -- 172.21.15.66:0/3087907989 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a9f6bbeb0 con 0x559a9f78b4e0 2023-07-21T22:33:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f1992ff5700 10 client.27064 handle_command_reply: tid=0 2023-07-21T22:33:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 client.27064 shutdown 2023-07-21T22:33:00.804 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f776ac0 msgr2=0x559a9f77b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f776ac0 0x559a9f77b600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f19a8019e30 tx=0x7f19a8004b40 comp rx=0 tx=0).stop 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 shutdown_connections 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f771930 0x559a9f77dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f78b4e0 0x559a9f78d8f0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f776ac0 0x559a9f77b600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 --2- 172.21.15.66:0/3087907989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f7761b0 0x559a9f77aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f1992ff5700 0 client.27064 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:00.805 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 >> 172.21.15.66:0/3087907989 conn(0x559a9f766ef0 msgr2=0x559a9f76c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 shutdown_connections 2023-07-21T22:33:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.747+0000 7f19c9123b80 1 -- 172.21.15.66:0/3087907989 wait complete. 2023-07-21T22:33:00.806 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.748+0000 7f19c9123b80 20 client.27064 trim_cache size 0 max 16384 2023-07-21T22:33:00.808 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.761+0000 7f19c9123b80 1 Processor -- start 2023-07-21T22:33:00.809 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 -- start start 2023-07-21T22:33:00.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f782570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 0x559a9f76af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.810 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f77d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f788ca0 con 0x559a9f776ac0 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f775e30 con 0x559a9f7761b0 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19c9123b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f764060 con 0x559a9f771930 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19ae7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 0x559a9f76af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19af7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f77d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19ae7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 0x559a9f76af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:42564/0 (socket says 172.21.15.66:42564) 2023-07-21T22:33:00.811 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19ae7fc700 1 -- 172.21.15.66:0/1342829203 learned_addr learned my addr 172.21.15.66:0/1342829203 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.762+0000 7f19aeffd700 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f782570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19ae7fc700 1 -- 172.21.15.66:0/1342829203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 msgr2=0x559a9f782570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f782570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19ae7fc700 1 -- 172.21.15.66:0/1342829203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 msgr2=0x559a9f77d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f77d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.812 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19ae7fc700 1 -- 172.21.15.66:0/1342829203 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9f517250 con 0x559a9f7761b0 2023-07-21T22:33:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19af7fe700 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f77d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 0x559a9f76af60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f19a40098c0 tx=0x7f19a4019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e81d4840147362ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f1995ffb700 1 -- 172.21.15.66:0/1342829203 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4023070 con 0x559a9f7761b0 2023-07-21T22:33:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f1995ffb700 1 -- 172.21.15.66:0/1342829203 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a4024040 con 0x559a9f7761b0 2023-07-21T22:33:00.813 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.763+0000 7f1995ffb700 1 -- 172.21.15.66:0/1342829203 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40055e0 con 0x559a9f7761b0 2023-07-21T22:33:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 -- 172.21.15.66:0/1342829203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 msgr2=0x559a9f76af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 0x559a9f76af60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f19a40098c0 tx=0x7f19a4019040 comp rx=0 tx=0).stop 2023-07-21T22:33:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 -- 172.21.15.66:0/1342829203 shutdown_connections 2023-07-21T22:33:00.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f77d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f782570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 --2- 172.21.15.66:0/1342829203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f7761b0 0x559a9f76af60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 -- 172.21.15.66:0/1342829203 >> 172.21.15.66:0/1342829203 conn(0x559a9f766ef0 msgr2=0x559a9f7878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 -- 172.21.15.66:0/1342829203 shutdown_connections 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 -- 172.21.15.66:0/1342829203 wait complete. 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.764+0000 7f19c9123b80 1 -- start start 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 Processor -- start 2023-07-21T22:33:00.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f77ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f7797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f779ce0 0x559a9f77a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f7e7b70 con 0x559a9f776ac0 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f7e7cb0 con 0x559a9f779ce0 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f7e7df0 con 0x559a9f771930 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19c9123b80 10 client.0 _ll_register_callbacks cb 0x7f19a8019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:00.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19ae7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f7797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.765+0000 7f19af7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f779ce0 0x559a9f77a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19ae7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f7797a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:51440/0 (socket says 172.21.15.66:51440) 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19ae7fc700 1 -- 172.21.15.66:0/2832313431 learned_addr learned my addr 172.21.15.66:0/2832313431 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19aeffd700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f77ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19af7fe700 1 -- 172.21.15.66:0/2832313431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 msgr2=0x559a9f77ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19af7fe700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f77ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19af7fe700 1 -- 172.21.15.66:0/2832313431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 msgr2=0x559a9f7797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:00.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19af7fe700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f7797a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19af7fe700 1 -- 172.21.15.66:0/2832313431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a9f4d9c60 con 0x559a9f779ce0 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19ae7fc700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f7797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19aeffd700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f77ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19af7fe700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f779ce0 0x559a9f77a150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x559a9f78b2b0 tx=0x7f19a800fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba8edf62899ec686 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f1992ff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f1992ff5700 1 -- 172.21.15.66:0/2832313431 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a800e040 con 0x559a9f779ce0 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f1992ff5700 1 -- 172.21.15.66:0/2832313431 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19a8022a80 con 0x559a9f779ce0 2023-07-21T22:33:00.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.766+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a0094e90 con 0x559a9f779ce0 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f1992ff5700 1 -- 172.21.15.66:0/2832313431 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a8009040 con 0x559a9f779ce0 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f19c9123b80 10 client.27095 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f19b010d1d0 con 0x559a9f779ce0 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f1992ff5700 1 -- 172.21.15.66:0/2832313431 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a0094e90 con 0x559a9f779ce0 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f1992ff5700 1 -- 172.21.15.66:0/2832313431 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f19a803b020 con 0x559a9f779ce0 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f19c9123b80 10 client.27095 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:00.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f19c9123b80 10 client.27095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:00.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f19c9123b80 20 client.27095 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:00.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.767+0000 7f19c9123b80 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f7e92d0 0x559a9f7eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:00.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f19c9123b80 4 client.27095 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:33:00.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a9f67e050 con 0x559a9f7e92d0 2023-07-21T22:33:00.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f19aeffd700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f7e92d0 0x559a9f7eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:00.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f19aeffd700 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f7e92d0 0x559a9f7eb6e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f70bcd1a75002 server_cookie=6d993101925da799 in_seq=0 out_seq=0 2023-07-21T22:33:00.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f1992ff5700 10 client.27095 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:00.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f1992ff5700 1 -- 172.21.15.66:0/2832313431 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a9f67e050 con 0x559a9f7e92d0 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f1992ff5700 10 client.27095 handle_command_reply: tid=0 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.768+0000 7f19c9123b80 1 client.27095 shutdown 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f779ce0 msgr2=0x559a9f77a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f779ce0 0x559a9f77a150 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x559a9f78b2b0 tx=0x7f19a800fe70 comp rx=0 tx=0).stop 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 shutdown_connections 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f779ce0 0x559a9f77a150 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x559a9f7e92d0 0x559a9f7eb6e0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f771930 0x559a9f77ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 --2- 172.21.15.66:0/2832313431 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f776ac0 0x559a9f7797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 >> 172.21.15.66:0/2832313431 conn(0x559a9f766ef0 msgr2=0x559a9f76c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f1992ff5700 0 client.27095 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 shutdown_connections 2023-07-21T22:33:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 1 -- 172.21.15.66:0/2832313431 wait complete. 2023-07-21T22:33:00.824 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19c9123b80 20 client.27095 trim_cache size 0 max 16384 2023-07-21T22:33:00.825 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:33:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19980a1340 msgr2=0x7f19980a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19980a1340 0x7f19980a37f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f19b019c7c0 tx=0x7f19a8009920 comp rx=0 tx=0).stop 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 msgr2=0x7f19b019bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.769+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 0x7f19b019bac0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f19a000ec10 tx=0x7f19a000ef20 comp rx=0 tx=0).stop 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 shutdown_connections 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19980a1340 0x7f19980a37f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19b010a080 0x7f19b0195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f19b0100b60 0x7f19b019bac0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 --2- 172.21.15.66:0/3376214837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f19b0101470 0x7f19b019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 >> 172.21.15.66:0/3376214837 conn(0x7f19b00fa4b0 msgr2=0x7f19b00fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 shutdown_connections 2023-07-21T22:33:00.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:00.770+0000 7f19b57ff700 1 -- 172.21.15.66:0/3376214837 wait complete. 2023-07-21T22:33:01.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 546755 5.34 MB/sec execute 508 sec latency 50.680 ms 2023-07-21T22:33:01.837 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:33:01.838 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:33:02.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 -- 172.21.15.66:0/2951627136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 msgr2=0x7fa024105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/2951627136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 0x7fa024105eb0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa014009990 tx=0x7fa014009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 -- 172.21.15.66:0/2951627136 shutdown_connections 2023-07-21T22:33:02.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/2951627136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa024100480 0x7fa02410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/2951627136 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 0x7fa024105eb0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/2951627136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa0240fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 -- 172.21.15.66:0/2951627136 >> 172.21.15.66:0/2951627136 conn(0x7fa0240fb5f0 msgr2=0x7fa0240fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 -- 172.21.15.66:0/2951627136 shutdown_connections 2023-07-21T22:33:02.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.059+0000 7fa02a1ff700 1 -- 172.21.15.66:0/2951627136 wait complete. 2023-07-21T22:33:02.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.061+0000 7fa02a1ff700 1 Processor -- start 2023-07-21T22:33:02.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.061+0000 7fa02a1ff700 1 -- start start 2023-07-21T22:33:02.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.061+0000 7fa02a1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa024072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.061+0000 7fa02a1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa024100480 0x7fa024072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.061+0000 7fa02a1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 0x7fa024073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa02a1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241a85e0 con 0x7fa024105ae0 2023-07-21T22:33:02.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa02a1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241a8720 con 0x7fa0240ffad0 2023-07-21T22:33:02.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa02a1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0241a8860 con 0x7fa024100480 2023-07-21T22:33:02.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa024072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa024072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:37142/0 (socket says 172.21.15.66:37142) 2023-07-21T22:33:02.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 -- 172.21.15.66:0/159905648 learned_addr learned my addr 172.21.15.66:0/159905648 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa022ffd700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa024100480 0x7fa024072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa023fff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 0x7fa024073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 -- 172.21.15.66:0/159905648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa024100480 msgr2=0x7fa024072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa024100480 0x7fa024072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 -- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 msgr2=0x7fa024073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 0x7fa024073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.062+0000 7fa0237fe700 1 -- 172.21.15.66:0/159905648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140095f0 con 0x7fa0240ffad0 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.063+0000 7fa0237fe700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa024072810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa014005910 tx=0x7fa014004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.063+0000 7fa020ff9700 1 -- 172.21.15.66:0/159905648 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01401d070 con 0x7fa0240ffad0 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.063+0000 7fa020ff9700 1 -- 172.21.15.66:0/159905648 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa014022470 con 0x7fa0240ffad0 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.063+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0241a8a90 con 0x7fa0240ffad0 2023-07-21T22:33:02.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.063+0000 7fa020ff9700 1 -- 172.21.15.66:0/159905648 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014017650 con 0x7fa0240ffad0 2023-07-21T22:33:02.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.063+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0241a8f10 con 0x7fa0240ffad0 2023-07-21T22:33:02.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.068+0000 7fa020ff9700 1 -- 172.21.15.66:0/159905648 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa01400fb10 con 0x7fa0240ffad0 2023-07-21T22:33:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.070+0000 7fa03dbb7b80 1 Processor -- start 2023-07-21T22:33:02.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 -- start start 2023-07-21T22:33:02.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa020ff9700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa00c0a1340 0x7fa00c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 0x55c5466f8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa020ff9700 1 -- 172.21.15.66:0/159905648 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa0140ce0e0 con 0x7fa0240ffad0 2023-07-21T22:33:02.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5465f8d30 con 0x55c5466f39b0 2023-07-21T22:33:02.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54664bac0 con 0x55c5466f8b40 2023-07-21T22:33:02.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c546644f50 con 0x55c5466f8230 2023-07-21T22:33:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa023fff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa00c0a1340 0x7fa00c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 0x55c5466f8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 0x55c5466f8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58364/0 (socket says 172.21.15.66:58364) 2023-07-21T22:33:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.071+0000 7fa0237fe700 1 -- 172.21.15.66:0/1941724520 learned_addr learned my addr 172.21.15.66:0/1941724520 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa022ffd700 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa023fff700 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa0237fe700 1 -- 172.21.15.66:0/1941724520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 msgr2=0x55c5466f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa0237fe700 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa0237fe700 1 -- 172.21.15.66:0/1941724520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 msgr2=0x55c5466f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa0237fe700 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.129 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa0237fe700 1 -- 172.21.15.66:0/1941724520 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c546472e60 con 0x55c5466f8230 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa0237fe700 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 0x55c5466f8600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa014097910 tx=0x7fa014097260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f33a8f3b46b219 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.072+0000 7fa009ffb700 1 -- 172.21.15.66:0/1941724520 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140a0050 con 0x55c5466f8230 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.073+0000 7fa009ffb700 1 -- 172.21.15.66:0/1941724520 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01409a800 con 0x55c5466f8230 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.073+0000 7fa009ffb700 1 -- 172.21.15.66:0/1941724520 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01409b100 con 0x55c5466f8230 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.073+0000 7fa023fff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa00c0a1340 0x7fa00c0a37f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009ec0 tx=0x7fa01c009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.073+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/1941724520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 msgr2=0x55c5466f8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.073+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 0x55c5466f8600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa014097910 tx=0x7fa014097260 comp rx=0 tx=0).stop 2023-07-21T22:33:02.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/1941724520 shutdown_connections 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8230 0x55c5466f8600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/1941724520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/1941724520 >> 172.21.15.66:0/1941724520 conn(0x55c5466e8f70 msgr2=0x55c5466ee8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/1941724520 shutdown_connections 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.074+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/1941724520 wait complete. 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 -- start start 2023-07-21T22:33:02.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 Processor -- start 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8230 0x55c5466fcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466fd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5465e4b40 con 0x55c5466f39b0 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5465e4840 con 0x55c5466f8230 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5464c89a0 con 0x55c5466f8b40 2023-07-21T22:33:02.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.075+0000 7fa03dbb7b80 10 client.0 _ll_register_callbacks cb 0x55c5465e27e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8230 0x55c5466fcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa023fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466fd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42724/0 (socket says 172.21.15.66:42724) 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa0237fe700 1 -- 172.21.15.66:0/80284960 learned_addr learned my addr 172.21.15.66:0/80284960 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 -- 172.21.15.66:0/80284960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 msgr2=0x55c5466fd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466fd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 -- 172.21.15.66:0/80284960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 msgr2=0x55c5466ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 -- 172.21.15.66:0/80284960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0241a8f10 con 0x55c5466f8230 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa022ffd700 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8230 0x55c5466fcee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55c5466efef0 tx=0x7fa018012a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bf22de4417ef8ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.076+0000 7fa002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa002ffd700 1 -- 172.21.15.66:0/80284960 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018017070 con 0x55c5466f8230 2023-07-21T22:33:02.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c54659e570 con 0x55c5466f8230 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa002ffd700 1 -- 172.21.15.66:0/80284960 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa01800f480 con 0x55c5466f8230 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa002ffd700 1 -- 172.21.15.66:0/80284960 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa018021650 con 0x55c5466f8230 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa03dbb7b80 10 client.27107 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c54645bce0 con 0x55c5466f8230 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.077+0000 7fa002ffd700 1 -- 172.21.15.66:0/80284960 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c54659e570 con 0x55c5466f8230 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa002ffd700 1 -- 172.21.15.66:0/80284960 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa018010430 con 0x55c5466f8230 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa03dbb7b80 10 client.27107 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:02.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa03dbb7b80 10 client.27107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa03dbb7b80 20 client.27107 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c54670d260 0x55c54670f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa03dbb7b80 4 client.27107 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c54663df30 con 0x55c54670d260 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.078+0000 7fa0237fe700 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c54670d260 0x55c54670f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.079+0000 7fa0237fe700 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c54670d260 0x55c54670f670 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a2a6cea46080c9 server_cookie=f07566241537bb31 in_seq=0 out_seq=0 2023-07-21T22:33:02.136 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.079+0000 7fa002ffd700 10 client.27107 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.079+0000 7fa002ffd700 1 -- 172.21.15.66:0/80284960 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c54663df30 con 0x55c54670d260 2023-07-21T22:33:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.079+0000 7fa002ffd700 10 client.27107 handle_command_reply: tid=0 2023-07-21T22:33:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 client.27107 shutdown 2023-07-21T22:33:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8230 msgr2=0x55c5466fcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.137 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8230 0x55c5466fcee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55c5466efef0 tx=0x7fa018012a20 comp rx=0 tx=0).stop 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 shutdown_connections 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c54670d260 0x55c54670f670 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466fd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f39b0 0x55c5466ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/80284960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8230 0x55c5466fcee0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa002ffd700 0 client.27107 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.138 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 >> 172.21.15.66:0/80284960 conn(0x55c5466e8f70 msgr2=0x55c5466ee4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 shutdown_connections 2023-07-21T22:33:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.080+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/80284960 wait complete. 2023-07-21T22:33:02.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.081+0000 7fa03dbb7b80 20 client.27107 trim_cache size 0 max 16384 2023-07-21T22:33:02.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 Processor -- start 2023-07-21T22:33:02.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 -- start start 2023-07-21T22:33:02.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 0x55c5466f3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c546704400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466f7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c54659e570 con 0x55c5466f8230 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5467001a0 con 0x55c5466f39b0 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.104+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5466e8a30 con 0x55c5466f8b40 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa022ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c546704400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 0x55c5466f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa022ffd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c546704400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42732/0 (socket says 172.21.15.66:42732) 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa022ffd700 1 -- 172.21.15.66:0/3295492819 learned_addr learned my addr 172.21.15.66:0/3295492819 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 0x55c5466f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:37166/0 (socket says 172.21.15.66:37166) 2023-07-21T22:33:02.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa023fff700 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 -- 172.21.15.66:0/3295492819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 msgr2=0x55c5466f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 -- 172.21.15.66:0/3295492819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 msgr2=0x55c546704400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c546704400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.105+0000 7fa0237fe700 1 -- 172.21.15.66:0/3295492819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5466ed2b0 con 0x55c5466f39b0 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.106+0000 7fa0237fe700 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 0x55c5466f3330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa01409aba0 tx=0x7fa014009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a998f9054da84599 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.106+0000 7fa00a7fc700 1 -- 172.21.15.66:0/3295492819 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140a0040 con 0x55c5466f39b0 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.106+0000 7fa00a7fc700 1 -- 172.21.15.66:0/3295492819 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa014098fe0 con 0x55c5466f39b0 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.106+0000 7fa00a7fc700 1 -- 172.21.15.66:0/3295492819 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01409a5f0 con 0x55c5466f39b0 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/3295492819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 msgr2=0x55c5466f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 0x55c5466f3330 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa01409aba0 tx=0x7fa014009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/3295492819 shutdown_connections 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f39b0 0x55c5466f3330 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.156 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f8b40 0x55c5466f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/3295492819 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c546704400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/3295492819 >> 172.21.15.66:0/3295492819 conn(0x55c5466e8f70 msgr2=0x55c5466f6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/3295492819 shutdown_connections 2023-07-21T22:33:02.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.107+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/3295492819 wait complete. 2023-07-21T22:33:02.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.108+0000 7fa03dbb7b80 1 -- start start 2023-07-21T22:33:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.108+0000 7fa03dbb7b80 1 Processor -- start 2023-07-21T22:33:02.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.108+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 0x55c5466fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c5466fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa03dbb7b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466fc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5467612a0 con 0x55c5466f8230 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c546761410 con 0x55c5466f8b40 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa03dbb7b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5467615d0 con 0x55c5466f39b0 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa03dbb7b80 10 client.0 _ll_register_callbacks cb 0x7fa018009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:02.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 0x55c5466fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 0x55c5466fbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58382/0 (socket says 172.21.15.66:58382) 2023-07-21T22:33:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 -- 172.21.15.66:0/434281673 learned_addr learned my addr 172.21.15.66:0/434281673 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa023fff700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466fc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa022ffd700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c5466fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 -- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 msgr2=0x55c5466fc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466fc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 -- 172.21.15.66:0/434281673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 msgr2=0x55c5466fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c5466fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa0237fe700 1 -- 172.21.15.66:0/434281673 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c546472e60 con 0x55c5466f39b0 2023-07-21T22:33:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.109+0000 7fa022ffd700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c5466fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa0237fe700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 0x55c5466fbe60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0140a2d30 tx=0x7fa0140a3260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baecc489b1063ad2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa002ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:33:02.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa002ffd700 1 -- 172.21.15.66:0/434281673 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0140a0070 con 0x55c5466f39b0 2023-07-21T22:33:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa01409a800 con 0x55c5466f39b0 2023-07-21T22:33:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa002ffd700 1 -- 172.21.15.66:0/434281673 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa014099300 con 0x55c5466f39b0 2023-07-21T22:33:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa002ffd700 1 -- 172.21.15.66:0/434281673 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa01409b100 con 0x55c5466f39b0 2023-07-21T22:33:02.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa03dbb7b80 10 client.27088 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.110+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c5464992d0 con 0x55c5466f39b0 2023-07-21T22:33:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa002ffd700 1 -- 172.21.15.66:0/434281673 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa01409a800 con 0x55c5466f39b0 2023-07-21T22:33:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa002ffd700 1 -- 172.21.15.66:0/434281673 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa0140a2dd0 con 0x55c5466f39b0 2023-07-21T22:33:02.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa03dbb7b80 10 client.27088 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa03dbb7b80 10 client.27088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa03dbb7b80 20 client.27088 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c546762930 0x55c546764d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa03dbb7b80 4 client.27088 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c5466000d0 con 0x55c546762930 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.111+0000 7fa022ffd700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c546762930 0x55c546764d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.112+0000 7fa022ffd700 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c546762930 0x55c546764d40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd656397414871d2 server_cookie=41831459034eb686 in_seq=0 out_seq=0 2023-07-21T22:33:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.112+0000 7fa002ffd700 10 client.27088 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa002ffd700 1 -- 172.21.15.66:0/434281673 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c5466000d0 con 0x55c546762930 2023-07-21T22:33:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa002ffd700 10 client.27088 handle_command_reply: tid=0 2023-07-21T22:33:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 client.27088 shutdown 2023-07-21T22:33:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 msgr2=0x55c5466fbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 0x55c5466fbe60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0140a2d30 tx=0x7fa0140a3260 comp rx=0 tx=0).stop 2023-07-21T22:33:02.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 shutdown_connections 2023-07-21T22:33:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c5466f8b40 0x55c5466fc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c546762930 0x55c546764d40 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5466f39b0 0x55c5466fbe60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa002ffd700 0 client.27088 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 --2- 172.21.15.66:0/434281673 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c5466f8230 0x55c5466fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 >> 172.21.15.66:0/434281673 conn(0x55c5466e8f70 msgr2=0x55c5466f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 shutdown_connections 2023-07-21T22:33:02.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 1 -- 172.21.15.66:0/434281673 wait complete. 2023-07-21T22:33:02.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.113+0000 7fa03dbb7b80 20 client.27088 trim_cache size 0 max 16384 2023-07-21T22:33:02.170 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:33:02.171 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:33:02.171 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "0c11da94-1d47-47bf-8175-40c52d81d652", 2023-07-21T22:33:02.171 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:33:02.171 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa00c0a1340 msgr2=0x7fa00c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa00c0a1340 0x7fa00c0a37f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009ec0 tx=0x7fa01c009920 comp rx=0 tx=0).stop 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 msgr2=0x7fa024072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa024072810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa014005910 tx=0x7fa014004710 comp rx=0 tx=0).stop 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 shutdown_connections 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0240ffad0 0x7fa024072810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa00c0a1340 0x7fa00c0a37f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa024105ae0 0x7fa024073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 --2- 172.21.15.66:0/159905648 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa024100480 0x7fa024072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 >> 172.21.15.66:0/159905648 conn(0x7fa0240fb5f0 msgr2=0x7fa0240fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 shutdown_connections 2023-07-21T22:33:02.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.114+0000 7fa02a1ff700 1 -- 172.21.15.66:0/159905648 wait complete. 2023-07-21T22:33:02.183 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:02 smithi066 ceph-mon[126379]: pgmap v359: 129 pgs: 129 active+clean; 528 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2023-07-21T22:33:02.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:33:02.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:02 smithi104 ceph-mon[129021]: pgmap v359: 129 pgs: 129 active+clean; 528 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2023-07-21T22:33:02.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 -- 172.21.15.66:0/872036411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74580ff9d0 msgr2=0x7f74580ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 --2- 172.21.15.66:0/872036411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74580ff9d0 0x7f74580ffe40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f744c009990 tx=0x7f744c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 -- 172.21.15.66:0/872036411 shutdown_connections 2023-07-21T22:33:02.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 --2- 172.21.15.66:0/872036411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7458100380 0x7f745810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 --2- 172.21.15.66:0/872036411 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74581059e0 0x7f7458105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 --2- 172.21.15.66:0/872036411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74580ff9d0 0x7f74580ffe40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 -- 172.21.15.66:0/872036411 >> 172.21.15.66:0/872036411 conn(0x7f74580fb4b0 msgr2=0x7f74580fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 -- 172.21.15.66:0/872036411 shutdown_connections 2023-07-21T22:33:02.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.388+0000 7f745d59d700 1 -- 172.21.15.66:0/872036411 wait complete. 2023-07-21T22:33:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.389+0000 7f745d59d700 1 Processor -- start 2023-07-21T22:33:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.389+0000 7f745d59d700 1 -- start start 2023-07-21T22:33:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745d59d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74580ff9d0 0x7f7458195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745d59d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7458100380 0x7f7458195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745d59d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 0x7f745819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745d59d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745819a910 con 0x7f74580ff9d0 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745d59d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745819aa80 con 0x7f74581059e0 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745d59d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f745819ac40 con 0x7f7458100380 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745cd9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 0x7f745819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745cd9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 0x7f745819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:37178/0 (socket says 172.21.15.66:37178) 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f745cd9c700 1 -- 172.21.15.66:0/3743927709 learned_addr learned my addr 172.21.15.66:0/3743927709 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.390+0000 7f7457fff700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74580ff9d0 0x7f7458195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f74577fe700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7458100380 0x7f7458195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745cd9c700 1 -- 172.21.15.66:0/3743927709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7458100380 msgr2=0x7f7458195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745cd9c700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7458100380 0x7f7458195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745cd9c700 1 -- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74580ff9d0 msgr2=0x7f7458195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745cd9c700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74580ff9d0 0x7f7458195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745cd9c700 1 -- 172.21.15.66:0/3743927709 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0095f0 con 0x7f74581059e0 2023-07-21T22:33:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745cd9c700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 0x7f745819a340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f745000ef30 tx=0x7f745000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f74557fa700 1 -- 172.21.15.66:0/3743927709 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450018070 con 0x7f74581059e0 2023-07-21T22:33:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f74557fa700 1 -- 172.21.15.66:0/3743927709 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f745000f040 con 0x7f74581059e0 2023-07-21T22:33:02.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f74557fa700 1 -- 172.21.15.66:0/3743927709 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450003dc0 con 0x7f74581059e0 2023-07-21T22:33:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.391+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745819aec0 con 0x7f74581059e0 2023-07-21T22:33:02.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.392+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7458072920 con 0x7f74581059e0 2023-07-21T22:33:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.397+0000 7f74557fa700 1 -- 172.21.15.66:0/3743927709 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f745000caa0 con 0x7f74581059e0 2023-07-21T22:33:02.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 Processor -- start 2023-07-21T22:33:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 -- start start 2023-07-21T22:33:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868c490 0x55770868c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868cda0 0x557708697700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x55770869a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557708592cb0 con 0x557708697c40 2023-07-21T22:33:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577085e5a40 con 0x55770868c490 2023-07-21T22:33:02.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f7472208b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577085deed0 con 0x55770868cda0 2023-07-21T22:33:02.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.398+0000 7f74577fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868c490 0x55770868c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f74577fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868c490 0x55770868c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:37182/0 (socket says 172.21.15.66:37182) 2023-07-21T22:33:02.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868cda0 0x557708697700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f74577fe700 1 -- 172.21.15.66:0/4219622005 learned_addr learned my addr 172.21.15.66:0/4219622005 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f74577fe700 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x55770869a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f74557fa700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74400a14c0 0x7f74400a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 -- 172.21.15.66:0/4219622005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868c490 msgr2=0x55770868c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868c490 0x55770868c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 -- 172.21.15.66:0/4219622005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 msgr2=0x55770869a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x55770869a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 -- 172.21.15.66:0/4219622005 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55770840cde0 con 0x55770868cda0 2023-07-21T22:33:02.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f74557fa700 1 -- 172.21.15.66:0/3743927709 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f74500ccc40 con 0x7f74581059e0 2023-07-21T22:33:02.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f745cd9c700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74400a14c0 0x7f74400a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.399+0000 7f7457fff700 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868cda0 0x557708697700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7448009bb0 tx=0x7f7448009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89bac54fe2f751f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.400+0000 7f743e7fc700 1 -- 172.21.15.66:0/4219622005 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448010b80 con 0x55770868cda0 2023-07-21T22:33:02.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.400+0000 7f743e7fc700 1 -- 172.21.15.66:0/4219622005 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74480044f0 con 0x55770868cda0 2023-07-21T22:33:02.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.400+0000 7f743e7fc700 1 -- 172.21.15.66:0/4219622005 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448003d50 con 0x55770868cda0 2023-07-21T22:33:02.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.400+0000 7f7472208b80 1 -- 172.21.15.66:0/4219622005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868cda0 msgr2=0x557708697700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.400+0000 7f745cd9c700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74400a14c0 0x7f74400a3970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f74500ca0a0 tx=0x7f7450009ed0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.400+0000 7f7472208b80 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868cda0 0x557708697700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7448009bb0 tx=0x7f7448009ec0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 -- 172.21.15.66:0/4219622005 shutdown_connections 2023-07-21T22:33:02.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868c490 0x55770868c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x55770869a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 --2- 172.21.15.66:0/4219622005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868cda0 0x557708697700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 -- 172.21.15.66:0/4219622005 >> 172.21.15.66:0/4219622005 conn(0x557708682ef0 msgr2=0x557708688860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 -- 172.21.15.66:0/4219622005 shutdown_connections 2023-07-21T22:33:02.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 -- 172.21.15.66:0/4219622005 wait complete. 2023-07-21T22:33:02.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 -- start start 2023-07-21T22:33:02.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.401+0000 7f7472208b80 1 Processor -- start 2023-07-21T22:33:02.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x557708697670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 0x557708694880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x557708694fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770857eac0 con 0x557708697c40 2023-07-21T22:33:02.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770857e7c0 con 0x55770868cda0 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557708462920 con 0x55770868c490 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7472208b80 10 client.0 _ll_register_callbacks cb 0x55770857c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7457fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 0x557708694880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7457fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 0x557708694880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:37194/0 (socket says 172.21.15.66:37194) 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f7457fff700 1 -- 172.21.15.66:0/730230023 learned_addr learned my addr 172.21.15.66:0/730230023 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f74577fe700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x557708697670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.402+0000 7f74577fe700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x557708694fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7457fff700 1 -- 172.21.15.66:0/730230023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 msgr2=0x557708697670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7457fff700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x557708697670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7457fff700 1 -- 172.21.15.66:0/730230023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 msgr2=0x557708694fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7457fff700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x557708694fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7457fff700 1 -- 172.21.15.66:0/730230023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7448009810 con 0x55770868cda0 2023-07-21T22:33:02.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7457fff700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 0x557708694880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74480097b0 tx=0x7f7448010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a67127ce0416a377 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f74377fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f74377fe700 1 -- 172.21.15.66:0/730230023 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448021070 con 0x55770868cda0 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f74377fe700 1 -- 172.21.15.66:0/730230023 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7448017030 con 0x55770868cda0 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.403+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577085384f0 con 0x55770868cda0 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f74377fe700 1 -- 172.21.15.66:0/730230023 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7448010e80 con 0x55770868cda0 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 10 client.27125 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:02.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5577083f5c60 con 0x55770868cda0 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f74377fe700 1 -- 172.21.15.66:0/730230023 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5577085384f0 con 0x55770868cda0 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f74377fe700 1 -- 172.21.15.66:0/730230023 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7448011440 con 0x55770868cda0 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 10 client.27125 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 10 client.27125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 20 client.27125 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5577086afd40 0x5577086b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 4 client.27125 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.404+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577085d7eb0 con 0x5577086afd40 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.405+0000 7f74577fe700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5577086afd40 0x5577086b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.405+0000 7f74577fe700 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5577086afd40 0x5577086b2150 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1f0068922d407f server_cookie=b856b64028567a6c in_seq=0 out_seq=0 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.405+0000 7f74377fe700 10 client.27125 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f74377fe700 1 -- 172.21.15.66:0/730230023 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5577085d7eb0 con 0x5577086afd40 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f74377fe700 10 client.27125 handle_command_reply: tid=0 2023-07-21T22:33:02.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 client.27125 shutdown 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 msgr2=0x557708694880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 0x557708694880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74480097b0 tx=0x7f7448010c60 comp rx=0 tx=0).stop 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 shutdown_connections 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5577086afd40 0x5577086b2150 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557708697c40 0x557708694fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f74377fe700 0 client.27125 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x557708697670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 --2- 172.21.15.66:0/730230023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55770868cda0 0x557708694880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 >> 172.21.15.66:0/730230023 conn(0x557708682ef0 msgr2=0x5577086883f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 shutdown_connections 2023-07-21T22:33:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.406+0000 7f7472208b80 1 -- 172.21.15.66:0/730230023 wait complete. 2023-07-21T22:33:02.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.407+0000 7f7472208b80 20 client.27125 trim_cache size 0 max 16384 2023-07-21T22:33:02.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 Processor -- start 2023-07-21T22:33:02.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 -- start start 2023-07-21T22:33:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 0x5577086ac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x5577086971a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557708689470 con 0x55770868cda0 2023-07-21T22:33:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770868d170 con 0x557708697c40 2023-07-21T22:33:02.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7472208b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770868d5f0 con 0x55770868c490 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7457fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 0x5577086ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7457fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 0x5577086ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:42772/0 (socket says 172.21.15.66:42772) 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f7457fff700 1 -- 172.21.15.66:0/631770108 learned_addr learned my addr 172.21.15.66:0/631770108 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.426+0000 7f74577fe700 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f74577fe700 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x5577086971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f7457fff700 1 -- 172.21.15.66:0/631770108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 msgr2=0x55770869e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f7457fff700 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f7457fff700 1 -- 172.21.15.66:0/631770108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 msgr2=0x5577086971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f7457fff700 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x5577086971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f7457fff700 1 -- 172.21.15.66:0/631770108 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557708433250 con 0x55770868cda0 2023-07-21T22:33:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f7457fff700 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 0x5577086ac130 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7448000c00 tx=0x7f7448005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efad9a899e46d03b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f743effd700 1 -- 172.21.15.66:0/631770108 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448021040 con 0x55770868cda0 2023-07-21T22:33:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f743effd700 1 -- 172.21.15.66:0/631770108 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7448005710 con 0x55770868cda0 2023-07-21T22:33:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.427+0000 7f743effd700 1 -- 172.21.15.66:0/631770108 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448004d00 con 0x55770868cda0 2023-07-21T22:33:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 -- 172.21.15.66:0/631770108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 msgr2=0x5577086ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 0x5577086ac130 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7448000c00 tx=0x7f7448005ce0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 -- 172.21.15.66:0/631770108 shutdown_connections 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x5577086971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 --2- 172.21.15.66:0/631770108 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770868cda0 0x5577086ac130 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 -- 172.21.15.66:0/631770108 >> 172.21.15.66:0/631770108 conn(0x557708682ef0 msgr2=0x557708686f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 -- 172.21.15.66:0/631770108 shutdown_connections 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.428+0000 7f7472208b80 1 -- 172.21.15.66:0/631770108 wait complete. 2023-07-21T22:33:02.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.429+0000 7f7472208b80 1 -- start start 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.429+0000 7f7472208b80 1 Processor -- start 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x55770869c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770869ccd0 0x55770869d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55770869ec50 con 0x55770869ccd0 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557708703bd0 con 0x557708697c40 2023-07-21T22:33:02.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557708703d10 con 0x55770868c490 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7472208b80 10 client.0 _ll_register_callbacks cb 0x7f7448005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f74577fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7457fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x55770869c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7457fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x55770869c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:37218/0 (socket says 172.21.15.66:37218) 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f74577fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:58432/0 (socket says 172.21.15.66:58432) 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f7457fff700 1 -- 172.21.15.66:0/2642487929 learned_addr learned my addr 172.21.15.66:0/2642487929 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.430+0000 7f74577fe700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770869ccd0 0x55770869d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7457fff700 1 -- 172.21.15.66:0/2642487929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 msgr2=0x55770869e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7457fff700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7457fff700 1 -- 172.21.15.66:0/2642487929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770869ccd0 msgr2=0x55770869d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7457fff700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770869ccd0 0x55770869d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7457fff700 1 -- 172.21.15.66:0/2642487929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577083f5c60 con 0x557708697c40 2023-07-21T22:33:02.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f74577fe700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770869ccd0 0x55770869d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7457fff700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x55770869c790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f74480097b0 tx=0x7f74480093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74ee4011b7dd8a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f74377fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f74377fe700 1 -- 172.21.15.66:0/2642487929 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448021030 con 0x557708697c40 2023-07-21T22:33:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74480044f0 con 0x557708697c40 2023-07-21T22:33:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f74377fe700 1 -- 172.21.15.66:0/2642487929 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7448017070 con 0x557708697c40 2023-07-21T22:33:02.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f74377fe700 1 -- 172.21.15.66:0/2642487929 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7448039070 con 0x557708697c40 2023-07-21T22:33:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7472208b80 10 client.27131 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.431+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55770840b2b0 con 0x557708697c40 2023-07-21T22:33:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f74377fe700 1 -- 172.21.15.66:0/2642487929 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f74480044f0 con 0x557708697c40 2023-07-21T22:33:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f74377fe700 1 -- 172.21.15.66:0/2642487929 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f7448034730 con 0x557708697c40 2023-07-21T22:33:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f7472208b80 10 client.27131 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:02.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f7472208b80 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f7472208b80 20 client.27131 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f7472208b80 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557708704f40 0x557708707350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f7472208b80 4 client.27131 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:33:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55770859a050 con 0x557708704f40 2023-07-21T22:33:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.432+0000 7f74577fe700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557708704f40 0x557708707350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:02.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f74577fe700 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557708704f40 0x557708707350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b626c3da7d91ccb server_cookie=ca3793a3e4a0f952 in_seq=0 out_seq=0 2023-07-21T22:33:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f74377fe700 10 client.27131 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f74377fe700 1 -- 172.21.15.66:0/2642487929 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55770859a050 con 0x557708704f40 2023-07-21T22:33:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f74377fe700 10 client.27131 handle_command_reply: tid=0 2023-07-21T22:33:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 client.27131 shutdown 2023-07-21T22:33:02.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 msgr2=0x55770869c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x55770869c790 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f74480097b0 tx=0x7f74480093d0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 shutdown_connections 2023-07-21T22:33:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557708704f40 0x557708707350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55770869ccd0 0x55770869d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f74377fe700 0 client.27131 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:02.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55770868c490 0x55770869e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 --2- 172.21.15.66:0/2642487929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557708697c40 0x55770869c790 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.489 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 >> 172.21.15.66:0/2642487929 conn(0x557708682ef0 msgr2=0x5577086ac130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.433+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 shutdown_connections 2023-07-21T22:33:02.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f7472208b80 1 -- 172.21.15.66:0/2642487929 wait complete. 2023-07-21T22:33:02.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f7472208b80 20 client.27131 trim_cache size 0 max 16384 2023-07-21T22:33:02.491 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:33:02.491 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:33:02.491 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:33:02.491 INFO:teuthology.orchestra.run.smithi066.stdout: "0c11da94-1d47-47bf-8175-40c52d81d652": { 2023-07-21T22:33:02.491 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:33:02.491 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "0c11da94-1d47-47bf-8175-40c52d81d652", 2023-07-21T22:33:02.492 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:33:02.492 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:33:02.492 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:33:02.493 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:33:02.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74400a14c0 msgr2=0x7f74400a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f745d59d700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74400a14c0 0x7f74400a3970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f74500ca0a0 tx=0x7f7450009ed0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 msgr2=0x7f745819a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f745d59d700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 0x7f745819a340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f745000ef30 tx=0x7f745000c450 comp rx=0 tx=0).stop 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.434+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 shutdown_connections 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581059e0 0x7f745819a340 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74400a14c0 0x7f74400a3970 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f74580ff9d0 0x7f7458195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 --2- 172.21.15.66:0/3743927709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7458100380 0x7f7458195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:02.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 >> 172.21.15.66:0/3743927709 conn(0x7f74580fb4b0 msgr2=0x7f74581067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:02.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 shutdown_connections 2023-07-21T22:33:02.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:02.435+0000 7f745d59d700 1 -- 172.21.15.66:0/3743927709 wait complete. 2023-07-21T22:33:02.495 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:02 smithi149 ceph-mon[129268]: pgmap v359: 129 pgs: 129 active+clean; 528 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 278 op/s 2023-07-21T22:33:02.500 INFO:tasks.cephfs.filesystem:scrub status for tag:0c11da94-1d47-47bf-8175-40c52d81d652 - {'path': '/', 'tag': '0c11da94-1d47-47bf-8175-40c52d81d652', 'options': 'recursive,force'} 2023-07-21T22:33:02.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 547177 5.34 MB/sec execute 509 sec latency 78.216 ms 2023-07-21T22:33:03.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:33:03.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:03.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:03.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:03.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:03 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:03.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:03.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:03.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:03 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:33:03.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:03.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:03.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:03 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:03.569 INFO:tasks.workunit.client.0.smithi066.stdout: 1 548049 5.33 MB/sec execute 510 sec latency 19.844 ms 2023-07-21T22:33:04.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:04 smithi104 ceph-mon[129021]: pgmap v360: 129 pgs: 129 active+clean; 528 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 192 op/s 2023-07-21T22:33:04.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:04 smithi066 ceph-mon[126379]: pgmap v360: 129 pgs: 129 active+clean; 528 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 192 op/s 2023-07-21T22:33:04.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:04 smithi149 ceph-mon[129268]: pgmap v360: 129 pgs: 129 active+clean; 528 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 192 op/s 2023-07-21T22:33:04.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 549405 5.33 MB/sec execute 511 sec latency 25.114 ms 2023-07-21T22:33:05.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:05 smithi066 ceph-mon[126379]: pgmap v361: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 314 op/s 2023-07-21T22:33:05.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:05 smithi149 ceph-mon[129268]: pgmap v361: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 314 op/s 2023-07-21T22:33:05.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 550860 5.34 MB/sec execute 512 sec latency 26.634 ms 2023-07-21T22:33:05.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:05 smithi104 ceph-mon[129021]: pgmap v361: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 314 op/s 2023-07-21T22:33:06.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 551797 5.34 MB/sec execute 513 sec latency 42.851 ms 2023-07-21T22:33:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:07 smithi066 ceph-mon[126379]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T22:33:07.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:07 smithi066 ceph-mon[126379]: pgmap v363: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2023-07-21T22:33:07.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:07 smithi149 ceph-mon[129268]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T22:33:07.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:07 smithi149 ceph-mon[129268]: pgmap v363: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2023-07-21T22:33:07.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 553218 5.34 MB/sec execute 514 sec latency 15.638 ms 2023-07-21T22:33:07.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:07 smithi104 ceph-mon[129021]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T22:33:07.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:07 smithi104 ceph-mon[129021]: pgmap v363: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2023-07-21T22:33:08.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 554630 5.35 MB/sec execute 515 sec latency 11.906 ms 2023-07-21T22:33:09.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 555769 5.35 MB/sec execute 516 sec latency 63.653 ms 2023-07-21T22:33:10.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:09 smithi104 ceph-mon[129021]: pgmap v364: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2023-07-21T22:33:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:09 smithi066 ceph-mon[126379]: pgmap v364: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2023-07-21T22:33:10.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:09 smithi149 ceph-mon[129268]: pgmap v364: 129 pgs: 129 active+clean; 535 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 262 op/s 2023-07-21T22:33:10.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 557517 5.36 MB/sec execute 517 sec latency 23.746 ms 2023-07-21T22:33:11.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:10 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:33:11.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:10 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:33:11.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:10 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:33:11.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 558427 5.36 MB/sec execute 518 sec latency 36.196 ms 2023-07-21T22:33:12.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:11 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:33:12.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:11 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:33:12.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:11 smithi104 ceph-mon[129021]: pgmap v365: 129 pgs: 129 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2023-07-21T22:33:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:11 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:33:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:11 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:33:12.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:11 smithi066 ceph-mon[126379]: pgmap v365: 129 pgs: 129 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2023-07-21T22:33:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:11 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:33:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:11 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:33:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:11 smithi149 ceph-mon[129268]: pgmap v365: 129 pgs: 129 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2023-07-21T22:33:12.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 559684 5.36 MB/sec execute 519 sec latency 18.423 ms 2023-07-21T22:33:13.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 561472 5.37 MB/sec execute 520 sec latency 18.664 ms 2023-07-21T22:33:14.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:13 smithi104 ceph-mon[129021]: pgmap v366: 129 pgs: 129 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2023-07-21T22:33:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:13 smithi104 ceph-mon[129021]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T22:33:14.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:13 smithi066 ceph-mon[126379]: pgmap v366: 129 pgs: 129 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2023-07-21T22:33:14.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:13 smithi066 ceph-mon[126379]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T22:33:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:13 smithi149 ceph-mon[129268]: pgmap v366: 129 pgs: 129 active+clean; 543 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2023-07-21T22:33:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:13 smithi149 ceph-mon[129268]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T22:33:14.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 562234 5.36 MB/sec execute 521 sec latency 53.892 ms 2023-07-21T22:33:15.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 563748 5.37 MB/sec execute 522 sec latency 47.130 ms 2023-07-21T22:33:16.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:15 smithi104 ceph-mon[129021]: pgmap v368: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-07-21T22:33:16.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:15 smithi066 ceph-mon[126379]: pgmap v368: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-07-21T22:33:16.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:15 smithi149 ceph-mon[129268]: pgmap v368: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 261 op/s 2023-07-21T22:33:16.570 INFO:tasks.workunit.client.0.smithi066.stdout: 1 565162 5.38 MB/sec execute 523 sec latency 28.269 ms 2023-07-21T22:33:17.571 INFO:tasks.workunit.client.0.smithi066.stdout: 1 566324 5.38 MB/sec execute 524 sec latency 15.867 ms 2023-07-21T22:33:18.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:17 smithi104 ceph-mon[129021]: pgmap v369: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-21T22:33:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:17 smithi066 ceph-mon[126379]: pgmap v369: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-21T22:33:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:17 smithi149 ceph-mon[129268]: pgmap v369: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-21T22:33:18.571 INFO:tasks.workunit.client.0.smithi066.stdout: 1 568147 5.38 MB/sec execute 525 sec latency 17.173 ms 2023-07-21T22:33:19.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 568953 5.38 MB/sec execute 526 sec latency 37.801 ms 2023-07-21T22:33:20.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:19 smithi104 ceph-mon[129021]: pgmap v370: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-21T22:33:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:19 smithi066 ceph-mon[126379]: pgmap v370: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-21T22:33:20.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:19 smithi149 ceph-mon[129268]: pgmap v370: 129 pgs: 129 active+clean; 556 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 229 op/s 2023-07-21T22:33:20.571 INFO:tasks.workunit.client.0.smithi066.stdout: 1 570484 5.38 MB/sec execute 527 sec latency 25.494 ms 2023-07-21T22:33:21.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 572096 5.40 MB/sec execute 528 sec latency 25.468 ms 2023-07-21T22:33:22.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:21 smithi104 ceph-mon[129021]: pgmap v371: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2023-07-21T22:33:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:21 smithi066 ceph-mon[126379]: pgmap v371: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2023-07-21T22:33:22.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:21 smithi149 ceph-mon[129268]: pgmap v371: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2023-07-21T22:33:22.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 572980 5.39 MB/sec execute 529 sec latency 13.711 ms 2023-07-21T22:33:23.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 574655 5.40 MB/sec execute 530 sec latency 17.394 ms 2023-07-21T22:33:24.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:23 smithi104 ceph-mon[129021]: pgmap v372: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2023-07-21T22:33:24.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:23 smithi066 ceph-mon[126379]: pgmap v372: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2023-07-21T22:33:24.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:23 smithi149 ceph-mon[129268]: pgmap v372: 129 pgs: 129 active+clean; 567 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2023-07-21T22:33:24.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 575903 5.40 MB/sec execute 531 sec latency 12.107 ms 2023-07-21T22:33:25.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 577043 5.40 MB/sec execute 532 sec latency 46.046 ms 2023-07-21T22:33:26.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:25 smithi104 ceph-mon[129021]: pgmap v373: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 303 op/s 2023-07-21T22:33:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:25 smithi066 ceph-mon[126379]: pgmap v373: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 303 op/s 2023-07-21T22:33:26.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:25 smithi149 ceph-mon[129268]: pgmap v373: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 303 op/s 2023-07-21T22:33:26.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 578703 5.41 MB/sec execute 533 sec latency 59.460 ms 2023-07-21T22:33:27.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 579663 5.41 MB/sec execute 534 sec latency 29.417 ms 2023-07-21T22:33:28.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:27 smithi104 ceph-mon[129021]: pgmap v374: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-21T22:33:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:27 smithi066 ceph-mon[126379]: pgmap v374: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-21T22:33:28.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:27 smithi149 ceph-mon[129268]: pgmap v374: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-21T22:33:28.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 581394 5.41 MB/sec execute 535 sec latency 16.547 ms 2023-07-21T22:33:29.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 582913 5.43 MB/sec execute 536 sec latency 12.117 ms 2023-07-21T22:33:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:29 smithi104 ceph-mon[129021]: pgmap v375: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-21T22:33:30.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:29 smithi066 ceph-mon[126379]: pgmap v375: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-21T22:33:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:29 smithi149 ceph-mon[129268]: pgmap v375: 129 pgs: 129 active+clean; 573 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 206 op/s 2023-07-21T22:33:30.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 583974 5.42 MB/sec execute 537 sec latency 35.159 ms 2023-07-21T22:33:31.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 585461 5.43 MB/sec execute 538 sec latency 48.344 ms 2023-07-21T22:33:32.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:32 smithi104 ceph-mon[129021]: pgmap v376: 129 pgs: 129 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2023-07-21T22:33:32.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:32 smithi066 ceph-mon[126379]: pgmap v376: 129 pgs: 129 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2023-07-21T22:33:32.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:32 smithi149 ceph-mon[129268]: pgmap v376: 129 pgs: 129 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2023-07-21T22:33:32.501 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:33:32.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 586673 5.43 MB/sec execute 539 sec latency 19.607 ms 2023-07-21T22:33:32.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 -- 172.21.15.66:0/614899114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 msgr2=0x7f111c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 --2- 172.21.15.66:0/614899114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 0x7f111c105eb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f110c009990 tx=0x7f110c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 -- 172.21.15.66:0/614899114 shutdown_connections 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 --2- 172.21.15.66:0/614899114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c100480 0x7f111c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 --2- 172.21.15.66:0/614899114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 0x7f111c105eb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 --2- 172.21.15.66:0/614899114 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111c0ffad0 0x7f111c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 -- 172.21.15.66:0/614899114 >> 172.21.15.66:0/614899114 conn(0x7f111c0fb5f0 msgr2=0x7f111c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 -- 172.21.15.66:0/614899114 shutdown_connections 2023-07-21T22:33:32.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.684+0000 7f1120bff700 1 -- 172.21.15.66:0/614899114 wait complete. 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.685+0000 7f1120bff700 1 Processor -- start 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.685+0000 7f1120bff700 1 -- start start 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1120bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 0x7f111c19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1120bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111c100480 0x7f111c19bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1120bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 0x7f111c195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1120bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c196190 con 0x7f111c100480 2023-07-21T22:33:32.733 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1120bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c196300 con 0x7f111c0ffad0 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1120bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c1964c0 con 0x7f111c105ae0 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 0x7f111c19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111c100480 0x7f111c19bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 0x7f111c19ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41112/0 (socket says 172.21.15.66:41112) 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 -- 172.21.15.66:0/2268531048 learned_addr learned my addr 172.21.15.66:0/2268531048 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111ad9d700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 0x7f111c195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 -- 172.21.15.66:0/2268531048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 msgr2=0x7f111c195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 0x7f111c195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 -- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111c100480 msgr2=0x7f111c19bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111c100480 0x7f111c19bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.686+0000 7f111a59c700 1 -- 172.21.15.66:0/2268531048 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1114009c40 con 0x7f111c0ffad0 2023-07-21T22:33:32.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.687+0000 7f111a59c700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 0x7f111c19ba60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f110c005910 tx=0x7f110c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:32.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.687+0000 7f110b7fe700 1 -- 172.21.15.66:0/2268531048 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c01d070 con 0x7f111c0ffad0 2023-07-21T22:33:32.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.687+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c0095f0 con 0x7f111c0ffad0 2023-07-21T22:33:32.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.687+0000 7f110b7fe700 1 -- 172.21.15.66:0/2268531048 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f110c022470 con 0x7f111c0ffad0 2023-07-21T22:33:32.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.687+0000 7f110b7fe700 1 -- 172.21.15.66:0/2268531048 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c017650 con 0x7f111c0ffad0 2023-07-21T22:33:32.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.687+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c1a2870 con 0x7f111c0ffad0 2023-07-21T22:33:32.737 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.690+0000 7f110b7fe700 1 -- 172.21.15.66:0/2268531048 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f110c0225e0 con 0x7f111c0ffad0 2023-07-21T22:33:32.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 Processor -- start 2023-07-21T22:33:32.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 -- start start 2023-07-21T22:33:32.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d331990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d331ed0 0x55d73d328350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d328da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d22dcb0 con 0x55d73d331ed0 2023-07-21T22:33:32.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d280a40 con 0x55d73d3315c0 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f11344fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d279ed0 con 0x55d73d328930 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.692+0000 7f110b7fe700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f11040a12c0 0x7f11040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d328da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d328da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50268/0 (socket says 172.21.15.66:50268) 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 -- 172.21.15.66:0/826286066 learned_addr learned my addr 172.21.15.66:0/826286066 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f110b7fe700 1 -- 172.21.15.66:0/2268531048 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f110c0ce2d0 con 0x7f111c0ffad0 2023-07-21T22:33:32.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f111ad9d700 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d331990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f111ad9d700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f11040a12c0 0x7f11040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 -- 172.21.15.66:0/826286066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 msgr2=0x55d73d331990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d331990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 -- 172.21.15.66:0/826286066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d331ed0 msgr2=0x55d73d328350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f111a59c700 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d331ed0 0x55d73d328350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d331ed0 0x55d73d328350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f1119d9b700 1 -- 172.21.15.66:0/826286066 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d73d0a7de0 con 0x55d73d328930 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.693+0000 7f111a59c700 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d331ed0 0x55d73d328350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f1119d9b700 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d328da0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1110009d20 tx=0x7f1110011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674452b150122c55 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f111ad9d700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f11040a12c0 0x7f11040a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1114009550 tx=0x7f111400b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f1108ff9700 1 -- 172.21.15.66:0/826286066 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110019630 con 0x55d73d328930 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f1108ff9700 1 -- 172.21.15.66:0/826286066 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1110019c50 con 0x55d73d328930 2023-07-21T22:33:32.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f1108ff9700 1 -- 172.21.15.66:0/826286066 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000ec40 con 0x55d73d328930 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f11344fcb80 1 -- 172.21.15.66:0/826286066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 msgr2=0x55d73d328da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.694+0000 7f11344fcb80 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d328da0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1110009d20 tx=0x7f1110011720 comp rx=0 tx=0).stop 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 -- 172.21.15.66:0/826286066 shutdown_connections 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d331990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d328da0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 --2- 172.21.15.66:0/826286066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d331ed0 0x55d73d328350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 -- 172.21.15.66:0/826286066 >> 172.21.15.66:0/826286066 conn(0x55d73d31def0 msgr2=0x55d73d323830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 -- 172.21.15.66:0/826286066 shutdown_connections 2023-07-21T22:33:32.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 -- 172.21.15.66:0/826286066 wait complete. 2023-07-21T22:33:32.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 -- start start 2023-07-21T22:33:32.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.695+0000 7f11344fcb80 1 Processor -- start 2023-07-21T22:33:32.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d339050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d3315c0 0x55d73d336260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d3369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d219ac0 con 0x55d73d3315c0 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d2197c0 con 0x55d73d331ed0 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d0fd920 con 0x55d73d328930 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f11344fcb80 10 client.0 _ll_register_callbacks cb 0x55d73d217760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d3369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d3369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41140/0 (socket says 172.21.15.66:41140) 2023-07-21T22:33:32.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f111ad9d700 1 -- 172.21.15.66:0/3417858863 learned_addr learned my addr 172.21.15.66:0/3417858863 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:32.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f111a59c700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d339050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.696+0000 7f1119d9b700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d3315c0 0x55d73d336260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f111ad9d700 1 -- 172.21.15.66:0/3417858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 msgr2=0x55d73d339050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f111ad9d700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d339050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f111ad9d700 1 -- 172.21.15.66:0/3417858863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d3315c0 msgr2=0x55d73d336260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f111ad9d700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d3315c0 0x55d73d336260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f111ad9d700 1 -- 172.21.15.66:0/3417858863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1110009980 con 0x55d73d331ed0 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f1119d9b700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d3315c0 0x55d73d336260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f111ad9d700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d3369a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1114018f80 tx=0x7f1114004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892610d31f9f2e37 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f1101ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f1101ffb700 1 -- 172.21.15.66:0/3417858863 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400e070 con 0x55d73d331ed0 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d73d1d34f0 con 0x55d73d331ed0 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.697+0000 7f1101ffb700 1 -- 172.21.15.66:0/3417858863 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f111401f050 con 0x55d73d331ed0 2023-07-21T22:33:32.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f1101ffb700 1 -- 172.21.15.66:0/3417858863 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1114004bb0 con 0x55d73d331ed0 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 10 client.27143 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d73d090c60 con 0x55d73d331ed0 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f1101ffb700 1 -- 172.21.15.66:0/3417858863 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d73d1d34f0 con 0x55d73d331ed0 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f1101ffb700 1 -- 172.21.15.66:0/3417858863 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f1114005920 con 0x55d73d331ed0 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 10 client.27143 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 10 client.27143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 20 client.27143 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:32.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d346880 0x55d73d348c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 4 client.27143 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.698+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d73d272eb0 con 0x55d73d346880 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.699+0000 7f111a59c700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d346880 0x55d73d348c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.699+0000 7f1101ffb700 10 client.27143 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.699+0000 7f111a59c700 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d346880 0x55d73d348c90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2eadd19070b0ca7 server_cookie=46ec91850eda6ecb in_seq=0 out_seq=0 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f1101ffb700 1 -- 172.21.15.66:0/3417858863 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d73d272eb0 con 0x55d73d346880 2023-07-21T22:33:32.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f1101ffb700 10 client.27143 handle_command_reply: tid=0 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 client.27143 shutdown 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 msgr2=0x55d73d3369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d3369a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1114018f80 tx=0x7f1114004ad0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 shutdown_connections 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d3369a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d346880 0x55d73d348c90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d328930 0x55d73d339050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3417858863 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d3315c0 0x55d73d336260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 >> 172.21.15.66:0/3417858863 conn(0x55d73d31def0 msgr2=0x55d73d323420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f1101ffb700 0 client.27143 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 shutdown_connections 2023-07-21T22:33:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 1 -- 172.21.15.66:0/3417858863 wait complete. 2023-07-21T22:33:32.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.700+0000 7f11344fcb80 20 client.27143 trim_cache size 0 max 16384 2023-07-21T22:33:32.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.722+0000 7f11344fcb80 1 Processor -- start 2023-07-21T22:33:32.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.722+0000 7f11344fcb80 1 -- start start 2023-07-21T22:33:32.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.722+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 0x55d73d338b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d326980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f11344fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d339770 con 0x55d73d328930 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f11344fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d272eb0 con 0x55d73d331ed0 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f11344fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d1d34f0 con 0x55d73d3315c0 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 0x55d73d338b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d326980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 0x55d73d338b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50280/0 (socket says 172.21.15.66:50280) 2023-07-21T22:33:32.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 -- 172.21.15.66:0/2946011792 learned_addr learned my addr 172.21.15.66:0/2946011792 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:32.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f111a59c700 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 -- 172.21.15.66:0/2946011792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 msgr2=0x55d73d326980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d326980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 -- 172.21.15.66:0/2946011792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 msgr2=0x55d73d33d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f1119d9b700 1 -- 172.21.15.66:0/2946011792 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d73d0ce250 con 0x55d73d3315c0 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.723+0000 7f111ad9d700 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d326980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:32.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.724+0000 7f1119d9b700 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 0x55d73d338b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1110009c70 tx=0x7f1110005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ef108ab494c59 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.724+0000 7f11097fa700 1 -- 172.21.15.66:0/2946011792 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110019630 con 0x55d73d3315c0 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.724+0000 7f11097fa700 1 -- 172.21.15.66:0/2946011792 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1110005040 con 0x55d73d3315c0 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.724+0000 7f11097fa700 1 -- 172.21.15.66:0/2946011792 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110003ae0 con 0x55d73d3315c0 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 -- 172.21.15.66:0/2946011792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 msgr2=0x55d73d338b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 0x55d73d338b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1110009c70 tx=0x7f1110005d00 comp rx=0 tx=0).stop 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 -- 172.21.15.66:0/2946011792 shutdown_connections 2023-07-21T22:33:32.773 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d331ed0 0x55d73d326980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 --2- 172.21.15.66:0/2946011792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d3315c0 0x55d73d338b80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 -- 172.21.15.66:0/2946011792 >> 172.21.15.66:0/2946011792 conn(0x55d73d31def0 msgr2=0x55d73d3232d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 -- 172.21.15.66:0/2946011792 shutdown_connections 2023-07-21T22:33:32.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.725+0000 7f11344fcb80 1 -- 172.21.15.66:0/2946011792 wait complete. 2023-07-21T22:33:32.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.726+0000 7f11344fcb80 1 -- start start 2023-07-21T22:33:32.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.726+0000 7f11344fcb80 1 Processor -- start 2023-07-21T22:33:32.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d33bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d331ed0 0x55d73d33c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d33c9d0 con 0x55d73d328930 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d33cb40 con 0x55d73d3315c0 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d73d39a710 con 0x55d73d331ed0 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f11344fcb80 10 client.0 _ll_register_callbacks cb 0x7f1114008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:32.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d33bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f1119d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d33bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41150/0 (socket says 172.21.15.66:41150) 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f1119d9b700 1 -- 172.21.15.66:0/3760018985 learned_addr learned my addr 172.21.15.66:0/3760018985 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.727+0000 7f111a59c700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f111ad9d700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d331ed0 0x55d73d33c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1119d9b700 1 -- 172.21.15.66:0/3760018985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d331ed0 msgr2=0x55d73d33c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1119d9b700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d331ed0 0x55d73d33c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1119d9b700 1 -- 172.21.15.66:0/3760018985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 msgr2=0x55d73d33d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:32.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1119d9b700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1119d9b700 1 -- 172.21.15.66:0/3760018985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d73d090c60 con 0x55d73d3315c0 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f111a59c700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1119d9b700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d33bdc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1110013010 tx=0x7f1110011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a6fd1a93ca1419 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.728+0000 7f1101ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f1101ffb700 1 -- 172.21.15.66:0/3760018985 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100199c0 con 0x55d73d3315c0 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1110019c50 con 0x55d73d3315c0 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f1101ffb700 1 -- 172.21.15.66:0/3760018985 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1110003680 con 0x55d73d3315c0 2023-07-21T22:33:32.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f1101ffb700 1 -- 172.21.15.66:0/3760018985 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1110003840 con 0x55d73d3315c0 2023-07-21T22:33:32.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f11344fcb80 10 client.27149 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:32.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d73d0a7de0 con 0x55d73d3315c0 2023-07-21T22:33:32.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f1101ffb700 1 -- 172.21.15.66:0/3760018985 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1110019c50 con 0x55d73d3315c0 2023-07-21T22:33:32.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.729+0000 7f1101ffb700 1 -- 172.21.15.66:0/3760018985 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f1110037b20 con 0x55d73d3315c0 2023-07-21T22:33:32.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f11344fcb80 10 client.27149 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:32.780 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f11344fcb80 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f11344fcb80 20 client.27149 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d39b820 0x55d73d39dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f11344fcb80 4 client.27149 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:33:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d73d235050 con 0x55d73d39b820 2023-07-21T22:33:32.781 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f111a59c700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d39b820 0x55d73d39dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f111a59c700 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d39b820 0x55d73d39dc30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df27f423f383118 server_cookie=7cc9a1183fb9e2d4 in_seq=0 out_seq=0 2023-07-21T22:33:32.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.730+0000 7f1101ffb700 10 client.27149 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f1101ffb700 1 -- 172.21.15.66:0/3760018985 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d73d235050 con 0x55d73d39b820 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f1101ffb700 10 client.27149 handle_command_reply: tid=0 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 client.27149 shutdown 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 msgr2=0x55d73d33bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d33bdc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1110013010 tx=0x7f1110011720 comp rx=0 tx=0).stop 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 shutdown_connections 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d73d39b820 0x55d73d39dc30 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d73d331ed0 0x55d73d33c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d73d328930 0x55d73d33d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 --2- 172.21.15.66:0/3760018985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d73d3315c0 0x55d73d33bdc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f1101ffb700 0 client.27149 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 >> 172.21.15.66:0/3760018985 conn(0x55d73d31def0 msgr2=0x55d73d326980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 shutdown_connections 2023-07-21T22:33:32.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.731+0000 7f11344fcb80 1 -- 172.21.15.66:0/3760018985 wait complete. 2023-07-21T22:33:32.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.732+0000 7f11344fcb80 20 client.27149 trim_cache size 0 max 16384 2023-07-21T22:33:32.785 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:33:32.785 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:33:32.785 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:33:32.785 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:33:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.732+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f11040a12c0 msgr2=0x7f11040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.732+0000 7f1120bff700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f11040a12c0 0x7f11040a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1114009550 tx=0x7f111400b040 comp rx=0 tx=0).stop 2023-07-21T22:33:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.732+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 msgr2=0x7f111c19ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:32.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.732+0000 7f1120bff700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 0x7f111c19ba60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f110c005910 tx=0x7f110c004710 comp rx=0 tx=0).stop 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 shutdown_connections 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f111c0ffad0 0x7f111c19ba60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f11040a12c0 0x7f11040a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f111c105ae0 0x7f111c195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 --2- 172.21.15.66:0/2268531048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f111c100480 0x7f111c19bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 >> 172.21.15.66:0/2268531048 conn(0x7f111c0fb5f0 msgr2=0x7f111c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:32.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 shutdown_connections 2023-07-21T22:33:32.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.733+0000 7f1120bff700 1 -- 172.21.15.66:0/2268531048 wait complete. 2023-07-21T22:33:32.797 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:33:32.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:33:33.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 -- 172.21.15.66:0/720424727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe0105ae0 msgr2=0x7fcbe0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/720424727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe0105ae0 0x7fcbe0105eb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009990 tx=0x7fcbd0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 -- 172.21.15.66:0/720424727 shutdown_connections 2023-07-21T22:33:33.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/720424727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe0105ae0 0x7fcbe0105eb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/720424727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe0100480 0x7fcbe010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/720424727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe00ffad0 0x7fcbe00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 -- 172.21.15.66:0/720424727 >> 172.21.15.66:0/720424727 conn(0x7fcbe00fb590 msgr2=0x7fcbe00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 -- 172.21.15.66:0/720424727 shutdown_connections 2023-07-21T22:33:33.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.972+0000 7fcbe53ff700 1 -- 172.21.15.66:0/720424727 wait complete. 2023-07-21T22:33:33.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.973+0000 7fcbe53ff700 1 Processor -- start 2023-07-21T22:33:33.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.973+0000 7fcbe53ff700 1 -- start start 2023-07-21T22:33:33.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbe53ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 0x7fcbe0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbe53ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe0100480 0x7fcbe0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbe53ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0105ae0 0x7fcbe01a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbe53ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe0073580 con 0x7fcbe0100480 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbe53ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe01a42b0 con 0x7fcbe00ffad0 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbe53ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe01a43f0 con 0x7fcbe0105ae0 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbdeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 0x7fcbe0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbdeffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 0x7fcbe0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41174/0 (socket says 172.21.15.66:41174) 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbdeffd700 1 -- 172.21.15.66:0/968285799 learned_addr learned my addr 172.21.15.66:0/968285799 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbd7fff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe0100480 0x7fcbe0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.974+0000 7fcbdf7fe700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0105ae0 0x7fcbe01a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.975+0000 7fcbdeffd700 1 -- 172.21.15.66:0/968285799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0105ae0 msgr2=0x7fcbe01a3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.975+0000 7fcbdeffd700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0105ae0 0x7fcbe01a3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.975+0000 7fcbdeffd700 1 -- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe0100480 msgr2=0x7fcbe0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.975+0000 7fcbdeffd700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe0100480 0x7fcbe0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.975+0000 7fcbdeffd700 1 -- 172.21.15.66:0/968285799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd00095f0 con 0x7fcbe00ffad0 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.975+0000 7fcbdeffd700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 0x7fcbe0072810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005910 tx=0x7fcbd0005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.976+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd001d070 con 0x7fcbe00ffad0 2023-07-21T22:33:33.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.977+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcbd0022470 con 0x7fcbe00ffad0 2023-07-21T22:33:33.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.977+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe01a45f0 con 0x7fcbe00ffad0 2023-07-21T22:33:33.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.977+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0017650 con 0x7fcbe00ffad0 2023-07-21T22:33:33.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.977+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe01a4a70 con 0x7fcbe00ffad0 2023-07-21T22:33:33.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.978+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe0067300 con 0x7fcbe00ffad0 2023-07-21T22:33:33.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.980+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcbd00225e0 con 0x7fcbe00ffad0 2023-07-21T22:33:33.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.981+0000 7fcbdcff9700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcbcc0a1380 0x7fcbcc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.982+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcbd00ce380 con 0x7fcbe00ffad0 2023-07-21T22:33:33.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.982+0000 7fcbd7fff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcbcc0a1380 0x7fcbcc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.982+0000 7fcbd7fff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcbcc0a1380 0x7fcbcc0a3830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0074000 tx=0x7fcbc8009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:32.985+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcbd0097420 con 0x7fcbe00ffad0 2023-07-21T22:33:33.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.260+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbe01a4d90 con 0x7fcbe00ffad0 2023-07-21T22:33:33.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.261+0000 7fcbdcff9700 1 -- 172.21.15.66:0/968285799 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10827 (secure 0 0 0) 0x7fcbd009aa40 con 0x7fcbe00ffad0 2023-07-21T22:33:33.307 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:33:33.307 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:32:38.011518+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:33:33.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.263+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcbcc0a1380 msgr2=0x7fcbcc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.263+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcbcc0a1380 0x7fcbcc0a3830 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0074000 tx=0x7fcbc8009260 comp rx=0 tx=0).stop 2023-07-21T22:33:33.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.263+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 msgr2=0x7fcbe0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.263+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 0x7fcbe0072810 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0005910 tx=0x7fcbd0005720 comp rx=0 tx=0).stop 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 shutdown_connections 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbe00ffad0 0x7fcbe0072810 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fcbcc0a1380 0x7fcbcc0a3830 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcbe0105ae0 0x7fcbe01a3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 --2- 172.21.15.66:0/968285799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fcbe0100480 0x7fcbe0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 >> 172.21.15.66:0/968285799 conn(0x7fcbe00fb590 msgr2=0x7fcbe00fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 shutdown_connections 2023-07-21T22:33:33.312 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.264+0000 7fcbe53ff700 1 -- 172.21.15.66:0/968285799 wait complete. 2023-07-21T22:33:33.312 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 42 2023-07-21T22:33:33.333 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:33:33.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 -- 172.21.15.66:0/3559907231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00728b0 msgr2=0x7f4ed0072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/3559907231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00728b0 0x7f4ed0072c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009990 tx=0x7f4ecc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 -- 172.21.15.66:0/3559907231 shutdown_connections 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/3559907231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed010c320 0x7f4ed010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/3559907231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00728b0 0x7f4ed0072c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/3559907231 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed00731c0 0x7f4ed010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 -- 172.21.15.66:0/3559907231 >> 172.21.15.66:0/3559907231 conn(0x7f4ed00fb5e0 msgr2=0x7f4ed00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 -- 172.21.15.66:0/3559907231 shutdown_connections 2023-07-21T22:33:33.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.505+0000 7f4ed83ff700 1 -- 172.21.15.66:0/3559907231 wait complete. 2023-07-21T22:33:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.506+0000 7f4ed83ff700 1 Processor -- start 2023-07-21T22:33:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 -- start start 2023-07-21T22:33:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 0x7f4ed0197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00731c0 0x7f4ed0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed010c320 0x7f4ed019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed019ce00 con 0x7f4ed010c320 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed019cf70 con 0x7f4ed00728b0 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed83ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed019d130 con 0x7f4ed00731c0 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 0x7f4ed0197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 0x7f4ed0197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41180/0 (socket says 172.21.15.66:41180) 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.507+0000 7f4ed619b700 1 -- 172.21.15.66:0/1236526600 learned_addr learned my addr 172.21.15.66:0/1236526600 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed599a700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00731c0 0x7f4ed0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed699c700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed010c320 0x7f4ed019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed619b700 1 -- 172.21.15.66:0/1236526600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00731c0 msgr2=0x7f4ed0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed619b700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00731c0 0x7f4ed0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed619b700 1 -- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed010c320 msgr2=0x7f4ed019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed619b700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed010c320 0x7f4ed019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed619b700 1 -- 172.21.15.66:0/1236526600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0095f0 con 0x7f4ed00728b0 2023-07-21T22:33:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ed619b700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 0x7f4ed0197e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005910 tx=0x7f4ecc004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.508+0000 7f4ec77fe700 1 -- 172.21.15.66:0/1236526600 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01d070 con 0x7f4ed00728b0 2023-07-21T22:33:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.509+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed019d3b0 con 0x7f4ed00728b0 2023-07-21T22:33:33.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.509+0000 7f4ec77fe700 1 -- 172.21.15.66:0/1236526600 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ecc022470 con 0x7f4ed00728b0 2023-07-21T22:33:33.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.509+0000 7f4ec77fe700 1 -- 172.21.15.66:0/1236526600 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc017650 con 0x7f4ed00728b0 2023-07-21T22:33:33.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.509+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed01a4c50 con 0x7f4ed00728b0 2023-07-21T22:33:33.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.512+0000 7f4ec77fe700 1 -- 172.21.15.66:0/1236526600 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4ecc0225e0 con 0x7f4ed00728b0 2023-07-21T22:33:33.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 Processor -- start 2023-07-21T22:33:33.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 -- start start 2023-07-21T22:33:33.560 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4ec77fe700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4ebc0a1340 0x7f4ebc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c11b0 0x55bcd03c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4ec77fe700 1 -- 172.21.15.66:0/1236526600 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4ecc0ce380 con 0x7f4ed00728b0 2023-07-21T22:33:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.561 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd02c1cb0 con 0x55bcd03c11b0 2023-07-21T22:33:33.562 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd0314a40 con 0x55bcd03bc930 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd030ded0 con 0x55bcd03c1ac0 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.514+0000 7f4ed699c700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4ebc0a1340 0x7f4ebc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50320/0 (socket says 172.21.15.66:50320) 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 -- 172.21.15.66:0/2391355597 learned_addr learned my addr 172.21.15.66:0/2391355597 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed619b700 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c11b0 0x55bcd03c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.563 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 -- 172.21.15.66:0/2391355597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 msgr2=0x55bcd03bcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03bcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 -- 172.21.15.66:0/2391355597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c11b0 msgr2=0x55bcd03c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c11b0 0x55bcd03c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 -- 172.21.15.66:0/2391355597 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcd013bde0 con 0x55bcd03c1ac0 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed699c700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4ebc0a1340 0x7f4ebc0a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009510 tx=0x7f4ec8009220 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.515+0000 7f4ed599a700 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bc350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d20 tx=0x7f4ec0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d82dfcfa099178 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.564 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.516+0000 7f4ec4bfe700 1 -- 172.21.15.66:0/2391355597 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0019630 con 0x55bcd03c1ac0 2023-07-21T22:33:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.516+0000 7f4ec4bfe700 1 -- 172.21.15.66:0/2391355597 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec0019c50 con 0x55bcd03c1ac0 2023-07-21T22:33:33.565 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.516+0000 7f4ec4bfe700 1 -- 172.21.15.66:0/2391355597 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000ec90 con 0x55bcd03c1ac0 2023-07-21T22:33:33.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2391355597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 msgr2=0x55bcd03bc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bc350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d20 tx=0x7f4ec0011720 comp rx=0 tx=0).stop 2023-07-21T22:33:33.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2391355597 shutdown_connections 2023-07-21T22:33:33.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c11b0 0x55bcd03c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2391355597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bc350 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2391355597 >> 172.21.15.66:0/2391355597 conn(0x55bcd03b1ef0 msgr2=0x55bcd03b7830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2391355597 shutdown_connections 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2391355597 wait complete. 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 -- start start 2023-07-21T22:33:33.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.517+0000 7f4eebcd0b80 1 Processor -- start 2023-07-21T22:33:33.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd02adac0 con 0x55bcd03c1ac0 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd02ad7c0 con 0x55bcd03bc930 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd0191920 con 0x55bcd03c11b0 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4eebcd0b80 10 client.0 _ll_register_callbacks cb 0x55bcd02ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4ed599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41184/0 (socket says 172.21.15.66:41184) 2023-07-21T22:33:33.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4ed599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50330/0 (socket says 172.21.15.66:50330) 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4ed619b700 1 -- 172.21.15.66:0/2190435904 learned_addr learned my addr 172.21.15.66:0/2190435904 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.518+0000 7f4ed699c700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed619b700 1 -- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 msgr2=0x55bcd03c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed619b700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed619b700 1 -- 172.21.15.66:0/2190435904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 msgr2=0x55bcd03c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed619b700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed619b700 1 -- 172.21.15.66:0/2190435904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0009980 con 0x55bcd03bc930 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed599a700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4ed619b700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c8c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a2f70 tx=0x7f4ecc09ab30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47f7d44b1f470f19 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/2190435904 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0a34a0 con 0x55bcd03bc930 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/2190435904 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4ecc09b340 con 0x55bcd03bc930 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcd02674f0 con 0x55bcd03bc930 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/2190435904 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc09b500 con 0x55bcd03bc930 2023-07-21T22:33:33.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eebcd0b80 10 client.27167 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.519+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bcd0124c60 con 0x55bcd03bc930 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/2190435904 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ecc0b7080 con 0x55bcd03bc930 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/2190435904 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4ecc0a3010 con 0x55bcd03bc930 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eebcd0b80 10 client.27167 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eebcd0b80 10 client.27167 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eebcd0b80 20 client.27167 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:33.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd03d61a0 0x55bcd03d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eebcd0b80 4 client.27167 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcd0306eb0 con 0x55bcd03d61a0 2023-07-21T22:33:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.520+0000 7f4ed599a700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd03d61a0 0x55bcd03d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.521+0000 7f4ed599a700 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd03d61a0 0x55bcd03d85b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=802d950ce074ca22 server_cookie=504ad52680b5a996 in_seq=0 out_seq=0 2023-07-21T22:33:33.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.521+0000 7f4eb9ffb700 10 client.27167 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:33.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/2190435904 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bcd0306eb0 con 0x55bcd03d61a0 2023-07-21T22:33:33.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eb9ffb700 10 client.27167 handle_command_reply: tid=0 2023-07-21T22:33:33.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 client.27167 shutdown 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 msgr2=0x55bcd03c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c8c10 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0a2f70 tx=0x7f4ecc09ab30 comp rx=0 tx=0).stop 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 shutdown_connections 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd03d61a0 0x55bcd03d85b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c8c10 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/2190435904 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eb9ffb700 0 client.27167 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.522+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 >> 172.21.15.66:0/2190435904 conn(0x55bcd03b1ef0 msgr2=0x55bcd03b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.523+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 shutdown_connections 2023-07-21T22:33:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.523+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/2190435904 wait complete. 2023-07-21T22:33:33.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.523+0000 7f4eebcd0b80 20 client.27167 trim_cache size 0 max 16384 2023-07-21T22:33:33.577 INFO:tasks.workunit.client.0.smithi066.stdout: 1 587963 5.43 MB/sec execute 540 sec latency 28.447 ms 2023-07-21T22:33:33.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.538+0000 7f4eebcd0b80 1 Processor -- start 2023-07-21T22:33:33.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.538+0000 7f4eebcd0b80 1 -- start start 2023-07-21T22:33:33.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03bc930 0x55bcd03cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd03b8ca0 con 0x55bcd03bc930 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd03c0e00 con 0x55bcd03c1ac0 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd03b8320 con 0x55bcd03c11b0 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4ed699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03bc930 0x55bcd03cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4ed599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03bc930 0x55bcd03cd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35508/0 (socket says 172.21.15.66:35508) 2023-07-21T22:33:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4ed619b700 1 -- 172.21.15.66:0/3126428970 learned_addr learned my addr 172.21.15.66:0/3126428970 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.539+0000 7f4ed699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41188/0 (socket says 172.21.15.66:41188) 2023-07-21T22:33:33.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ed599a700 1 -- 172.21.15.66:0/3126428970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03c1ac0 msgr2=0x55bcd03bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ed599a700 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.588 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ed599a700 1 -- 172.21.15.66:0/3126428970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03bc930 msgr2=0x55bcd03cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ed599a700 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03bc930 0x55bcd03cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ed599a700 1 -- 172.21.15.66:0/3126428970 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcd03dd1e0 con 0x55bcd03c11b0 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ed599a700 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c87b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55bcd03bd5b0 tx=0x7f4ec0011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa18b5164fd15606 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ec53ff700 1 -- 172.21.15.66:0/3126428970 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec002a2b0 con 0x55bcd03c11b0 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ec53ff700 1 -- 172.21.15.66:0/3126428970 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec002a410 con 0x55bcd03c11b0 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.540+0000 7f4ec53ff700 1 -- 172.21.15.66:0/3126428970 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec002ac80 con 0x55bcd03c11b0 2023-07-21T22:33:33.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/3126428970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 msgr2=0x55bcd03c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c87b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55bcd03bd5b0 tx=0x7f4ec0011720 comp rx=0 tx=0).stop 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/3126428970 shutdown_connections 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03c1ac0 0x55bcd03bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03bc930 0x55bcd03cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/3126428970 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c87b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/3126428970 >> 172.21.15.66:0/3126428970 conn(0x55bcd03b1ef0 msgr2=0x55bcd03ba9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/3126428970 shutdown_connections 2023-07-21T22:33:33.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.541+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/3126428970 wait complete. 2023-07-21T22:33:33.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.542+0000 7f4eebcd0b80 1 -- start start 2023-07-21T22:33:33.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.542+0000 7f4eebcd0b80 1 Processor -- start 2023-07-21T22:33:33.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.593 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd03c4bd0 con 0x55bcd03c1ac0 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd03c4d10 con 0x55bcd03bc930 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcd03c4ed0 con 0x55bcd03c11b0 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4eebcd0b80 10 client.0 _ll_register_callbacks cb 0x7f4ecc09ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41194/0 (socket says 172.21.15.66:41194) 2023-07-21T22:33:33.594 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 -- 172.21.15.66:0/1996987804 learned_addr learned my addr 172.21.15.66:0/1996987804 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed699c700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed599a700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 -- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 msgr2=0x55bcd03c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 -- 172.21.15.66:0/1996987804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 msgr2=0x55bcd03c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.595 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.543+0000 7f4ed619b700 1 -- 172.21.15.66:0/1996987804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bcd013bde0 con 0x55bcd03bc930 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4ed619b700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c5860 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0975e0 tx=0x7f4ecc005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=786b595e3dc4f495 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eb9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/1996987804 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc09a7d0 con 0x55bcd03bc930 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/1996987804 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4ecc0a34a0 con 0x55bcd03bc930 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec0019c50 con 0x55bcd03bc930 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/1996987804 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc096f20 con 0x55bcd03bc930 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eebcd0b80 10 client.27173 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:33.596 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bcd0162250 con 0x55bcd03bc930 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/1996987804 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ecc07d070 con 0x55bcd03bc930 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/1996987804 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f4ecc0b9170 con 0x55bcd03bc930 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eebcd0b80 10 client.27173 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eebcd0b80 10 client.27173 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.544+0000 7f4eebcd0b80 20 client.27173 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd042aa60 0x55bcd042ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eebcd0b80 4 client.27173 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:33:33.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcd02c9050 con 0x55bcd042aa60 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4ed599a700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd042aa60 0x55bcd042ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4ed599a700 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd042aa60 0x55bcd042ce40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d29c90eeff6f9a9c server_cookie=6344bc99de62bef0 in_seq=0 out_seq=0 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eb9ffb700 10 client.27173 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eb9ffb700 1 -- 172.21.15.66:0/1996987804 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcd02c9050 con 0x55bcd042aa60 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eb9ffb700 10 client.27173 handle_command_reply: tid=0 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.545+0000 7f4eebcd0b80 1 client.27173 shutdown 2023-07-21T22:33:33.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 msgr2=0x55bcd03c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c5860 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0975e0 tx=0x7f4ecc005e80 comp rx=0 tx=0).stop 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 shutdown_connections 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55bcd042aa60 0x55bcd042ce40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bcd03bc930 0x55bcd03c5860 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eb9ffb700 0 client.27173 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55bcd03c1ac0 0x55bcd03c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 --2- 172.21.15.66:0/1996987804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bcd03c11b0 0x55bcd03c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 >> 172.21.15.66:0/1996987804 conn(0x55bcd03b1ef0 msgr2=0x55bcd03bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 shutdown_connections 2023-07-21T22:33:33.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 1 -- 172.21.15.66:0/1996987804 wait complete. 2023-07-21T22:33:33.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.546+0000 7f4eebcd0b80 20 client.27173 trim_cache size 0 max 16384 2023-07-21T22:33:33.600 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:33:33.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4ebc0a1340 msgr2=0x7f4ebc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4ebc0a1340 0x7f4ebc0a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009510 tx=0x7f4ec8009220 comp rx=0 tx=0).stop 2023-07-21T22:33:33.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 msgr2=0x7f4ed0197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 0x7f4ed0197e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005910 tx=0x7f4ecc004710 comp rx=0 tx=0).stop 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 shutdown_connections 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ed00728b0 0x7f4ed0197e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4ebc0a1340 0x7f4ebc0a37f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4ed010c320 0x7f4ed019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 --2- 172.21.15.66:0/1236526600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4ed00731c0 0x7f4ed0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 >> 172.21.15.66:0/1236526600 conn(0x7f4ed00fb5e0 msgr2=0x7f4ed010e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 shutdown_connections 2023-07-21T22:33:33.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.547+0000 7f4ed83ff700 1 -- 172.21.15.66:0/1236526600 wait complete. 2023-07-21T22:33:33.613 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:33:33.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.800+0000 7f2dba7ff700 1 -- 172.21.15.66:0/3681720916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 msgr2=0x7f2db40fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.800+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/3681720916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 0x7f2db40fff50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f2da80098f0 tx=0x7f2da8009c00 comp rx=0 tx=0).stop 2023-07-21T22:33:33.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.800+0000 7f2dba7ff700 1 -- 172.21.15.66:0/3681720916 shutdown_connections 2023-07-21T22:33:33.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.800+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/3681720916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4105af0 0x7f2db4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.800+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/3681720916 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db4100490 0x7f2db410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.801+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/3681720916 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 0x7f2db40fff50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.801+0000 7f2dba7ff700 1 -- 172.21.15.66:0/3681720916 >> 172.21.15.66:0/3681720916 conn(0x7f2db40fb5e0 msgr2=0x7f2db40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.801+0000 7f2dba7ff700 1 -- 172.21.15.66:0/3681720916 shutdown_connections 2023-07-21T22:33:33.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.801+0000 7f2dba7ff700 1 -- 172.21.15.66:0/3681720916 wait complete. 2023-07-21T22:33:33.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.802+0000 7f2dba7ff700 1 Processor -- start 2023-07-21T22:33:33.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.802+0000 7f2dba7ff700 1 -- start start 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2dba7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 0x7f2db4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2dba7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 0x7f2db4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2dba7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db4105af0 0x7f2db41a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2dba7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db4073580 con 0x7f2db40ffae0 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2dba7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41a4400 con 0x7f2db4100490 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2dba7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db41a4540 con 0x7f2db4105af0 2023-07-21T22:33:33.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 0x7f2db4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 0x7f2db4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 0x7f2db4072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41208/0 (socket says 172.21.15.66:41208) 2023-07-21T22:33:33.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2db37fe700 1 -- 172.21.15.66:0/4078305520 learned_addr learned my addr 172.21.15.66:0/4078305520 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.803+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db4105af0 0x7f2db41a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db37fe700 1 -- 172.21.15.66:0/4078305520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db4105af0 msgr2=0x7f2db41a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db37fe700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db4105af0 0x7f2db41a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db37fe700 1 -- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 msgr2=0x7f2db4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db37fe700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 0x7f2db4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db37fe700 1 -- 172.21.15.66:0/4078305520 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac009c40 con 0x7f2db4100490 2023-07-21T22:33:33.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db37fe700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 0x7f2db4072d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009810 tx=0x7f2da800bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db17fa700 1 -- 172.21.15.66:0/4078305520 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da801c070 con 0x7f2db4100490 2023-07-21T22:33:33.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db17fa700 1 -- 172.21.15.66:0/4078305520 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da8005040 con 0x7f2db4100490 2023-07-21T22:33:33.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da8009520 con 0x7f2db4100490 2023-07-21T22:33:33.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2db17fa700 1 -- 172.21.15.66:0/4078305520 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da80049c0 con 0x7f2db4100490 2023-07-21T22:33:33.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.804+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db41a49d0 con 0x7f2db4100490 2023-07-21T22:33:33.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.809+0000 7f2db17fa700 1 -- 172.21.15.66:0/4078305520 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2da8020070 con 0x7f2db4100490 2023-07-21T22:33:33.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.810+0000 7f2dce0c2b80 1 Processor -- start 2023-07-21T22:33:33.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.810+0000 7f2dce0c2b80 1 -- start start 2023-07-21T22:33:33.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.810+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.810+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc6f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcb74cb0 con 0x55d5bcc6f930 2023-07-21T22:33:33.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcbc7a40 con 0x55d5bcc741b0 2023-07-21T22:33:33.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcbc0ed0 con 0x55d5bcc74ac0 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc6f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc6f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50378/0 (socket says 172.21.15.66:50378) 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 -- 172.21.15.66:0/2640369978 learned_addr learned my addr 172.21.15.66:0/2640369978 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db37fe700 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 -- 172.21.15.66:0/2640369978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 msgr2=0x55d5bcc74580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.859 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc74580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 -- 172.21.15.66:0/2640369978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 msgr2=0x55d5bcc6fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc6fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.811+0000 7f2db3fff700 1 -- 172.21.15.66:0/2640369978 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5bc9eede0 con 0x55d5bcc74ac0 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2db17fa700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f2d9c0a12c0 0x7f2d9c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2db3fff700 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc6f350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009ce0 tx=0x7f2da4011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df5d18f134b72c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2db17fa700 1 -- 172.21.15.66:0/4078305520 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2da80cdd50 con 0x7f2db4100490 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f2d9c0a12c0 0x7f2d9c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.860 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2d9a7fc700 1 -- 172.21.15.66:0/2640369978 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4011e30 con 0x55d5bcc74ac0 2023-07-21T22:33:33.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2d9a7fc700 1 -- 172.21.15.66:0/2640369978 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da4004510 con 0x55d5bcc74ac0 2023-07-21T22:33:33.861 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.812+0000 7f2d9a7fc700 1 -- 172.21.15.66:0/2640369978 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4005260 con 0x55d5bcc74ac0 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2640369978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 msgr2=0x55d5bcc6f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc6f350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009ce0 tx=0x7f2da4011440 comp rx=0 tx=0).stop 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f2d9c0a12c0 0x7f2d9c0a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2db4074920 tx=0x7f2dac00b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2640369978 shutdown_connections 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc74580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc6fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.863 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2640369978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc6f350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2640369978 >> 172.21.15.66:0/2640369978 conn(0x55d5bcc64ef0 msgr2=0x55d5bcc6a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2640369978 shutdown_connections 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.813+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2640369978 wait complete. 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.814+0000 7f2dce0c2b80 1 -- start start 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.814+0000 7f2dce0c2b80 1 Processor -- start 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.864 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcb60ac0 con 0x55d5bcc741b0 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcb607c0 con 0x55d5bcc6f930 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bca44920 con 0x55d5bcc74ac0 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2dce0c2b80 10 client.0 _ll_register_callbacks cb 0x55d5bcb5e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41222/0 (socket says 172.21.15.66:41222) 2023-07-21T22:33:33.865 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db3fff700 1 -- 172.21.15.66:0/3594816390 learned_addr learned my addr 172.21.15.66:0/3594816390 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db37fe700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db3fff700 1 -- 172.21.15.66:0/3594816390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 msgr2=0x55d5bcc79580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db3fff700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc79580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.815+0000 7f2db3fff700 1 -- 172.21.15.66:0/3594816390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc741b0 msgr2=0x55d5bcc78e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2db3fff700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2db3fff700 1 -- 172.21.15.66:0/3594816390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da4009940 con 0x55d5bcc6f930 2023-07-21T22:33:33.866 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2db37fe700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2db3fff700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7bc30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004de0 tx=0x7f2da4004c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff0946a777e1ac2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2d937fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2d937fe700 1 -- 172.21.15.66:0/3594816390 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4003890 con 0x55d5bcc6f930 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5bcb1a4f0 con 0x55d5bcc6f930 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.816+0000 7f2d937fe700 1 -- 172.21.15.66:0/3594816390 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2da40039f0 con 0x55d5bcc6f930 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2d937fe700 1 -- 172.21.15.66:0/3594816390 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2da4003bb0 con 0x55d5bcc6f930 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 10 client.27185 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:33.867 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d5bc9d7c60 con 0x55d5bcc6f930 2023-07-21T22:33:33.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2d937fe700 1 -- 172.21.15.66:0/3594816390 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5bcb1a4f0 con 0x55d5bcc6f930 2023-07-21T22:33:33.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2d937fe700 1 -- 172.21.15.66:0/3594816390 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2da40225c0 con 0x55d5bcc6f930 2023-07-21T22:33:33.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 10 client.27185 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:33.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 10 client.27185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:33:33.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 20 client.27185 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:33.869 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc894b0 0x55d5bcc8b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 4 client.27185 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.817+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5bcbb9eb0 con 0x55d5bcc894b0 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.818+0000 7f2db37fe700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc894b0 0x55d5bcc8b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.818+0000 7f2db37fe700 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc894b0 0x55d5bcc8b8c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1105c2d8de9b70f server_cookie=6e633a762f9b0195 in_seq=0 out_seq=0 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.818+0000 7f2d937fe700 10 client.27185 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2d937fe700 1 -- 172.21.15.66:0/3594816390 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d5bcbb9eb0 con 0x55d5bcc894b0 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2d937fe700 10 client.27185 handle_command_reply: tid=0 2023-07-21T22:33:33.870 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 client.27185 shutdown 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 msgr2=0x55d5bcc7bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7bc30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004de0 tx=0x7f2da4004c00 comp rx=0 tx=0).stop 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 shutdown_connections 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc894b0 0x55d5bcc8b8c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7bc30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc74ac0 0x55d5bcc79580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2d937fe700 0 client.27185 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:33.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3594816390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 >> 172.21.15.66:0/3594816390 conn(0x55d5bcc64ef0 msgr2=0x55d5bcc6a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 shutdown_connections 2023-07-21T22:33:33.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.819+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3594816390 wait complete. 2023-07-21T22:33:33.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.820+0000 7f2dce0c2b80 20 client.27185 trim_cache size 0 max 16384 2023-07-21T22:33:33.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.835+0000 7f2dce0c2b80 1 Processor -- start 2023-07-21T22:33:33.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 -- start start 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc706f0 con 0x55d5bcc6f930 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc67010 con 0x55d5bcc64ef0 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc695e0 con 0x55d5bcc741b0 2023-07-21T22:33:33.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2db8d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2db8d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50402/0 (socket says 172.21.15.66:50402) 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc786e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35546/0 (socket says 172.21.15.66:35546) 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.836+0000 7f2db8d9c700 1 -- 172.21.15.66:0/2858558990 learned_addr learned my addr 172.21.15.66:0/2858558990 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2db3fff700 1 -- 172.21.15.66:0/2858558990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc741b0 msgr2=0x55d5bcc78c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2db3fff700 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2db3fff700 1 -- 172.21.15.66:0/2858558990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 msgr2=0x55d5bcc786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2db3fff700 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2db3fff700 1 -- 172.21.15.66:0/2858558990 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5bca15250 con 0x55d5bcc64ef0 2023-07-21T22:33:33.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2db3fff700 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80570 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004020 tx=0x7f2da4011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479ff6068f23c172 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2d9affd700 1 -- 172.21.15.66:0/2858558990 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4022b20 con 0x55d5bcc64ef0 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2d9affd700 1 -- 172.21.15.66:0/2858558990 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da402acb0 con 0x55d5bcc64ef0 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.837+0000 7f2d9affd700 1 -- 172.21.15.66:0/2858558990 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4003d90 con 0x55d5bcc64ef0 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2858558990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc64ef0 msgr2=0x55d5bcc80570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80570 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2da4004020 tx=0x7f2da4011440 comp rx=0 tx=0).stop 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2858558990 shutdown_connections 2023-07-21T22:33:33.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80570 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc741b0 0x55d5bcc78c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/2858558990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc6f930 0x55d5bcc786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.838+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2858558990 >> 172.21.15.66:0/2858558990 conn(0x55d5bcc70060 msgr2=0x55d5bcc84d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.839+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2858558990 shutdown_connections 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.839+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/2858558990 wait complete. 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.839+0000 7f2dce0c2b80 1 -- start start 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.839+0000 7f2dce0c2b80 1 Processor -- start 2023-07-21T22:33:33.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc7ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc7b1d0 con 0x55d5bcc64ef0 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc7b340 con 0x55d5bcc741b0 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5bcc7b500 con 0x55d5bcc6f930 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2dce0c2b80 10 client.0 _ll_register_callbacks cb 0x7f2da4009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:33.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2db3fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2db37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50418/0 (socket says 172.21.15.66:50418) 2023-07-21T22:33:33.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2db37fe700 1 -- 172.21.15.66:0/3767434263 learned_addr learned my addr 172.21.15.66:0/3767434263 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:33.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.840+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc7ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db3fff700 1 -- 172.21.15.66:0/3767434263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc6f930 msgr2=0x55d5bcc7a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db3fff700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db3fff700 1 -- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 msgr2=0x55d5bcc7ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:33.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db3fff700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc7ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db3fff700 1 -- 172.21.15.66:0/3767434263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5bc9d7c60 con 0x55d5bcc64ef0 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db8d9c700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc7ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2db3fff700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2da401b040 tx=0x7f2da4022630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1968bfc1f29d251 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.841+0000 7f2d937fe700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.842+0000 7f2d937fe700 1 -- 172.21.15.66:0/3767434263 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4042070 con 0x55d5bcc64ef0 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.842+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2da4004510 con 0x55d5bcc64ef0 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.842+0000 7f2d937fe700 1 -- 172.21.15.66:0/3767434263 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2da4000e40 con 0x55d5bcc64ef0 2023-07-21T22:33:33.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.842+0000 7f2d937fe700 1 -- 172.21.15.66:0/3767434263 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2da4009310 con 0x55d5bcc64ef0 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.842+0000 7f2dce0c2b80 10 client.17130 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.842+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55d5bc9ed2b0 con 0x55d5bcc64ef0 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2d937fe700 1 -- 172.21.15.66:0/3767434263 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2da4004510 con 0x55d5bcc64ef0 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2d937fe700 1 -- 172.21.15.66:0/3767434263 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2da4045030 con 0x55d5bcc64ef0 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2dce0c2b80 10 client.17130 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2dce0c2b80 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2dce0c2b80 20 client.17130 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:33.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc74ac0 0x55d5bcce0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2dce0c2b80 4 client.17130 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d5bcb7c050 con 0x55d5bcc74ac0 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.843+0000 7f2db37fe700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc74ac0 0x55d5bcce0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2d937fe700 10 client.17130 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2db37fe700 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc74ac0 0x55d5bcce0270 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=225f7a5ccb375acb server_cookie=8cf1d8e8ffea4a78 in_seq=0 out_seq=0 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2d937fe700 1 -- 172.21.15.66:0/3767434263 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d5bcb7c050 con 0x55d5bcc74ac0 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2d937fe700 10 client.17130 handle_command_reply: tid=0 2023-07-21T22:33:33.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 client.17130 shutdown 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc64ef0 msgr2=0x55d5bcc80580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2da401b040 tx=0x7f2da4022630 comp rx=0 tx=0).stop 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 shutdown_connections 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55d5bcc74ac0 0x55d5bcce0270 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5bcc741b0 0x55d5bcc7ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d5bcc64ef0 0x55d5bcc80580 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2d937fe700 0 client.17130 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:33:33.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 --2- 172.21.15.66:0/3767434263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d5bcc6f930 0x55d5bcc7a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.844+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 >> 172.21.15.66:0/3767434263 conn(0x55d5bcc70060 msgr2=0x55d5bcc786e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 shutdown_connections 2023-07-21T22:33:33.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dce0c2b80 1 -- 172.21.15.66:0/3767434263 wait complete. 2023-07-21T22:33:33.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dce0c2b80 20 client.17130 trim_cache size 0 max 16384 2023-07-21T22:33:33.897 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:33:33.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f2d9c0a12c0 msgr2=0x7f2d9c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f2d9c0a12c0 0x7f2d9c0a3770 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2db4074920 tx=0x7f2dac00b040 comp rx=0 tx=0).stop 2023-07-21T22:33:33.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 msgr2=0x7f2db4072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:33.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.845+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 0x7f2db4072d50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2da8009810 tx=0x7f2da800bfc0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 shutdown_connections 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f2d9c0a12c0 0x7f2d9c0a3770 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2db4105af0 0x7f2db41a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f2db40ffae0 0x7f2db4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 --2- 172.21.15.66:0/4078305520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2db4100490 0x7f2db4072d50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 >> 172.21.15.66:0/4078305520 conn(0x7f2db40fb5e0 msgr2=0x7f2db40fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 shutdown_connections 2023-07-21T22:33:33.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:33.846+0000 7f2dba7ff700 1 -- 172.21.15.66:0/4078305520 wait complete. 2023-07-21T22:33:33.914 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:33:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 -- 172.21.15.66:0/3808982580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b41049e0 msgr2=0x7fa4b4104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/3808982580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b41049e0 0x7fa4b4104db0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009990 tx=0x7fa4a4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 -- 172.21.15.66:0/3808982580 shutdown_connections 2023-07-21T22:33:34.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/3808982580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b41049e0 0x7fa4b4104db0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/3808982580 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b40ff380 0x7fa4b410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/3808982580 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40fe9d0 0x7fa4b40fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.118+0000 7fa4b99ff700 1 -- 172.21.15.66:0/3808982580 >> 172.21.15.66:0/3808982580 conn(0x7fa4b40fa4b0 msgr2=0x7fa4b40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.119+0000 7fa4b99ff700 1 -- 172.21.15.66:0/3808982580 shutdown_connections 2023-07-21T22:33:34.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.119+0000 7fa4b99ff700 1 -- 172.21.15.66:0/3808982580 wait complete. 2023-07-21T22:33:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.120+0000 7fa4b99ff700 1 Processor -- start 2023-07-21T22:33:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.120+0000 7fa4b99ff700 1 -- start start 2023-07-21T22:33:34.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.120+0000 7fa4b99ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b40fe9d0 0x7fa4b4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.120+0000 7fa4b99ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40ff380 0x7fa4b41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b99ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 0x7fa4b419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b99ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b419ce80 con 0x7fa4b40ff380 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b99ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b419cff0 con 0x7fa4b40fe9d0 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b99ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b419d1b0 con 0x7fa4b41049e0 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b40fe9d0 0x7fa4b4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40ff380 0x7fa4b41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 0x7fa4b419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 0x7fa4b419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50448/0 (socket says 172.21.15.66:50448) 2023-07-21T22:33:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2806614394 learned_addr learned my addr 172.21.15.66:0/2806614394 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:34.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b40fe9d0 0x7fa4b4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41250/0 (socket says 172.21.15.66:41250) 2023-07-21T22:33:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2806614394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b40fe9d0 msgr2=0x7fa4b4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b40fe9d0 0x7fa4b4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40ff380 msgr2=0x7fa4b41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40ff380 0x7fa4b41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2806614394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a800b730 con 0x7fa4b41049e0 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.121+0000 7fa4b27fc700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40ff380 0x7fa4b41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.122+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 0x7fa4b419c840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00ef30 tx=0x7fa4ac00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.122+0000 7fa4b89fd700 1 -- 172.21.15.66:0/2806614394 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac018070 con 0x7fa4b41049e0 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.122+0000 7fa4b89fd700 1 -- 172.21.15.66:0/2806614394 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4ac00f040 con 0x7fa4b41049e0 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.122+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a40095f0 con 0x7fa4b41049e0 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.122+0000 7fa4b89fd700 1 -- 172.21.15.66:0/2806614394 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00cd00 con 0x7fa4b41049e0 2023-07-21T22:33:34.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.122+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b41a4af0 con 0x7fa4b41049e0 2023-07-21T22:33:34.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.126+0000 7fa4b89fd700 1 -- 172.21.15.66:0/2806614394 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa4ac003a00 con 0x7fa4b41049e0 2023-07-21T22:33:34.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.127+0000 7fa4cd335b80 1 Processor -- start 2023-07-21T22:33:34.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 -- start start 2023-07-21T22:33:34.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4b89fd700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa49c0a1340 0x7fa49c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4b89fd700 1 -- 172.21.15.66:0/2806614394 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa4ac0ccad0 con 0x7fa4b41049e0 2023-07-21T22:33:34.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4b27fc700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa49c0a1340 0x7fa49c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd881b5cb0 con 0x55fd882b0930 2023-07-21T22:33:34.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd88208a40 con 0x55fd882b51b0 2023-07-21T22:33:34.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.128+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd88201ed0 con 0x55fd882b5ac0 2023-07-21T22:33:34.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35598/0 (socket says 172.21.15.66:35598) 2023-07-21T22:33:34.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b27fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 -- 172.21.15.66:0/3680108927 learned_addr learned my addr 172.21.15.66:0/3680108927 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:34.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 -- 172.21.15.66:0/3680108927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 msgr2=0x55fd882b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 -- 172.21.15.66:0/3680108927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 msgr2=0x55fd882b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 -- 172.21.15.66:0/3680108927 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd8802fde0 con 0x55fd882b0930 2023-07-21T22:33:34.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b27fc700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa49c0a1340 0x7fa49c0a37f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4b41994b0 tx=0x7fa4a800a8b0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:34.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.129+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882b0da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac096300 tx=0x7fa4ac0994c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ee3e6e0ed31879 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:34.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.130+0000 7fa499ffb700 1 -- 172.21.15.66:0/3680108927 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac01b320 con 0x55fd882b0930 2023-07-21T22:33:34.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.130+0000 7fa499ffb700 1 -- 172.21.15.66:0/3680108927 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4ac00ce60 con 0x55fd882b0930 2023-07-21T22:33:34.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.130+0000 7fa499ffb700 1 -- 172.21.15.66:0/3680108927 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b90d0 con 0x55fd882b0930 2023-07-21T22:33:34.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 -- 172.21.15.66:0/3680108927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 msgr2=0x55fd882b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882b0da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac096300 tx=0x7fa4ac0994c0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 -- 172.21.15.66:0/3680108927 shutdown_connections 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882b0da0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/3680108927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 -- 172.21.15.66:0/3680108927 >> 172.21.15.66:0/3680108927 conn(0x55fd882a5ef0 msgr2=0x55fd882ab830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 -- 172.21.15.66:0/3680108927 shutdown_connections 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.131+0000 7fa4cd335b80 1 -- 172.21.15.66:0/3680108927 wait complete. 2023-07-21T22:33:34.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.132+0000 7fa4cd335b80 1 -- start start 2023-07-21T22:33:34.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.132+0000 7fa4cd335b80 1 Processor -- start 2023-07-21T22:33:34.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882ba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd881a1ac0 con 0x55fd882b0930 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd881a17c0 con 0x55fd882b51b0 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd88085920 con 0x55fd882b5ac0 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4cd335b80 10 client.0 _ll_register_callbacks cb 0x55fd8819f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882ba620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50462/0 (socket says 172.21.15.66:50462) 2023-07-21T22:33:34.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2500901878 learned_addr learned my addr 172.21.15.66:0/2500901878 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b27fc700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 msgr2=0x55fd882b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2500901878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 msgr2=0x55fd882bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2500901878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b41a4af0 con 0x55fd882b5ac0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.133+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882ba620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0a1720 tx=0x7fa4ac0092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=550ac588983350c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa492ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa492ffd700 1 -- 172.21.15.66:0/2500901878 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b92e0 con 0x55fd882b5ac0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa492ffd700 1 -- 172.21.15.66:0/2500901878 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa4ac09d8c0 con 0x55fd882b5ac0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd8815b4f0 con 0x55fd882b5ac0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa492ffd700 1 -- 172.21.15.66:0/2500901878 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4ac099890 con 0x55fd882b5ac0 2023-07-21T22:33:34.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa4cd335b80 10 client.27178 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.134+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55fd88018c60 con 0x55fd882b5ac0 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa492ffd700 1 -- 172.21.15.66:0/2500901878 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd8815b4f0 con 0x55fd882b5ac0 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa492ffd700 1 -- 172.21.15.66:0/2500901878 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa4ac098e60 con 0x55fd882b5ac0 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4cd335b80 10 client.27178 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4cd335b80 10 client.27178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4cd335b80 20 client.27178 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd882ca050 0x55fd882cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4cd335b80 4 client.27178 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd881faeb0 con 0x55fd882ca050 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.135+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd882ca050 0x55fd882cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.136+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd882ca050 0x55fd882cc430 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5fe8dfdfe416101 server_cookie=67befd7bb8ed2adc in_seq=0 out_seq=0 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.136+0000 7fa492ffd700 10 client.27178 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.136+0000 7fa492ffd700 1 -- 172.21.15.66:0/2500901878 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd881faeb0 con 0x55fd882ca050 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.136+0000 7fa492ffd700 10 client.27178 handle_command_reply: tid=0 2023-07-21T22:33:34.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 client.27178 shutdown 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 msgr2=0x55fd882ba620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882ba620 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac0a1720 tx=0x7fa4ac0092e0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 shutdown_connections 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd882ca050 0x55fd882cc430 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b5ac0 0x55fd882ba620 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b0930 0x55fd882bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2500901878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b51b0 0x55fd882b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa492ffd700 0 client.27178 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:34.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 >> 172.21.15.66:0/2500901878 conn(0x55fd882a5ef0 msgr2=0x55fd882ab3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:34.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 shutdown_connections 2023-07-21T22:33:34.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2500901878 wait complete. 2023-07-21T22:33:34.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.137+0000 7fa4cd335b80 20 client.27178 trim_cache size 0 max 16384 2023-07-21T22:33:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:34 smithi066 ceph-mon[126379]: pgmap v377: 129 pgs: 129 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 206 op/s 2023-07-21T22:33:34.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:34 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/968285799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:33:34.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 Processor -- start 2023-07-21T22:33:34.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 -- start start 2023-07-21T22:33:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 0x55fd882c1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b51b0 0x55fd882bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b5ac0 0x55fd882aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd882acca0 con 0x55fd882b5ac0 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd882c1ab0 con 0x55fd882b0930 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd882aa320 con 0x55fd882b51b0 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 0x55fd882c1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b5ac0 0x55fd882aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 0x55fd882c1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41272/0 (socket says 172.21.15.66:41272) 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4b2ffd700 1 -- 172.21.15.66:0/4231130871 learned_addr learned my addr 172.21.15.66:0/4231130871 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:34.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.156+0000 7fa4b27fc700 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b51b0 0x55fd882bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b2ffd700 1 -- 172.21.15.66:0/4231130871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b51b0 msgr2=0x55fd882bc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b51b0 0x55fd882bc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b2ffd700 1 -- 172.21.15.66:0/4231130871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b5ac0 msgr2=0x55fd882aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b5ac0 0x55fd882aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b2ffd700 1 -- 172.21.15.66:0/4231130871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd88056250 con 0x55fd882b0930 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b5ac0 0x55fd882aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:34.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 0x55fd882c1570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009990 tx=0x7fa4a4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c739c9bc28f2e130 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa49a7fc700 1 -- 172.21.15.66:0/4231130871 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a401d070 con 0x55fd882b0930 2023-07-21T22:33:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa49a7fc700 1 -- 172.21.15.66:0/4231130871 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4a401e040 con 0x55fd882b0930 2023-07-21T22:33:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.157+0000 7fa49a7fc700 1 -- 172.21.15.66:0/4231130871 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400f6c0 con 0x55fd882b0930 2023-07-21T22:33:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 -- 172.21.15.66:0/4231130871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 msgr2=0x55fd882c1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 0x55fd882c1570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009990 tx=0x7fa4a4009310 comp rx=0 tx=0).stop 2023-07-21T22:33:34.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 -- 172.21.15.66:0/4231130871 shutdown_connections 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b0930 0x55fd882c1570 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b5ac0 0x55fd882aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/4231130871 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b51b0 0x55fd882bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 -- 172.21.15.66:0/4231130871 >> 172.21.15.66:0/4231130871 conn(0x55fd882a5ef0 msgr2=0x55fd882d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 -- 172.21.15.66:0/4231130871 shutdown_connections 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.158+0000 7fa4cd335b80 1 -- 172.21.15.66:0/4231130871 wait complete. 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 -- start start 2023-07-21T22:33:34.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 Processor -- start 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 0x55fd882c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 0x55fd882bf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b5ac0 0x55fd882bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd882bffe0 con 0x55fd882b51b0 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd882c0150 con 0x55fd882b5ac0 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd882c0310 con 0x55fd882b0930 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.159+0000 7fa4cd335b80 10 client.0 _ll_register_callbacks cb 0x7fa4a8005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:34.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 0x55fd882bf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 0x55fd882c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b5ac0 0x55fd882bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b2ffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 0x55fd882c1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50476/0 (socket says 172.21.15.66:50476) 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b2ffd700 1 -- 172.21.15.66:0/2450163731 learned_addr learned my addr 172.21.15.66:0/2450163731 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b27fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 0x55fd882bf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35606/0 (socket says 172.21.15.66:35606) 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2450163731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 msgr2=0x55fd882c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 0x55fd882c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2450163731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 msgr2=0x55fd882bf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 0x55fd882bf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 -- 172.21.15.66:0/2450163731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd88018c60 con 0x55fd882b5ac0 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b27fc700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 0x55fd882bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 0x55fd882c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa4b37fe700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b5ac0 0x55fd882bfaa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c6f0 tx=0x7fa4ac0c9f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66c3067800b0f857 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.160+0000 7fa492ffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:34.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa492ffd700 1 -- 172.21.15.66:0/2450163731 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac099cf0 con 0x55fd882b5ac0 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa492ffd700 1 -- 172.21.15.66:0/2450163731 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa4ac09d8c0 con 0x55fd882b5ac0 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4ac00ce60 con 0x55fd882b5ac0 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa492ffd700 1 -- 172.21.15.66:0/2450163731 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4ac0993d0 con 0x55fd882b5ac0 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 10 client.27209 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fa4b410c110 con 0x55fd882b5ac0 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa492ffd700 1 -- 172.21.15.66:0/2450163731 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4ac00ce60 con 0x55fd882b5ac0 2023-07-21T22:33:34.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa492ffd700 1 -- 172.21.15.66:0/2450163731 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fa4ac098d80 con 0x55fd882b5ac0 2023-07-21T22:33:34.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 10 client.27209 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:34.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 10 client.27209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:34.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 20 client.27209 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:34.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd8831ee00 0x55fd88321210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:34.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 4 client.27209 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:33:34.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.161+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd881bd050 con 0x55fd8831ee00 2023-07-21T22:33:34.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.162+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd8831ee00 0x55fd88321210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:34.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.162+0000 7fa4b2ffd700 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd8831ee00 0x55fd88321210 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5246f6961c06c6 server_cookie=6aba513c316faa1 in_seq=0 out_seq=0 2023-07-21T22:33:34.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.162+0000 7fa492ffd700 10 client.27209 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:34.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.162+0000 7fa492ffd700 1 -- 172.21.15.66:0/2450163731 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd881bd050 con 0x55fd8831ee00 2023-07-21T22:33:34.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.162+0000 7fa492ffd700 10 client.27209 handle_command_reply: tid=0 2023-07-21T22:33:34.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.162+0000 7fa4cd335b80 1 client.27209 shutdown 2023-07-21T22:33:34.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b5ac0 msgr2=0x55fd882bfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b5ac0 0x55fd882bfaa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00c6f0 tx=0x7fa4ac0c9f30 comp rx=0 tx=0).stop 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 shutdown_connections 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fd882b5ac0 0x55fd882bfaa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55fd8831ee00 0x55fd88321210 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fd882b0930 0x55fd882c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 --2- 172.21.15.66:0/2450163731 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55fd882b51b0 0x55fd882bf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa492ffd700 0 client.27209 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:34.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 >> 172.21.15.66:0/2450163731 conn(0x55fd882a5ef0 msgr2=0x55fd882aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:34.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 shutdown_connections 2023-07-21T22:33:34.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 1 -- 172.21.15.66:0/2450163731 wait complete. 2023-07-21T22:33:34.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.163+0000 7fa4cd335b80 20 client.27209 trim_cache size 0 max 16384 2023-07-21T22:33:34.218 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:33:34.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa49c0a1340 msgr2=0x7fa49c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa49c0a1340 0x7fa49c0a37f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa4b41994b0 tx=0x7fa4a800a8b0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 msgr2=0x7fa4b419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:34.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 0x7fa4b419c840 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00ef30 tx=0x7fa4ac00c3a0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 shutdown_connections 2023-07-21T22:33:34.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4b40fe9d0 0x7fa4b4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fa49c0a1340 0x7fa49c0a37f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4b41049e0 0x7fa4b419c840 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 --2- 172.21.15.66:0/2806614394 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fa4b40ff380 0x7fa4b41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:34.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 >> 172.21.15.66:0/2806614394 conn(0x7fa4b40fa4b0 msgr2=0x7fa4b40fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:34.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 shutdown_connections 2023-07-21T22:33:34.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:34.164+0000 7fa4b99ff700 1 -- 172.21.15.66:0/2806614394 wait complete. 2023-07-21T22:33:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:34 smithi104 ceph-mon[129021]: pgmap v377: 129 pgs: 129 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 206 op/s 2023-07-21T22:33:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:34 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/968285799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:33:34.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:34 smithi149 ceph-mon[129268]: pgmap v377: 129 pgs: 129 active+clean; 587 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 206 op/s 2023-07-21T22:33:34.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:34 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/968285799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:33:34.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 588950 5.43 MB/sec execute 541 sec latency 482.088 ms 2023-07-21T22:33:35.231 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:33:35.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:33:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 -- 172.21.15.66:0/459323337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610067670 msgr2=0x7f0610067ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 --2- 172.21.15.66:0/459323337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610067670 0x7f0610067ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009990 tx=0x7f05fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 -- 172.21.15.66:0/459323337 shutdown_connections 2023-07-21T22:33:35.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 --2- 172.21.15.66:0/459323337 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610068020 0x7f061010c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 --2- 172.21.15.66:0/459323337 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0610105b50 0x7f0610105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 --2- 172.21.15.66:0/459323337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610067670 0x7f0610067ae0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 -- 172.21.15.66:0/459323337 >> 172.21.15.66:0/459323337 conn(0x7f0610074b10 msgr2=0x7f0610074f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 -- 172.21.15.66:0/459323337 shutdown_connections 2023-07-21T22:33:35.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.407+0000 7f06171ff700 1 -- 172.21.15.66:0/459323337 wait complete. 2023-07-21T22:33:35.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 Processor -- start 2023-07-21T22:33:35.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 -- start start 2023-07-21T22:33:35.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 0x7f0610197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610068020 0x7f06101982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0610105b50 0x7f061019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061019cd90 con 0x7f0610067670 2023-07-21T22:33:35.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061019cf00 con 0x7f0610068020 2023-07-21T22:33:35.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.408+0000 7f06171ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061019d0c0 con 0x7f0610105b50 2023-07-21T22:33:35.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 0x7f0610197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610068020 0x7f06101982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 0x7f0610197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35626/0 (socket says 172.21.15.66:35626) 2023-07-21T22:33:35.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f0614f9b700 1 -- 172.21.15.66:0/622721959 learned_addr learned my addr 172.21.15.66:0/622721959 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f061579c700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0610105b50 0x7f061019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 -- 172.21.15.66:0/622721959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0610105b50 msgr2=0x7f061019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0610105b50 0x7f061019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 -- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 msgr2=0x7f0610197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 0x7f0610197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 -- 172.21.15.66:0/622721959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604009520 con 0x7f0610068020 2023-07-21T22:33:35.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f0614f9b700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 0x7f0610197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.409+0000 7f060ffff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610068020 0x7f06101982f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098e0 tx=0x7f05fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.410+0000 7f060dffb700 1 -- 172.21.15.66:0/622721959 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc01d070 con 0x7f0610068020 2023-07-21T22:33:35.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.410+0000 7f060dffb700 1 -- 172.21.15.66:0/622721959 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc005030 con 0x7f0610068020 2023-07-21T22:33:35.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.410+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc0095f0 con 0x7f0610068020 2023-07-21T22:33:35.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.410+0000 7f060dffb700 1 -- 172.21.15.66:0/622721959 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc004ab0 con 0x7f0610068020 2023-07-21T22:33:35.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.410+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061019d5d0 con 0x7f0610068020 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.415+0000 7f060dffb700 1 -- 172.21.15.66:0/622721959 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f05fc005410 con 0x7f0610068020 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.416+0000 7f062aa30b80 1 Processor -- start 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.416+0000 7f062aa30b80 1 -- start start 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f060dffb700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f06000a1340 0x7f06000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 0x5650ef205600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205b40 0x5650ef2003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef2009b0 0x5650ef200e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f062aa30b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef105d30 con 0x5650ef2009b0 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f062aa30b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef158ac0 con 0x5650ef205230 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f062aa30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef151f50 con 0x5650ef205b40 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f060dffb700 1 -- 172.21.15.66:0/622721959 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f05fc0ce0b0 con 0x7f0610068020 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 0x5650ef205600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f061579c700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f06000a1340 0x7f06000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f060ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205b40 0x5650ef2003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 0x5650ef205600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41308/0 (socket says 172.21.15.66:41308) 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f0614f9b700 1 -- 172.21.15.66:0/4035572990 learned_addr learned my addr 172.21.15.66:0/4035572990 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f061579c700 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef2009b0 0x5650ef200e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.417+0000 7f060ffff700 1 -- 172.21.15.66:0/4035572990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 msgr2=0x5650ef205600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f060ffff700 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 0x5650ef205600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f060ffff700 1 -- 172.21.15.66:0/4035572990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef2009b0 msgr2=0x5650ef200e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f060ffff700 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef2009b0 0x5650ef200e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f060ffff700 1 -- 172.21.15.66:0/4035572990 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650eef7fe60 con 0x5650ef205b40 2023-07-21T22:33:35.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f061579c700 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef2009b0 0x5650ef200e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:33:35.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f0614f9b700 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 0x5650ef205600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f060ffff700 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205b40 0x5650ef2003d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0cb460 tx=0x7f05fc09a8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b842b71066f92677 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f05faffd700 1 -- 172.21.15.66:0/4035572990 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0b2070 con 0x5650ef205b40 2023-07-21T22:33:35.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f05faffd700 1 -- 172.21.15.66:0/4035572990 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc09eea0 con 0x5650ef205b40 2023-07-21T22:33:35.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.418+0000 7f05faffd700 1 -- 172.21.15.66:0/4035572990 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc09b330 con 0x5650ef205b40 2023-07-21T22:33:35.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f061579c700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f06000a1340 0x7f06000a37f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0608007610 tx=0x7f060800b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f062aa30b80 1 -- 172.21.15.66:0/4035572990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205b40 msgr2=0x5650ef2003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f062aa30b80 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205b40 0x5650ef2003d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0cb460 tx=0x7f05fc09a8a0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f062aa30b80 1 -- 172.21.15.66:0/4035572990 shutdown_connections 2023-07-21T22:33:35.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f062aa30b80 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205230 0x5650ef205600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f062aa30b80 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef2009b0 0x5650ef200e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.419+0000 7f062aa30b80 1 --2- 172.21.15.66:0/4035572990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205b40 0x5650ef2003d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.420+0000 7f062aa30b80 1 -- 172.21.15.66:0/4035572990 >> 172.21.15.66:0/4035572990 conn(0x5650ef1f5f70 msgr2=0x5650ef1fb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.420+0000 7f062aa30b80 1 -- 172.21.15.66:0/4035572990 shutdown_connections 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.420+0000 7f062aa30b80 1 -- 172.21.15.66:0/4035572990 wait complete. 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.420+0000 7f062aa30b80 1 -- start start 2023-07-21T22:33:35.472 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 Processor -- start 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef20cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef20a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef0f1b40 con 0x5650ef205230 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef0f1840 con 0x5650ef205b40 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650eefd59a0 con 0x5650ef2009b0 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.421+0000 7f062aa30b80 10 client.0 _ll_register_callbacks cb 0x5650ef0ef7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:35.473 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef20cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef20cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50506/0 (socket says 172.21.15.66:50506) 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f061579c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef20a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 -- 172.21.15.66:0/2707629960 learned_addr learned my addr 172.21.15.66:0/2707629960 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f060ffff700 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 -- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 msgr2=0x5650ef20a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef20a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 -- 172.21.15.66:0/2707629960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 msgr2=0x5650ef209d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 -- 172.21.15.66:0/2707629960 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0604011440 con 0x5650ef2009b0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f0614f9b700 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef20cb00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0604009770 tx=0x7f0604011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0605495e7244988 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f05effff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f05effff700 1 -- 172.21.15.66:0/2707629960 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604016070 con 0x5650ef2009b0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.422+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650ef0ab570 con 0x5650ef2009b0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f05effff700 1 -- 172.21.15.66:0/2707629960 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f060400fe30 con 0x5650ef2009b0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f05effff700 1 -- 172.21.15.66:0/2707629960 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0604004830 con 0x5650ef2009b0 2023-07-21T22:33:35.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 10 client.27196 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f061019d5d0 con 0x5650ef2009b0 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f05effff700 1 -- 172.21.15.66:0/2707629960 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650ef0ab570 con 0x5650ef2009b0 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f05effff700 1 -- 172.21.15.66:0/2707629960 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0604019550 con 0x5650ef2009b0 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 10 client.27196 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 10 client.27196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 20 client.27196 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:35.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef21a200 0x5650ef21c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 4 client.27196 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.423+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650ef14af30 con 0x5650ef21a200 2023-07-21T22:33:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.424+0000 7f060ffff700 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef21a200 0x5650ef21c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.424+0000 7f060ffff700 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef21a200 0x5650ef21c610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500e6795032c88c9 server_cookie=c0beac24c937c63f in_seq=0 out_seq=0 2023-07-21T22:33:35.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.424+0000 7f05effff700 10 client.27196 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f05effff700 1 -- 172.21.15.66:0/2707629960 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5650ef14af30 con 0x5650ef21a200 2023-07-21T22:33:35.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f05effff700 10 client.27196 handle_command_reply: tid=0 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 client.27196 shutdown 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 msgr2=0x5650ef20cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef20cb00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0604009770 tx=0x7f0604011f90 comp rx=0 tx=0).stop 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 shutdown_connections 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef20a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef21a200 0x5650ef21c610 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef20cb00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 --2- 172.21.15.66:0/2707629960 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f05effff700 0 client.27196 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 >> 172.21.15.66:0/2707629960 conn(0x5650ef1f5f70 msgr2=0x5650ef1fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 shutdown_connections 2023-07-21T22:33:35.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 1 -- 172.21.15.66:0/2707629960 wait complete. 2023-07-21T22:33:35.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.426+0000 7f062aa30b80 20 client.27196 trim_cache size 0 max 16384 2023-07-21T22:33:35.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.444+0000 7f062aa30b80 1 Processor -- start 2023-07-21T22:33:35.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.444+0000 7f062aa30b80 1 -- start start 2023-07-21T22:33:35.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.444+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef2009b0 0x5650ef207960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.444+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 0x5650ef2166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.444+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205b40 0x5650ef211400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.444+0000 7f062aa30b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef0ab570 con 0x5650ef205b40 2023-07-21T22:33:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f062aa30b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef1f8090 con 0x5650ef2009b0 2023-07-21T22:33:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f062aa30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef201750 con 0x5650ef205230 2023-07-21T22:33:35.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 0x5650ef2166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef2009b0 0x5650ef207960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f061579c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205b40 0x5650ef211400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 0x5650ef2166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50522/0 (socket says 172.21.15.66:50522) 2023-07-21T22:33:35.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 -- 172.21.15.66:0/1620645165 learned_addr learned my addr 172.21.15.66:0/1620645165 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 -- 172.21.15.66:0/1620645165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef2009b0 msgr2=0x5650ef207960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef2009b0 0x5650ef207960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 -- 172.21.15.66:0/1620645165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205b40 msgr2=0x5650ef211400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205b40 0x5650ef211400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 -- 172.21.15.66:0/1620645165 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650eef68ce0 con 0x5650ef205230 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.445+0000 7f060ffff700 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 0x5650ef2166f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0ce390 tx=0x7f05fc09ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5b046c91952653 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f05fb7fe700 1 -- 172.21.15.66:0/1620645165 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc0b2040 con 0x5650ef205230 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f05fb7fe700 1 -- 172.21.15.66:0/1620645165 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc0b3040 con 0x5650ef205230 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f05fb7fe700 1 -- 172.21.15.66:0/1620645165 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc09b0d0 con 0x5650ef205230 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 -- 172.21.15.66:0/1620645165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 msgr2=0x5650ef2166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 0x5650ef2166f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0ce390 tx=0x7f05fc09ad70 comp rx=0 tx=0).stop 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 -- 172.21.15.66:0/1620645165 shutdown_connections 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef2009b0 0x5650ef207960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205b40 0x5650ef211400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 --2- 172.21.15.66:0/1620645165 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef205230 0x5650ef2166f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.446+0000 7f062aa30b80 1 -- 172.21.15.66:0/1620645165 >> 172.21.15.66:0/1620645165 conn(0x5650ef1f5f70 msgr2=0x5650ef203e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.447+0000 7f062aa30b80 1 -- 172.21.15.66:0/1620645165 shutdown_connections 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.447+0000 7f062aa30b80 1 -- 172.21.15.66:0/1620645165 wait complete. 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.447+0000 7f062aa30b80 1 -- start start 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.447+0000 7f062aa30b80 1 Processor -- start 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef209050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef26e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef26e600 con 0x5650ef205230 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef26e740 con 0x5650ef205b40 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ef26e880 con 0x5650ef2009b0 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f062aa30b80 10 client.0 _ll_register_callbacks cb 0x7f0604011a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef209050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f0614f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef209050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50524/0 (socket says 172.21.15.66:50524) 2023-07-21T22:33:35.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f0614f9b700 1 -- 172.21.15.66:0/3714218905 learned_addr learned my addr 172.21.15.66:0/3714218905 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f060ffff700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.448+0000 7f061579c700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef26e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f0614f9b700 1 -- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 msgr2=0x5650ef26e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f0614f9b700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef26e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f0614f9b700 1 -- 172.21.15.66:0/3714218905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 msgr2=0x5650ef209590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f0614f9b700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f0614f9b700 1 -- 172.21.15.66:0/3714218905 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650eef7fe60 con 0x5650ef2009b0 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f060ffff700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f0614f9b700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef209050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0604000c00 tx=0x7f0604009f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811438aca8daf74f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f05effff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f05effff700 1 -- 172.21.15.66:0/3714218905 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604016040 con 0x5650ef2009b0 2023-07-21T22:33:35.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05fc09eea0 con 0x5650ef2009b0 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.449+0000 7f05effff700 1 -- 172.21.15.66:0/3714218905 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0604019590 con 0x5650ef2009b0 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f05effff700 1 -- 172.21.15.66:0/3714218905 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0604005040 con 0x5650ef2009b0 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 10 client.27202 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5650eefa62d0 con 0x5650ef2009b0 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f05effff700 1 -- 172.21.15.66:0/3714218905 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05fc09eea0 con 0x5650ef2009b0 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f05effff700 1 -- 172.21.15.66:0/3714218905 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f06040039b0 con 0x5650ef2009b0 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 10 client.27202 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:35.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 10 client.27202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 20 client.27202 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef26fec0 0x5650ef2722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 4 client.27202 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5650ef10d0d0 con 0x5650ef26fec0 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.450+0000 7f060ffff700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef26fec0 0x5650ef2722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.451+0000 7f05effff700 10 client.27202 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.451+0000 7f060ffff700 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef26fec0 0x5650ef2722d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b283fd0fa4a8d948 server_cookie=3099e72d4dc9d0ce in_seq=0 out_seq=0 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f05effff700 1 -- 172.21.15.66:0/3714218905 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5650ef10d0d0 con 0x5650ef26fec0 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f05effff700 10 client.27202 handle_command_reply: tid=0 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 client.27202 shutdown 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 msgr2=0x5650ef209050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef209050 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0604000c00 tx=0x7f0604009f20 comp rx=0 tx=0).stop 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 shutdown_connections 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650ef205b40 0x5650ef26e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.505 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5650ef26fec0 0x5650ef2722d0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650ef2009b0 0x5650ef209050 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 --2- 172.21.15.66:0/3714218905 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5650ef205230 0x5650ef209590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 >> 172.21.15.66:0/3714218905 conn(0x5650ef1f5f70 msgr2=0x5650ef200180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f05effff700 0 client.27202 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 shutdown_connections 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 1 -- 172.21.15.66:0/3714218905 wait complete. 2023-07-21T22:33:35.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.452+0000 7f062aa30b80 20 client.27202 trim_cache size 0 max 16384 2023-07-21T22:33:35.507 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:33:35.507 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:33:35.507 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "002da751-3814-4bee-a120-0de2974fb9d7", 2023-07-21T22:33:35.507 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:33:35.507 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:33:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f06000a1340 msgr2=0x7f06000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f06000a1340 0x7f06000a37f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0608007610 tx=0x7f060800b040 comp rx=0 tx=0).stop 2023-07-21T22:33:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610068020 msgr2=0x7f06101982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610068020 0x7f06101982f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0098e0 tx=0x7f05fc004730 comp rx=0 tx=0).stop 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 shutdown_connections 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f06000a1340 0x7f06000a37f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0610105b50 0x7f061019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0610067670 0x7f0610197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 --2- 172.21.15.66:0/622721959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0610068020 0x7f06101982f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 >> 172.21.15.66:0/622721959 conn(0x7f0610074b10 msgr2=0x7f061010ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 shutdown_connections 2023-07-21T22:33:35.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.453+0000 7f06171ff700 1 -- 172.21.15.66:0/622721959 wait complete. 2023-07-21T22:33:35.520 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:33:35.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 590037 5.44 MB/sec execute 542 sec latency 41.564 ms 2023-07-21T22:33:35.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.694+0000 7f5056bff700 1 -- 172.21.15.66:0/2050036376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 msgr2=0x7f5050101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.694+0000 7f5056bff700 1 --2- 172.21.15.66:0/2050036376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050101070 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5044009990 tx=0x7f5044009ca0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.694+0000 7f5056bff700 1 -- 172.21.15.66:0/2050036376 shutdown_connections 2023-07-21T22:33:35.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.694+0000 7f5056bff700 1 --2- 172.21.15.66:0/2050036376 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 0x7f505010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.694+0000 7f5056bff700 1 --2- 172.21.15.66:0/2050036376 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050101070 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.695+0000 7f5056bff700 1 --2- 172.21.15.66:0/2050036376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 0x7f5050109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.695+0000 7f5056bff700 1 -- 172.21.15.66:0/2050036376 >> 172.21.15.66:0/2050036376 conn(0x7f50500fa4b0 msgr2=0x7f50500fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.695+0000 7f5056bff700 1 -- 172.21.15.66:0/2050036376 shutdown_connections 2023-07-21T22:33:35.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.695+0000 7f5056bff700 1 -- 172.21.15.66:0/2050036376 wait complete. 2023-07-21T22:33:35.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 Processor -- start 2023-07-21T22:33:35.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 -- start start 2023-07-21T22:33:35.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 0x7f5050198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 0x7f505019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505019ce10 con 0x7f5050100ca0 2023-07-21T22:33:35.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505019cf80 con 0x7f50501015b0 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.696+0000 7f5056bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505019d140 con 0x7f505010a070 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35692/0 (socket says 172.21.15.66:35692) 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 -- 172.21.15.66:0/4054036042 learned_addr learned my addr 172.21.15.66:0/4054036042 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505519c700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 0x7f505019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f504ffff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 0x7f5050198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 -- 172.21.15.66:0/4054036042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 msgr2=0x7f505019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 0x7f505019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 -- 172.21.15.66:0/4054036042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 msgr2=0x7f5050198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 0x7f5050198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 -- 172.21.15.66:0/4054036042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c009520 con 0x7f5050100ca0 2023-07-21T22:33:35.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505519c700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 0x7f505019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f504ffff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 0x7f5050198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.697+0000 7f505499b700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050197e30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5044005910 tx=0x7f5044004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.698+0000 7f504dffb700 1 -- 172.21.15.66:0/4054036042 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504401d070 con 0x7f5050100ca0 2023-07-21T22:33:35.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.698+0000 7f504dffb700 1 -- 172.21.15.66:0/4054036042 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5044022470 con 0x7f5050100ca0 2023-07-21T22:33:35.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.698+0000 7f504dffb700 1 -- 172.21.15.66:0/4054036042 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044017650 con 0x7f5050100ca0 2023-07-21T22:33:35.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.698+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440095f0 con 0x7f5050100ca0 2023-07-21T22:33:35.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.698+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50501a4a80 con 0x7f5050100ca0 2023-07-21T22:33:35.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.701+0000 7f504dffb700 1 -- 172.21.15.66:0/4054036042 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f50440225e0 con 0x7f5050100ca0 2023-07-21T22:33:35.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.703+0000 7f506a580b80 1 Processor -- start 2023-07-21T22:33:35.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.703+0000 7f504dffb700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f50400a1410 0x7f50400a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.703+0000 7f506a580b80 1 -- start start 2023-07-21T22:33:35.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.703+0000 7f504dffb700 1 -- 172.21.15.66:0/4054036042 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f50440ce270 con 0x7f5050100ca0 2023-07-21T22:33:35.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f506a580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 0x557415f0b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f506a580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0bac0 0x557415f06350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f506a580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f06da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f506a580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415e0bcb0 con 0x557415f0b1b0 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f506a580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415e5ea40 con 0x557415f0bac0 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f506a580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415e57ed0 con 0x557415f06930 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f505519c700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f50400a1410 0x7f50400a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f505499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 0x557415f0b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f504ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0bac0 0x557415f06350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f505499b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 0x557415f0b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:35708/0 (socket says 172.21.15.66:35708) 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f505499b700 1 -- 172.21.15.66:0/3749235280 learned_addr learned my addr 172.21.15.66:0/3749235280 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f505519c700 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f06da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f504ffff700 1 -- 172.21.15.66:0/3749235280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 msgr2=0x557415f06da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f504ffff700 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f06da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f504ffff700 1 -- 172.21.15.66:0/3749235280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 msgr2=0x557415f0b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f504ffff700 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 0x557415f0b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f504ffff700 1 -- 172.21.15.66:0/3749235280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557415c85de0 con 0x557415f0bac0 2023-07-21T22:33:35.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.704+0000 7f505499b700 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 0x557415f0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:33:35.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.705+0000 7f505519c700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f50400a1410 0x7f50400a38c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5048004230 tx=0x7f50480045f0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.705+0000 7f504ffff700 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0bac0 0x557415f06350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f503c011600 tx=0x7f503c011910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90d431f8731f4bc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.705+0000 7f503affd700 1 -- 172.21.15.66:0/3749235280 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c004020 con 0x557415f0bac0 2023-07-21T22:33:35.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.705+0000 7f503affd700 1 -- 172.21.15.66:0/3749235280 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f503c004830 con 0x557415f0bac0 2023-07-21T22:33:35.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.705+0000 7f503affd700 1 -- 172.21.15.66:0/3749235280 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c0053c0 con 0x557415f0bac0 2023-07-21T22:33:35.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 -- 172.21.15.66:0/3749235280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0bac0 msgr2=0x557415f06350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0bac0 0x557415f06350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f503c011600 tx=0x7f503c011910 comp rx=0 tx=0).stop 2023-07-21T22:33:35.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 -- 172.21.15.66:0/3749235280 shutdown_connections 2023-07-21T22:33:35.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f06da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0b1b0 0x557415f0b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 --2- 172.21.15.66:0/3749235280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0bac0 0x557415f06350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 -- 172.21.15.66:0/3749235280 >> 172.21.15.66:0/3749235280 conn(0x557415efbef0 msgr2=0x557415f01830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 -- 172.21.15.66:0/3749235280 shutdown_connections 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 -- 172.21.15.66:0/3749235280 wait complete. 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.706+0000 7f506a580b80 1 -- start start 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 Processor -- start 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 0x557415f0fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0bac0 0x557415f10480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415df7ac0 con 0x557415f0bac0 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415df77c0 con 0x557415f0b1b0 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415cdb920 con 0x557415f06930 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f506a580b80 10 client.0 _ll_register_callbacks cb 0x557415df5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f504ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 0x557415f0fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f504ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 0x557415f0fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41360/0 (socket says 172.21.15.66:41360) 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f504ffff700 1 -- 172.21.15.66:0/993360763 learned_addr learned my addr 172.21.15.66:0/993360763 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.707+0000 7f505519c700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0bac0 0x557415f10480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f505499b700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f504ffff700 1 -- 172.21.15.66:0/993360763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 msgr2=0x557415f12c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f504ffff700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f12c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f504ffff700 1 -- 172.21.15.66:0/993360763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0bac0 msgr2=0x557415f10480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f504ffff700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0bac0 0x557415f10480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f504ffff700 1 -- 172.21.15.66:0/993360763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50501a4a80 con 0x557415f0b1b0 2023-07-21T22:33:35.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f504ffff700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 0x557415f0fe50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f503c01b040 tx=0x7f503c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43cc38ae9a0afa15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f502ffff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f502ffff700 1 -- 172.21.15.66:0/993360763 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c004fc0 con 0x557415f0b1b0 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f502ffff700 1 -- 172.21.15.66:0/993360763 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f503c005120 con 0x557415f0b1b0 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.708+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557415db14f0 con 0x557415f0b1b0 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f502ffff700 1 -- 172.21.15.66:0/993360763 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f503c0052e0 con 0x557415f0b1b0 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 10 client.27239 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:35.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f503c009e20 con 0x557415f0b1b0 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f502ffff700 1 -- 172.21.15.66:0/993360763 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557415db14f0 con 0x557415f0b1b0 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f502ffff700 1 -- 172.21.15.66:0/993360763 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f503c003c40 con 0x557415f0b1b0 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 10 client.27239 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 10 client.27239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 20 client.27239 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f20330 0x557415f22740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 4 client.27239 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:33:35.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.709+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557415e50eb0 con 0x557415f20330 2023-07-21T22:33:35.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.710+0000 7f505499b700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f20330 0x557415f22740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.710+0000 7f505499b700 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f20330 0x557415f22740 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741ea816c0e21b01 server_cookie=70bedf73c386af6d in_seq=0 out_seq=0 2023-07-21T22:33:35.763 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.710+0000 7f502ffff700 10 client.27239 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f502ffff700 1 -- 172.21.15.66:0/993360763 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557415e50eb0 con 0x557415f20330 2023-07-21T22:33:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f502ffff700 10 client.27239 handle_command_reply: tid=0 2023-07-21T22:33:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 client.27239 shutdown 2023-07-21T22:33:35.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 msgr2=0x557415f0fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 0x557415f0fe50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f503c01b040 tx=0x7f503c0042f0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 shutdown_connections 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f20330 0x557415f22740 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f0bac0 0x557415f10480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f06930 0x557415f12c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 --2- 172.21.15.66:0/993360763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f0b1b0 0x557415f0fe50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 >> 172.21.15.66:0/993360763 conn(0x557415efbef0 msgr2=0x557415f01420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f502ffff700 0 client.27239 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 shutdown_connections 2023-07-21T22:33:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.711+0000 7f506a580b80 1 -- 172.21.15.66:0/993360763 wait complete. 2023-07-21T22:33:35.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.712+0000 7f506a580b80 20 client.27239 trim_cache size 0 max 16384 2023-07-21T22:33:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.729+0000 7f506a580b80 1 Processor -- start 2023-07-21T22:33:35.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.729+0000 7f506a580b80 1 -- start start 2023-07-21T22:33:35.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.729+0000 7f506a580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415efbef0 0x557415f12770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.729+0000 7f506a580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 0x557415f0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f506a580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f0fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f506a580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415efb9b0 con 0x557415efbef0 2023-07-21T22:33:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f506a580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415f10a80 con 0x557415f06930 2023-07-21T22:33:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f506a580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415f005e0 con 0x557415f0b1b0 2023-07-21T22:33:35.777 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f505519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f0fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 0x557415f0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f505519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f0fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50564/0 (socket says 172.21.15.66:50564) 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f505519c700 1 -- 172.21.15.66:0/3445621270 learned_addr learned my addr 172.21.15.66:0/3445621270 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 0x557415f0f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41370/0 (socket says 172.21.15.66:41370) 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f505499b700 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415efbef0 0x557415f12770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 -- 172.21.15.66:0/3445621270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 msgr2=0x557415f0fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.778 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f0fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 -- 172.21.15.66:0/3445621270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415efbef0 msgr2=0x557415f12770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415efbef0 0x557415f12770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 -- 172.21.15.66:0/3445621270 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557415c6ec60 con 0x557415f06930 2023-07-21T22:33:35.779 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.730+0000 7f504ffff700 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 0x557415f0f710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f503c009340 tx=0x7f503c003fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10cde4beb791ed74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f503b7fe700 1 -- 172.21.15.66:0/3445621270 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c004610 con 0x557415f06930 2023-07-21T22:33:35.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f503b7fe700 1 -- 172.21.15.66:0/3445621270 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f503c011c60 con 0x557415f06930 2023-07-21T22:33:35.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f503b7fe700 1 -- 172.21.15.66:0/3445621270 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c003cc0 con 0x557415f06930 2023-07-21T22:33:35.782 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 -- 172.21.15.66:0/3445621270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 msgr2=0x557415f0f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 0x557415f0f710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f503c009340 tx=0x7f503c003fd0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 -- 172.21.15.66:0/3445621270 shutdown_connections 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f0fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415efbef0 0x557415f12770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 --2- 172.21.15.66:0/3445621270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415f06930 0x557415f0f710 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.731+0000 7f506a580b80 1 -- 172.21.15.66:0/3445621270 >> 172.21.15.66:0/3445621270 conn(0x557415f07060 msgr2=0x557415f04a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.783 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.732+0000 7f506a580b80 1 -- 172.21.15.66:0/3445621270 shutdown_connections 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.732+0000 7f506a580b80 1 -- 172.21.15.66:0/3445621270 wait complete. 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.732+0000 7f506a580b80 1 -- start start 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.732+0000 7f506a580b80 1 Processor -- start 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 0x557415f174a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f06930 0x557415f15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f15cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.784 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415f161f0 con 0x557415f06930 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415f16330 con 0x557415efbef0 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557415f164f0 con 0x557415f0b1b0 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f506a580b80 10 client.0 _ll_register_callbacks cb 0x7f5044009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f15cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 0x557415f174a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f504ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f06930 0x557415f15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505519c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f15cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:50572/0 (socket says 172.21.15.66:50572) 2023-07-21T22:33:35.785 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 0x557415f174a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41384/0 (socket says 172.21.15.66:41384) 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505519c700 1 -- 172.21.15.66:0/2795884273 learned_addr learned my addr 172.21.15.66:0/2795884273 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 -- 172.21.15.66:0/2795884273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 msgr2=0x557415f15cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f15cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 -- 172.21.15.66:0/2795884273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f06930 msgr2=0x557415f15770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f06930 0x557415f15770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.733+0000 7f505499b700 1 -- 172.21.15.66:0/2795884273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c009e20 con 0x557415efbef0 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f505499b700 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 0x557415f174a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5044024000 tx=0x7f50440ce040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f8632fea8ef582 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:33:35.786 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f502ffff700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f502ffff700 1 -- 172.21.15.66:0/2795884273 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504409afd0 con 0x557415efbef0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f502ffff700 1 -- 172.21.15.66:0/2795884273 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f50440b3070 con 0x557415efbef0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f503c004830 con 0x557415efbef0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f502ffff700 1 -- 172.21.15.66:0/2795884273 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504409f000 con 0x557415efbef0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 10 client.27245 fetch_fsmap learned FSMap version 42 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x557415cac250 con 0x557415efbef0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f502ffff700 1 -- 172.21.15.66:0/2795884273 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f503c004830 con 0x557415efbef0 2023-07-21T22:33:35.787 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f502ffff700 1 -- 172.21.15.66:0/2795884273 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f50440b9170 con 0x557415efbef0 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 10 client.27245 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 10 client.27245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 20 client.27245 populate_metadata read hostname 'smithi066' 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f0bac0 0x557415f77330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 4 client.27245 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.734+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557415e13050 con 0x557415f0bac0 2023-07-21T22:33:35.788 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f504ffff700 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f0bac0 0x557415f77330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f502ffff700 10 client.27245 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f504ffff700 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f0bac0 0x557415f77330 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39cd4b9fc30b7ad7 server_cookie=92d57a2aaf3b71f1 in_seq=0 out_seq=0 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f502ffff700 1 -- 172.21.15.66:0/2795884273 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557415e13050 con 0x557415f0bac0 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f502ffff700 10 client.27245 handle_command_reply: tid=0 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f506a580b80 1 client.27245 shutdown 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 msgr2=0x557415f174a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.789 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.735+0000 7f506a580b80 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 0x557415f174a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5044024000 tx=0x7f50440ce040 comp rx=0 tx=0).stop 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 shutdown_connections 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557415efbef0 0x557415f174a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x557415f0bac0 0x557415f77330 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557415f0b1b0 0x557415f15cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 --2- 172.21.15.66:0/2795884273 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x557415f06930 0x557415f15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f502ffff700 0 client.27245 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 >> 172.21.15.66:0/2795884273 conn(0x557415f07060 msgr2=0x557415f0f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.790 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 shutdown_connections 2023-07-21T22:33:35.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 1 -- 172.21.15.66:0/2795884273 wait complete. 2023-07-21T22:33:35.791 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f506a580b80 20 client.27245 trim_cache size 0 max 16384 2023-07-21T22:33:35.791 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:33:35.791 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:33:35.791 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout: "002da751-3814-4bee-a120-0de2974fb9d7": { 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "002da751-3814-4bee-a120-0de2974fb9d7", 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:33:35.792 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:33:35.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f50400a1410 msgr2=0x7f50400a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f5056bff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f50400a1410 0x7f50400a38c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5048004230 tx=0x7f50480045f0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 msgr2=0x7f5050197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:33:35.793 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.736+0000 7f5056bff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050197e30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5044005910 tx=0x7f5044004710 comp rx=0 tx=0).stop 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 shutdown_connections 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f50400a1410 0x7f50400a38c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f505010a070 0x7f505019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5050100ca0 0x7f5050197e30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 --2- 172.21.15.66:0/4054036042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50501015b0 0x7f5050198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 >> 172.21.15.66:0/4054036042 conn(0x7f50500fa4b0 msgr2=0x7f50500fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 shutdown_connections 2023-07-21T22:33:35.794 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:33:35.737+0000 7f5056bff700 1 -- 172.21.15.66:0/4054036042 wait complete. 2023-07-21T22:33:35.801 INFO:tasks.cephfs.filesystem:scrub status for tag:002da751-3814-4bee-a120-0de2974fb9d7 - {'path': '/', 'tag': '002da751-3814-4bee-a120-0de2974fb9d7', 'options': 'recursive,force'} 2023-07-21T22:33:36.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: pgmap v378: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2023-07-21T22:33:36.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:33:36.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:36 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: pgmap v378: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2023-07-21T22:33:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:33:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:36.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:36.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:36 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: pgmap v378: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2023-07-21T22:33:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:36.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:36 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:36.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 590805 5.43 MB/sec execute 543 sec latency 309.806 ms 2023-07-21T22:33:37.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:37 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:37.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:37 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:37.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:37 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:33:37.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:37 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:37.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:37 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:37.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:37 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:37.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:37 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:33:37.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:37 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:37.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:37 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:37.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:37 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:37.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:37 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:33:37.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:37 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:33:37.572 INFO:tasks.workunit.client.0.smithi066.stdout: 1 590805 5.42 MB/sec execute 544 sec latency 1309.953 ms 2023-07-21T22:33:38.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:38 smithi104 ceph-mon[129021]: pgmap v379: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-21T22:33:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:38 smithi066 ceph-mon[126379]: pgmap v379: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-21T22:33:38.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:38 smithi149 ceph-mon[129268]: pgmap v379: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-21T22:33:38.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 590805 5.41 MB/sec execute 545 sec latency 2310.080 ms 2023-07-21T22:33:39.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 590805 5.40 MB/sec execute 546 sec latency 3310.233 ms 2023-07-21T22:33:40.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:40 smithi104 ceph-mon[129021]: pgmap v380: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-21T22:33:40.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:40 smithi066 ceph-mon[126379]: pgmap v380: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-21T22:33:40.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:40 smithi149 ceph-mon[129268]: pgmap v380: 129 pgs: 129 active+clean; 590 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.5 MiB/s wr, 220 op/s 2023-07-21T22:33:40.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 591288 5.40 MB/sec execute 547 sec latency 3941.467 ms 2023-07-21T22:33:41.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:41 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:33:41.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:41 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:33:41.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:41 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:33:41.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 593131 5.41 MB/sec execute 548 sec latency 90.148 ms 2023-07-21T22:33:42.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:42 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:33:42.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:42 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:33:42.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:42 smithi104 ceph-mon[129021]: pgmap v381: 129 pgs: 129 active+clean; 593 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2023-07-21T22:33:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:42 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:33:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:42 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:33:42.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:42 smithi066 ceph-mon[126379]: pgmap v381: 129 pgs: 129 active+clean; 593 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2023-07-21T22:33:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:42 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:33:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:42 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:33:42.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:42 smithi149 ceph-mon[129268]: pgmap v381: 129 pgs: 129 active+clean; 593 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.1 MiB/s wr, 260 op/s 2023-07-21T22:33:42.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 594079 5.40 MB/sec execute 549 sec latency 16.787 ms 2023-07-21T22:33:43.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 595607 5.40 MB/sec execute 550 sec latency 33.361 ms 2023-07-21T22:33:44.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:44 smithi104 ceph-mon[129021]: pgmap v382: 129 pgs: 129 active+clean; 593 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2023-07-21T22:33:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:44 smithi066 ceph-mon[126379]: pgmap v382: 129 pgs: 129 active+clean; 593 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2023-07-21T22:33:44.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:44 smithi149 ceph-mon[129268]: pgmap v382: 129 pgs: 129 active+clean; 593 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2023-07-21T22:33:44.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 597092 5.42 MB/sec execute 551 sec latency 19.620 ms 2023-07-21T22:33:45.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 598241 5.42 MB/sec execute 552 sec latency 67.892 ms 2023-07-21T22:33:46.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:46 smithi104 ceph-mon[129021]: pgmap v383: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2023-07-21T22:33:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:46 smithi066 ceph-mon[126379]: pgmap v383: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2023-07-21T22:33:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:46 smithi149 ceph-mon[129268]: pgmap v383: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2023-07-21T22:33:46.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 599919 5.42 MB/sec execute 553 sec latency 44.535 ms 2023-07-21T22:33:47.573 INFO:tasks.workunit.client.0.smithi066.stdout: 1 600994 5.42 MB/sec execute 554 sec latency 15.408 ms 2023-07-21T22:33:48.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:48 smithi104 ceph-mon[129021]: pgmap v384: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 139 op/s 2023-07-21T22:33:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:48 smithi066 ceph-mon[126379]: pgmap v384: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 139 op/s 2023-07-21T22:33:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:48 smithi149 ceph-mon[129268]: pgmap v384: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 139 op/s 2023-07-21T22:33:48.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 602222 5.42 MB/sec execute 555 sec latency 51.201 ms 2023-07-21T22:33:49.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 603938 5.43 MB/sec execute 556 sec latency 21.830 ms 2023-07-21T22:33:50.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:50 smithi104 ceph-mon[129021]: pgmap v385: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 139 op/s 2023-07-21T22:33:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:50 smithi066 ceph-mon[126379]: pgmap v385: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 139 op/s 2023-07-21T22:33:50.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:50 smithi149 ceph-mon[129268]: pgmap v385: 129 pgs: 129 active+clean; 602 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 139 op/s 2023-07-21T22:33:50.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 604772 5.43 MB/sec execute 557 sec latency 26.222 ms 2023-07-21T22:33:51.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 606272 5.43 MB/sec execute 558 sec latency 40.887 ms 2023-07-21T22:33:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:52 smithi104 ceph-mon[129021]: pgmap v386: 129 pgs: 129 active+clean; 612 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2023-07-21T22:33:52.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:52 smithi066 ceph-mon[126379]: pgmap v386: 129 pgs: 129 active+clean; 612 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2023-07-21T22:33:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:52 smithi149 ceph-mon[129268]: pgmap v386: 129 pgs: 129 active+clean; 612 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2023-07-21T22:33:52.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 607691 5.44 MB/sec execute 559 sec latency 68.187 ms 2023-07-21T22:33:53.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 608837 5.44 MB/sec execute 560 sec latency 15.485 ms 2023-07-21T22:33:54.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:54 smithi104 ceph-mon[129021]: pgmap v387: 129 pgs: 129 active+clean; 612 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2023-07-21T22:33:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:54 smithi066 ceph-mon[126379]: pgmap v387: 129 pgs: 129 active+clean; 612 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2023-07-21T22:33:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:54 smithi149 ceph-mon[129268]: pgmap v387: 129 pgs: 129 active+clean; 612 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 201 op/s 2023-07-21T22:33:54.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 610552 5.45 MB/sec execute 561 sec latency 18.726 ms 2023-07-21T22:33:55.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:55 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:55.374 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:55 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:55.374 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:55 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:55.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:55 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:55.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:55 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:55.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:55 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:55.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:55 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:33:55.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:55 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:55.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:55 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:55.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 611459 5.45 MB/sec execute 562 sec latency 73.741 ms 2023-07-21T22:33:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:56 smithi104 ceph-mon[129021]: pgmap v388: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.2 MiB/s wr, 305 op/s 2023-07-21T22:33:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:56 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:56 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:56 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:33:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:56 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:33:56.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:56 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:56 smithi066 ceph-mon[126379]: pgmap v388: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.2 MiB/s wr, 305 op/s 2023-07-21T22:33:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:56 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:56 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:56 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:33:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:56 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:33:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:56 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:56 smithi149 ceph-mon[129268]: pgmap v388: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.2 MiB/s wr, 305 op/s 2023-07-21T22:33:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:56 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:56 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:56 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:33:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:56 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:33:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:56 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:33:56.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 612475 5.45 MB/sec execute 563 sec latency 34.583 ms 2023-07-21T22:33:57.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 614418 5.46 MB/sec execute 564 sec latency 15.139 ms 2023-07-21T22:33:58.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:33:58 smithi104 ceph-mon[129021]: pgmap v389: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-21T22:33:58.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:33:58 smithi066 ceph-mon[126379]: pgmap v389: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-21T22:33:58.495 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:33:58 smithi149 ceph-mon[129268]: pgmap v389: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-21T22:33:58.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 615349 5.45 MB/sec execute 565 sec latency 14.034 ms 2023-07-21T22:33:59.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 616744 5.45 MB/sec execute 566 sec latency 28.235 ms 2023-07-21T22:34:00.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:00 smithi066 ceph-mon[126379]: pgmap v390: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-21T22:34:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:00 smithi149 ceph-mon[129268]: pgmap v390: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-21T22:34:00.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 618217 5.47 MB/sec execute 567 sec latency 37.460 ms 2023-07-21T22:34:00.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:00 smithi104 ceph-mon[129021]: pgmap v390: 129 pgs: 129 active+clean; 621 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2023-07-21T22:34:01.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:01 smithi066 ceph-mon[126379]: pgmap v391: 129 pgs: 129 active+clean; 629 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2023-07-21T22:34:01.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:01 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:01.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:01 smithi149 ceph-mon[129268]: pgmap v391: 129 pgs: 129 active+clean; 629 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2023-07-21T22:34:01.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:01 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:01.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 618991 5.46 MB/sec execute 568 sec latency 25.083 ms 2023-07-21T22:34:01.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:01 smithi104 ceph-mon[129021]: pgmap v391: 129 pgs: 129 active+clean; 629 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 302 op/s 2023-07-21T22:34:01.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:01 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:02.574 INFO:tasks.workunit.client.0.smithi066.stdout: 1 619418 5.46 MB/sec execute 569 sec latency 81.606 ms 2023-07-21T22:34:03.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 620996 5.46 MB/sec execute 570 sec latency 25.810 ms 2023-07-21T22:34:04.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:03 smithi104 ceph-mon[129021]: pgmap v392: 129 pgs: 129 active+clean; 629 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 200 op/s 2023-07-21T22:34:04.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:03 smithi066 ceph-mon[126379]: pgmap v392: 129 pgs: 129 active+clean; 629 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 200 op/s 2023-07-21T22:34:04.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:03 smithi149 ceph-mon[129268]: pgmap v392: 129 pgs: 129 active+clean; 629 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 200 op/s 2023-07-21T22:34:04.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 622101 5.46 MB/sec execute 571 sec latency 13.490 ms 2023-07-21T22:34:05.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 623219 5.46 MB/sec execute 572 sec latency 127.164 ms 2023-07-21T22:34:05.802 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:34:06.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.062+0000 7fe09f5ff700 1 -- 172.21.15.66:0/3819226350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806d3a0 msgr2=0x7fe09811b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.062+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/3819226350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806d3a0 0x7fe09811b470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe09001c0d0 tx=0x7fe09001c3e0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.064+0000 7fe09f5ff700 1 -- 172.21.15.66:0/3819226350 shutdown_connections 2023-07-21T22:34:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.064+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/3819226350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806d3a0 0x7fe09811b470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.064+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/3819226350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09810e280 0x7fe09810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.064+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/3819226350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09806c9a0 0x7fe09806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.064+0000 7fe09f5ff700 1 -- 172.21.15.66:0/3819226350 >> 172.21.15.66:0/3819226350 conn(0x7fe09806bf70 msgr2=0x7fe098070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.066+0000 7fe09f5ff700 1 -- 172.21.15.66:0/3819226350 shutdown_connections 2023-07-21T22:34:06.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.066+0000 7fe09f5ff700 1 -- 172.21.15.66:0/3819226350 wait complete. 2023-07-21T22:34:06.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.067+0000 7fe09f5ff700 1 Processor -- start 2023-07-21T22:34:06.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.068+0000 7fe09f5ff700 1 -- start start 2023-07-21T22:34:06.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09f5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806c9a0 0x7fe098116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09f5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09806d3a0 0x7fe0981172b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09f5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09810e280 0x7fe0981a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.115 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09f5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981a8510 con 0x7fe09806d3a0 2023-07-21T22:34:06.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09f5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981a8650 con 0x7fe09806c9a0 2023-07-21T22:34:06.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09f5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0981a8810 con 0x7fe09810e280 2023-07-21T22:34:06.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.069+0000 7fe09d39b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806c9a0 0x7fe098116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09d39b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806c9a0 0x7fe098116d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49238/0 (socket says 172.21.15.66:49238) 2023-07-21T22:34:06.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09d39b700 1 -- 172.21.15.66:0/893035793 learned_addr learned my addr 172.21.15.66:0/893035793 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09806d3a0 0x7fe0981172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09cb9a700 1 -- 172.21.15.66:0/893035793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09810e280 msgr2=0x7fe0981a7f40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:34:06.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09810e280 0x7fe0981a7f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09cb9a700 1 -- 172.21.15.66:0/893035793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806c9a0 msgr2=0x7fe098116d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806c9a0 0x7fe098116d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.070+0000 7fe09cb9a700 1 -- 172.21.15.66:0/893035793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09001ad10 con 0x7fe09806d3a0 2023-07-21T22:34:06.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.072+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09806d3a0 0x7fe0981172b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe094006fd0 tx=0x7fe09400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.073+0000 7fe08a7fc700 1 -- 172.21.15.66:0/893035793 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094011830 con 0x7fe09806d3a0 2023-07-21T22:34:06.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.073+0000 7fe08a7fc700 1 -- 172.21.15.66:0/893035793 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe094011e50 con 0x7fe09806d3a0 2023-07-21T22:34:06.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.074+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0981a8af0 con 0x7fe09806d3a0 2023-07-21T22:34:06.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.074+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0981a9000 con 0x7fe09806d3a0 2023-07-21T22:34:06.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.074+0000 7fe08a7fc700 1 -- 172.21.15.66:0/893035793 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400f2e0 con 0x7fe09806d3a0 2023-07-21T22:34:06.128 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.082+0000 7fe0b2e4fb80 1 Processor -- start 2023-07-21T22:34:06.130 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.084+0000 7fe0b2e4fb80 1 -- start start 2023-07-21T22:34:06.131 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d111b0 0x555e34d11580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 0x555e34d0cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34c11cb0 con 0x555e34d111b0 2023-07-21T22:34:06.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34c64a40 con 0x555e34d0c930 2023-07-21T22:34:06.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34c5ded0 con 0x555e34d11ac0 2023-07-21T22:34:06.132 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe09cb9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 0x555e34d0cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe08a7fc700 1 -- 172.21.15.66:0/893035793 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe094010580 con 0x7fe09806d3a0 2023-07-21T22:34:06.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe09cb9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 0x555e34d0cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49244/0 (socket says 172.21.15.66:49244) 2023-07-21T22:34:06.133 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.085+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3779890470 learned_addr learned my addr 172.21.15.66:0/3779890470 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.087+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3779890470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 msgr2=0x555e34d0c350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T22:34:06.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.087+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.087+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3779890470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d111b0 msgr2=0x555e34d11580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.087+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d111b0 0x555e34d11580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.087+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3779890470 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e34a8bde0 con 0x555e34d0c930 2023-07-21T22:34:06.134 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.087+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 0x555e34d0cda0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe0940c9c00 tx=0x7fe0940d2c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548c93b2e038a811 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.088+0000 7fe08a7fc700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe0840a14d0 0x7fe0840a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.135 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.088+0000 7fe08a7fc700 1 -- 172.21.15.66:0/893035793 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe0940cc8c0 con 0x7fe09806d3a0 2023-07-21T22:34:06.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.090+0000 7fe0837fe700 1 -- 172.21.15.66:0/3779890470 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940cfea0 con 0x555e34d0c930 2023-07-21T22:34:06.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.090+0000 7fe0837fe700 1 -- 172.21.15.66:0/3779890470 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe094095980 con 0x555e34d0c930 2023-07-21T22:34:06.139 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.090+0000 7fe0837fe700 1 -- 172.21.15.66:0/3779890470 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094099410 con 0x555e34d0c930 2023-07-21T22:34:06.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.091+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3779890470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 msgr2=0x555e34d0cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.140 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.091+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 0x555e34d0cda0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe0940c9c00 tx=0x7fe0940d2c80 comp rx=0 tx=0).stop 2023-07-21T22:34:06.141 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.094+0000 7fe09db9c700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe0840a14d0 0x7fe0840a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.102+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3779890470 shutdown_connections 2023-07-21T22:34:06.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.102+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d0c930 0x555e34d0cda0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.102+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d111b0 0x555e34d11580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.102+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3779890470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.102+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3779890470 >> 172.21.15.66:0/3779890470 conn(0x555e34d01ef0 msgr2=0x555e34d07830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.105+0000 7fe09db9c700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe0840a14d0 0x7fe0840a3980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe09001c0d0 tx=0x7fe09001c730 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.107+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3779890470 shutdown_connections 2023-07-21T22:34:06.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.108+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3779890470 wait complete. 2023-07-21T22:34:06.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.109+0000 7fe0b2e4fb80 1 -- start start 2023-07-21T22:34:06.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.109+0000 7fe0b2e4fb80 1 Processor -- start 2023-07-21T22:34:06.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d15e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d16590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34bfdac0 con 0x555e34d0c930 2023-07-21T22:34:06.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34bfd7c0 con 0x555e34d11ac0 2023-07-21T22:34:06.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34ae1920 con 0x555e34d111b0 2023-07-21T22:34:06.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.110+0000 7fe0b2e4fb80 10 client.0 _ll_register_callbacks cb 0x555e34bfb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:06.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.111+0000 7fe09db9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d16590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.111+0000 7fe09db9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d16590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49248/0 (socket says 172.21.15.66:49248) 2023-07-21T22:34:06.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.111+0000 7fe09db9c700 1 -- 172.21.15.66:0/2773296787 learned_addr learned my addr 172.21.15.66:0/2773296787 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.111+0000 7fe09d39b700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.111+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d15e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.112+0000 7fe09d39b700 1 -- 172.21.15.66:0/2773296787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 msgr2=0x555e34d15e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.112+0000 7fe09d39b700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d15e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.112+0000 7fe09d39b700 1 -- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 msgr2=0x555e34d16590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.112+0000 7fe09d39b700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d16590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.112+0000 7fe09d39b700 1 -- 172.21.15.66:0/2773296787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0940d3040 con 0x555e34d0c930 2023-07-21T22:34:06.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.113+0000 7fe09d39b700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18c40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x555e34d08e70 tx=0x7fe08c0119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6eca79f5a6b7df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.113+0000 7fe080ff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:34:06.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.115+0000 7fe080ff9700 1 -- 172.21.15.66:0/2773296787 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c012860 con 0x555e34d0c930 2023-07-21T22:34:06.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.115+0000 7fe080ff9700 1 -- 172.21.15.66:0/2773296787 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe08c009e10 con 0x555e34d0c930 2023-07-21T22:34:06.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.115+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e34bb74f0 con 0x555e34d0c930 2023-07-21T22:34:06.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.115+0000 7fe080ff9700 1 -- 172.21.15.66:0/2773296787 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe08c0129c0 con 0x555e34d0c930 2023-07-21T22:34:06.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.116+0000 7fe0b2e4fb80 10 client.17193 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:06.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.116+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555e34a74c60 con 0x555e34d0c930 2023-07-21T22:34:06.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe080ff9700 1 -- 172.21.15.66:0/2773296787 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e34bb74f0 con 0x555e34d0c930 2023-07-21T22:34:06.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe080ff9700 1 -- 172.21.15.66:0/2773296787 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe08c00fd10 con 0x555e34d0c930 2023-07-21T22:34:06.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe0b2e4fb80 10 client.17193 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:06.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe0b2e4fb80 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:06.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe0b2e4fb80 20 client.17193 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:06.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d26640 0x555e34d28a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe0b2e4fb80 4 client.17193 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:06.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.117+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e34c56eb0 con 0x555e34d26640 2023-07-21T22:34:06.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.118+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d26640 0x555e34d28a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.120+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d26640 0x555e34d28a50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ac925bc8c7b974 server_cookie=eae04816a9bc1d10 in_seq=0 out_seq=0 2023-07-21T22:34:06.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.120+0000 7fe080ff9700 10 client.17193 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:06.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe080ff9700 1 -- 172.21.15.66:0/2773296787 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e34c56eb0 con 0x555e34d26640 2023-07-21T22:34:06.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe080ff9700 10 client.17193 handle_command_reply: tid=0 2023-07-21T22:34:06.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 client.17193 shutdown 2023-07-21T22:34:06.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 msgr2=0x555e34d18c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18c40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x555e34d08e70 tx=0x7fe08c0119f0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 shutdown_connections 2023-07-21T22:34:06.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d16590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d26640 0x555e34d28a50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18c40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/2773296787 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d15e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe080ff9700 0 client.17193 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 >> 172.21.15.66:0/2773296787 conn(0x555e34d01ef0 msgr2=0x555e34d073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 shutdown_connections 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.121+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/2773296787 wait complete. 2023-07-21T22:34:06.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.122+0000 7fe0b2e4fb80 20 client.17193 trim_cache size 0 max 16384 2023-07-21T22:34:06.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.141+0000 7fe0b2e4fb80 1 Processor -- start 2023-07-21T22:34:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.141+0000 7fe0b2e4fb80 1 -- start start 2023-07-21T22:34:06.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 0x555e34d18cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34d08b70 con 0x555e34d0c930 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34d065e0 con 0x555e34d111b0 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34d13ea0 con 0x555e34d11ac0 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe09d39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe09cb9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 0x555e34d18cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe09d39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56572/0 (socket says 172.21.15.66:56572) 2023-07-21T22:34:06.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe09cb9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 0x555e34d18cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49258/0 (socket says 172.21.15.66:49258) 2023-07-21T22:34:06.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe09d39b700 1 -- 172.21.15.66:0/3573348720 learned_addr learned my addr 172.21.15.66:0/3573348720 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.190 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.142+0000 7fe09db9c700 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3573348720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 msgr2=0x555e34d0b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3573348720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 msgr2=0x555e34d18770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe09cb9a700 1 -- 172.21.15.66:0/3573348720 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e34d2d640 con 0x555e34d111b0 2023-07-21T22:34:06.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 0x555e34d18cb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe0940c9c00 tx=0x7fe0940d2470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c04665ab9a637bca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe083fff700 1 -- 172.21.15.66:0/3573348720 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940cfea0 con 0x555e34d111b0 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe083fff700 1 -- 172.21.15.66:0/3573348720 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe094010f00 con 0x555e34d111b0 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.143+0000 7fe083fff700 1 -- 172.21.15.66:0/3573348720 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094098fa0 con 0x555e34d111b0 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3573348720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 msgr2=0x555e34d18cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 0x555e34d18cb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe0940c9c00 tx=0x7fe0940d2470 comp rx=0 tx=0).stop 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3573348720 shutdown_connections 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d11ac0 0x555e34d0b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d18770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/3573348720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d111b0 0x555e34d18cb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3573348720 >> 172.21.15.66:0/3573348720 conn(0x555e34d01ef0 msgr2=0x555e34d0cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3573348720 shutdown_connections 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/3573348720 wait complete. 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.144+0000 7fe0b2e4fb80 1 -- start start 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 Processor -- start 2023-07-21T22:34:06.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d1cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d1d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d1b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34d1bbc0 con 0x555e34d0c930 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34d1bd30 con 0x555e34d11ac0 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e34d1bef0 con 0x555e34d111b0 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.145+0000 7fe0b2e4fb80 10 client.0 _ll_register_callbacks cb 0x7fe08c009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09d39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d1cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09d39b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d1cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56580/0 (socket says 172.21.15.66:56580) 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09db9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d1b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09d39b700 1 -- 172.21.15.66:0/1291866725 learned_addr learned my addr 172.21.15.66:0/1291866725 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d1d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 -- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 msgr2=0x555e34d1b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d1b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 -- 172.21.15.66:0/1291866725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 msgr2=0x555e34d1cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d1cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 -- 172.21.15.66:0/1291866725 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e34a8bde0 con 0x555e34d111b0 2023-07-21T22:34:06.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09d39b700 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe09cb9a700 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d1d4e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x555e34d191f0 tx=0x7fe094006dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d5f2e097246fc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.146+0000 7fe080ff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe080ff9700 1 -- 172.21.15.66:0/1291866725 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940cfea0 con 0x555e34d111b0 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe094095980 con 0x555e34d111b0 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe080ff9700 1 -- 172.21.15.66:0/1291866725 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe094010380 con 0x555e34d111b0 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe080ff9700 1 -- 172.21.15.66:0/1291866725 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0940ccad0 con 0x555e34d111b0 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 10 client.27229 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:06.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555e34ab2250 con 0x555e34d111b0 2023-07-21T22:34:06.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe080ff9700 1 -- 172.21.15.66:0/1291866725 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe094095980 con 0x555e34d111b0 2023-07-21T22:34:06.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe080ff9700 1 -- 172.21.15.66:0/1291866725 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe094095d50 con 0x555e34d111b0 2023-07-21T22:34:06.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 10 client.27229 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:06.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 10 client.27229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:06.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 20 client.27229 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d7b100 0x555e34d7d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 4 client.27229 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.147+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e34c19050 con 0x555e34d7b100 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe09d39b700 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d7b100 0x555e34d7d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe09d39b700 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d7b100 0x555e34d7d4e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d72bcce39c9a5a5 server_cookie=460d2fa27f313aed in_seq=0 out_seq=0 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe080ff9700 10 client.27229 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:06.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe080ff9700 1 -- 172.21.15.66:0/1291866725 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e34c19050 con 0x555e34d7b100 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe080ff9700 10 client.27229 handle_command_reply: tid=0 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe0b2e4fb80 1 client.27229 shutdown 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 msgr2=0x555e34d1d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.148+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d1d4e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x555e34d191f0 tx=0x7fe094006dc0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 shutdown_connections 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555e34d11ac0 0x555e34d1b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555e34d7b100 0x555e34d7d4e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555e34d0c930 0x555e34d1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 --2- 172.21.15.66:0/1291866725 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555e34d111b0 0x555e34d1d4e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe080ff9700 0 client.27229 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 >> 172.21.15.66:0/1291866725 conn(0x555e34d01ef0 msgr2=0x555e34d0b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 shutdown_connections 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 1 -- 172.21.15.66:0/1291866725 wait complete. 2023-07-21T22:34:06.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.149+0000 7fe0b2e4fb80 20 client.27229 trim_cache size 0 max 16384 2023-07-21T22:34:06.205 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:34:06.205 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:34:06.205 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:34:06.205 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:34:06.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe0840a14d0 msgr2=0x7fe0840a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe0840a14d0 0x7fe0840a3980 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe09001c0d0 tx=0x7fe09001c730 comp rx=0 tx=0).stop 2023-07-21T22:34:06.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09806d3a0 msgr2=0x7fe0981172b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09806d3a0 0x7fe0981172b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe094006fd0 tx=0x7fe09400b7e0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 shutdown_connections 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe09806c9a0 0x7fe098116d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe0840a14d0 0x7fe0840a3980 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe09810e280 0x7fe0981a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 --2- 172.21.15.66:0/893035793 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe09806d3a0 0x7fe0981172b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 >> 172.21.15.66:0/893035793 conn(0x7fe09806bf70 msgr2=0x7fe09806f160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 shutdown_connections 2023-07-21T22:34:06.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.150+0000 7fe09f5ff700 1 -- 172.21.15.66:0/893035793 wait complete. 2023-07-21T22:34:06.208 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:05 smithi066 ceph-mon[126379]: pgmap v393: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2023-07-21T22:34:06.216 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:34:06.216 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:34:06.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:05 smithi104 ceph-mon[129021]: pgmap v393: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2023-07-21T22:34:06.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 -- 172.21.15.66:0/3757609215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 msgr2=0x7f5b2010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3757609215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b2010c910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800b0a0 tx=0x7f5b1800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 -- 172.21.15.66:0/3757609215 shutdown_connections 2023-07-21T22:34:06.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3757609215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b2010c910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3757609215 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b20105ae0 0x7f5b20105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3757609215 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b200ffad0 0x7f5b200fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 -- 172.21.15.66:0/3757609215 >> 172.21.15.66:0/3757609215 conn(0x7f5b200fb590 msgr2=0x7f5b200fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 -- 172.21.15.66:0/3757609215 shutdown_connections 2023-07-21T22:34:06.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.396+0000 7f5b26fff700 1 -- 172.21.15.66:0/3757609215 wait complete. 2023-07-21T22:34:06.445 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.397+0000 7f5b26fff700 1 Processor -- start 2023-07-21T22:34:06.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 -- start start 2023-07-21T22:34:06.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b200ffad0 0x7f5b20197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b201983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b20105ae0 0x7f5b2019c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019ce60 con 0x7f5b200ffad0 2023-07-21T22:34:06.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019cfd0 con 0x7f5b20100480 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b26fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019d190 con 0x7f5b20105ae0 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b201983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b24d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b200ffad0 0x7f5b20197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b1ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b201983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49290/0 (socket says 172.21.15.66:49290) 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.398+0000 7f5b1ffff700 1 -- 172.21.15.66:0/3476756422 learned_addr learned my addr 172.21.15.66:0/3476756422 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b2559c700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b20105ae0 0x7f5b2019c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1ffff700 1 -- 172.21.15.66:0/3476756422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b20105ae0 msgr2=0x7f5b2019c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1ffff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b20105ae0 0x7f5b2019c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1ffff700 1 -- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b200ffad0 msgr2=0x7f5b20197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1ffff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b200ffad0 0x7f5b20197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1ffff700 1 -- 172.21.15.66:0/3476756422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14009520 con 0x7f5b20100480 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1ffff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b201983c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9a0 tx=0x7f5b0c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c009790 con 0x7f5b20100480 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b0c010430 con 0x7f5b20100480 2023-07-21T22:34:06.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c018680 con 0x7f5b20100480 2023-07-21T22:34:06.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.399+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b18009d00 con 0x7f5b20100480 2023-07-21T22:34:06.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.400+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a4ad0 con 0x7f5b20100480 2023-07-21T22:34:06.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.401+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20067300 con 0x7f5b20100480 2023-07-21T22:34:06.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.403+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5b0c0105a0 con 0x7f5b20100480 2023-07-21T22:34:06.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.405+0000 7f5b1dffb700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b100a1380 0x7f5b100a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.405+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f5b0c014070 con 0x7f5b20100480 2023-07-21T22:34:06.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.405+0000 7f5b24d9b700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b100a1380 0x7f5b100a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.406+0000 7f5b24d9b700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b100a1380 0x7f5b100a3830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d70 tx=0x7f5b14009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.406+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5b0c0962b0 con 0x7f5b20100480 2023-07-21T22:34:06.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:05 smithi149 ceph-mon[129268]: pgmap v393: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2023-07-21T22:34:06.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 624691 5.47 MB/sec execute 573 sec latency 70.481 ms 2023-07-21T22:34:06.738 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.691+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b201991d0 con 0x7f5b20100480 2023-07-21T22:34:06.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.692+0000 7f5b1dffb700 1 -- 172.21.15.66:0/3476756422 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10826 (secure 0 0 0) 0x7f5b0c0998d0 con 0x7f5b20100480 2023-07-21T22:34:06.739 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:34:06.739 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:34:01.079965+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:34:06.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b100a1380 msgr2=0x7f5b100a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b100a1380 0x7f5b100a3830 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d70 tx=0x7f5b14009260 comp rx=0 tx=0).stop 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 msgr2=0x7f5b201983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b201983c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9a0 tx=0x7f5b0c00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 shutdown_connections 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5b100a1380 0x7f5b100a3830 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5b20105ae0 0x7f5b2019c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5b200ffad0 0x7f5b20197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 --2- 172.21.15.66:0/3476756422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b20100480 0x7f5b201983c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 >> 172.21.15.66:0/3476756422 conn(0x7f5b200fb590 msgr2=0x7f5b2010af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 shutdown_connections 2023-07-21T22:34:06.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.695+0000 7f5b26fff700 1 -- 172.21.15.66:0/3476756422 wait complete. 2023-07-21T22:34:06.743 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 43 2023-07-21T22:34:06.761 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:34:06.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.943+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/2244452521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec01013b0 msgr2=0x7f0ec0109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.943+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/2244452521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec01013b0 0x7f0ec0109a80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009990 tx=0x7f0eac009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/2244452521 shutdown_connections 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/2244452521 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec0109fc0 0x7f0ec010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/2244452521 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ec0100aa0 0x7f0ec0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/2244452521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec01013b0 0x7f0ec0109a80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/2244452521 >> 172.21.15.66:0/2244452521 conn(0x7f0ec00745d0 msgr2=0x7f0ec00749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/2244452521 shutdown_connections 2023-07-21T22:34:06.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.944+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/2244452521 wait complete. 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.945+0000 7f0ec5d9d700 1 Processor -- start 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.945+0000 7f0ec5d9d700 1 -- start start 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec5d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 0x7f0ec019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec5d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec01013b0 0x7f0ec019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec5d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ec0109fc0 0x7f0ec0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec5d9d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01981b0 con 0x7f0ec01013b0 2023-07-21T22:34:06.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec5d9d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0198320 con 0x7f0ec0100aa0 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec5d9d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01984e0 con 0x7f0ec0109fc0 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 0x7f0ec019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 0x7f0ec019da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49314/0 (socket says 172.21.15.66:49314) 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/31876428 learned_addr learned my addr 172.21.15.66:0/31876428 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ebffff700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec01013b0 0x7f0ec019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec559c700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ec0109fc0 0x7f0ec0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:06.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/31876428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ec0109fc0 msgr2=0x7f0ec0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ec0109fc0 0x7f0ec0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec01013b0 msgr2=0x7f0ec019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:06.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec01013b0 0x7f0ec019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:06.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.946+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/31876428 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac0095f0 con 0x7f0ec0100aa0 2023-07-21T22:34:06.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.947+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 0x7f0ec019da30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e9a0 tx=0x7f0eb400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:06.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.947+0000 7f0ebdffb700 1 -- 172.21.15.66:0/31876428 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4009790 con 0x7f0ec0100aa0 2023-07-21T22:34:06.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.947+0000 7f0ebdffb700 1 -- 172.21.15.66:0/31876428 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eb4010430 con 0x7f0ec0100aa0 2023-07-21T22:34:06.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.947+0000 7f0ebdffb700 1 -- 172.21.15.66:0/31876428 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4018680 con 0x7f0ec0100aa0 2023-07-21T22:34:06.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.947+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec01987c0 con 0x7f0ec0100aa0 2023-07-21T22:34:06.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.947+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01a4a20 con 0x7f0ec0100aa0 2023-07-21T22:34:06.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.950+0000 7f0ebdffb700 1 -- 172.21.15.66:0/31876428 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0eb40105a0 con 0x7f0ec0100aa0 2023-07-21T22:34:06.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.953+0000 7f0edaa16b80 1 Processor -- start 2023-07-21T22:34:06.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.953+0000 7f0edaa16b80 1 -- start start 2023-07-21T22:34:06.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.953+0000 7f0ebdffb700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0eb00a13e0 0x7f0eb00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.953+0000 7f0ebdffb700 1 -- 172.21.15.66:0/31876428 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0eb4014070 con 0x7f0ec0100aa0 2023-07-21T22:34:07.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.953+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 0x55d8d81af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.953+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81afda0 0x55d8d81ba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d80b5cb0 con 0x55d8d81afda0 2023-07-21T22:34:07.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d8108a40 con 0x55d8d81af490 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d8101ed0 con 0x55d8d81bac40 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec559c700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0eb00a13e0 0x7f0eb00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81afda0 0x55d8d81ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 0x55d8d81af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81afda0 0x55d8d81ba700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56610/0 (socket says 172.21.15.66:56610) 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ebffff700 1 -- 172.21.15.66:0/3403836983 learned_addr learned my addr 172.21.15.66:0/3403836983 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 0x55d8d81af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49316/0 (socket says 172.21.15.66:49316) 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/3403836983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 msgr2=0x55d8d81bd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/3403836983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81afda0 msgr2=0x55d8d81ba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81afda0 0x55d8d81ba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.954+0000 7f0ec4d9b700 1 -- 172.21.15.66:0/3403836983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8d7f2fde0 con 0x55d8d81af490 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.955+0000 7f0ec559c700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0eb00a13e0 0x7f0eb00a3890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ec019e730 tx=0x7f0eb8009920 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.955+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 0x55d8d81af860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40ca5e0 tx=0x7f0eb4099990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc5eb5342a4e28d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.003 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.955+0000 7f0eaaffd700 1 -- 172.21.15.66:0/3403836983 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40a23b0 con 0x55d8d81af490 2023-07-21T22:34:07.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.955+0000 7f0eaaffd700 1 -- 172.21.15.66:0/3403836983 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eb4010880 con 0x55d8d81af490 2023-07-21T22:34:07.004 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.955+0000 7f0eaaffd700 1 -- 172.21.15.66:0/3403836983 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40996b0 con 0x55d8d81af490 2023-07-21T22:34:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 -- 172.21.15.66:0/3403836983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 msgr2=0x55d8d81af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 0x55d8d81af860 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40ca5e0 tx=0x7f0eb4099990 comp rx=0 tx=0).stop 2023-07-21T22:34:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 -- 172.21.15.66:0/3403836983 shutdown_connections 2023-07-21T22:34:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81af490 0x55d8d81af860 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.006 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/3403836983 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81afda0 0x55d8d81ba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 -- 172.21.15.66:0/3403836983 >> 172.21.15.66:0/3403836983 conn(0x55d8d81a5ef0 msgr2=0x55d8d81ab860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 -- 172.21.15.66:0/3403836983 shutdown_connections 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.956+0000 7f0edaa16b80 1 -- 172.21.15.66:0/3403836983 wait complete. 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.957+0000 7f0edaa16b80 1 -- start start 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.957+0000 7f0edaa16b80 1 Processor -- start 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.957+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.957+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81afda0 0x55d8d81b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.007 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d80a1ac0 con 0x55d8d81af490 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d80a17c0 con 0x55d8d81afda0 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d7f85920 con 0x55d8d81bac40 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0edaa16b80 10 client.0 _ll_register_callbacks cb 0x55d8d809f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec4d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.008 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35462/0 (socket says 172.21.15.66:35462) 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 -- 172.21.15.66:0/2087931507 learned_addr learned my addr 172.21.15.66:0/2087931507 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ebffff700 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81afda0 0x55d8d81b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 -- 172.21.15.66:0/2087931507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81afda0 msgr2=0x55d8d81b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81afda0 0x55d8d81b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 -- 172.21.15.66:0/2087931507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 msgr2=0x55d8d81ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81ba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.958+0000 7f0ec559c700 1 -- 172.21.15.66:0/2087931507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec01a4a20 con 0x55d8d81bac40 2023-07-21T22:34:07.009 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0ec559c700 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81b80c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800f340 tx=0x7f0eb8004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87409d87a94c35f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0e9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0e9ffff700 1 -- 172.21.15.66:0/2087931507 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800e040 con 0x55d8d81bac40 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8d805b4f0 con 0x55d8d81bac40 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0e9ffff700 1 -- 172.21.15.66:0/2087931507 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0eb8019af0 con 0x55d8d81bac40 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0e9ffff700 1 -- 172.21.15.66:0/2087931507 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eb8019cf0 con 0x55d8d81bac40 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0edaa16b80 10 client.27244 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.959+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d8d7f18c60 con 0x55d8d81bac40 2023-07-21T22:34:07.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0e9ffff700 1 -- 172.21.15.66:0/2087931507 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8d805b4f0 con 0x55d8d81bac40 2023-07-21T22:34:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0e9ffff700 1 -- 172.21.15.66:0/2087931507 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0eb80163d0 con 0x55d8d81bac40 2023-07-21T22:34:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0edaa16b80 10 client.27244 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0edaa16b80 10 client.27244 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:34:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0edaa16b80 20 client.27244 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d81ca690 0x55d8d81ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.012 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0edaa16b80 4 client.27244 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8d80faeb0 con 0x55d8d81ca690 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.960+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d81ca690 0x55d8d81ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.961+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d81ca690 0x55d8d81ccaa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e271d363b42bc6e server_cookie=8a7a5e08257f9fe6 in_seq=0 out_seq=0 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.961+0000 7f0e9ffff700 10 client.27244 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0e9ffff700 1 -- 172.21.15.66:0/2087931507 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8d80faeb0 con 0x55d8d81ca690 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0e9ffff700 10 client.27244 handle_command_reply: tid=0 2023-07-21T22:34:07.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 client.27244 shutdown 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 msgr2=0x55d8d81b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81b80c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800f340 tx=0x7f0eb8004b40 comp rx=0 tx=0).stop 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 shutdown_connections 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d81ca690 0x55d8d81ccaa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81bac40 0x55d8d81b80c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81ba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0e9ffff700 0 client.27244 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:07.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/2087931507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81afda0 0x55d8d81b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 >> 172.21.15.66:0/2087931507 conn(0x55d8d81a5ef0 msgr2=0x55d8d81ab3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 shutdown_connections 2023-07-21T22:34:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 1 -- 172.21.15.66:0/2087931507 wait complete. 2023-07-21T22:34:07.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.962+0000 7f0edaa16b80 20 client.27244 trim_cache size 0 max 16384 2023-07-21T22:34:07.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 Processor -- start 2023-07-21T22:34:07.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 -- start start 2023-07-21T22:34:07.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81ba230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d81b8900 con 0x55d8d81af490 2023-07-21T22:34:07.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d81b8cd0 con 0x55d8d81bac40 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d81b2350 con 0x55d8d81afda0 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81ba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81ba230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35468/0 (socket says 172.21.15.66:35468) 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0ebffff700 1 -- 172.21.15.66:0/1147323937 learned_addr learned my addr 172.21.15.66:0/1147323937 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.979+0000 7f0ec559c700 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ebffff700 1 -- 172.21.15.66:0/1147323937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 msgr2=0x55d8d81b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ebffff700 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ebffff700 1 -- 172.21.15.66:0/1147323937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 msgr2=0x55d8d81bd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ebffff700 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81bd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ebffff700 1 -- 172.21.15.66:0/1147323937 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8d7f56250 con 0x55d8d81afda0 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0ebffff700 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81ba230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55d8d81b2560 tx=0x7f0eac00b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff4372cacd40e7a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0eab7fe700 1 -- 172.21.15.66:0/1147323937 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01d030 con 0x55d8d81afda0 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0eab7fe700 1 -- 172.21.15.66:0/1147323937 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eac01e040 con 0x55d8d81afda0 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.980+0000 7f0eab7fe700 1 -- 172.21.15.66:0/1147323937 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac004760 con 0x55d8d81afda0 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1147323937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 msgr2=0x55d8d81ba230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81ba230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55d8d81b2560 tx=0x7f0eac00b260 comp rx=0 tx=0).stop 2023-07-21T22:34:07.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1147323937 shutdown_connections 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1147323937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81ba230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.981+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1147323937 >> 172.21.15.66:0/1147323937 conn(0x55d8d81a5ef0 msgr2=0x55d8d81c6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.982+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1147323937 shutdown_connections 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.982+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1147323937 wait complete. 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.982+0000 7f0edaa16b80 1 -- start start 2023-07-21T22:34:07.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.982+0000 7f0edaa16b80 1 Processor -- start 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81bfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d81c02b0 con 0x55d8d81af490 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d81c0420 con 0x55d8d81bac40 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d81c05e0 con 0x55d8d81afda0 2023-07-21T22:34:07.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0edaa16b80 10 client.0 _ll_register_callbacks cb 0x7f0eb801c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0ebffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81bf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35476/0 (socket says 172.21.15.66:35476) 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0ec559c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81bfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0ebffff700 1 -- 172.21.15.66:0/1219230328 learned_addr learned my addr 172.21.15.66:0/1219230328 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.983+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0ebffff700 1 -- 172.21.15.66:0/1219230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 msgr2=0x55d8d81bfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0ebffff700 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81bfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0ebffff700 1 -- 172.21.15.66:0/1219230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 msgr2=0x55d8d81c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0ebffff700 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0ebffff700 1 -- 172.21.15.66:0/1219230328 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec010d0f0 con 0x55d8d81afda0 2023-07-21T22:34:07.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0ebffff700 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81bf830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000c00 tx=0x7f0eac00bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcbe966f16155987 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0e9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0e9ffff700 1 -- 172.21.15.66:0/1219230328 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01d040 con 0x55d8d81afda0 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0eb4010880 con 0x55d8d81afda0 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.984+0000 7f0e9ffff700 1 -- 172.21.15.66:0/1219230328 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0eac033030 con 0x55d8d81afda0 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.985+0000 7f0e9ffff700 1 -- 172.21.15.66:0/1219230328 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eac00f870 con 0x55d8d81afda0 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.985+0000 7f0edaa16b80 10 client.27250 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.985+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d8d7f2e2b0 con 0x55d8d81afda0 2023-07-21T22:34:07.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.985+0000 7f0e9ffff700 1 -- 172.21.15.66:0/1219230328 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0eb4010880 con 0x55d8d81afda0 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.985+0000 7f0e9ffff700 1 -- 172.21.15.66:0/1219230328 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0eac025a30 con 0x55d8d81afda0 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0edaa16b80 10 client.27250 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0edaa16b80 10 client.27250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0edaa16b80 20 client.27250 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d821f610 0x55d8d8221a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0edaa16b80 4 client.27250 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:34:07.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8d80bd050 con 0x55d8d821f610 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d821f610 0x55d8d8221a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0ec4d9b700 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d821f610 0x55d8d8221a20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f618a48dbf6ac20 server_cookie=635845aa632bcf49 in_seq=0 out_seq=0 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.986+0000 7f0e9ffff700 10 client.27250 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0e9ffff700 1 -- 172.21.15.66:0/1219230328 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8d80bd050 con 0x55d8d821f610 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0e9ffff700 10 client.27250 handle_command_reply: tid=0 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 client.27250 shutdown 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 msgr2=0x55d8d81bf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81bf830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0eac000c00 tx=0x7f0eac00bf80 comp rx=0 tx=0).stop 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 shutdown_connections 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x55d8d821f610 0x55d8d8221a20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8d81bac40 0x55d8d81bfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d8d81af490 0x55d8d81c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0e9ffff700 0 client.27250 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 --2- 172.21.15.66:0/1219230328 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d8d81afda0 0x55d8d81bf830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 >> 172.21.15.66:0/1219230328 conn(0x55d8d81a5ef0 msgr2=0x55d8d81b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.039 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 shutdown_connections 2023-07-21T22:34:07.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 1 -- 172.21.15.66:0/1219230328 wait complete. 2023-07-21T22:34:07.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.987+0000 7f0edaa16b80 20 client.27250 trim_cache size 0 max 16384 2023-07-21T22:34:07.040 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:34:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0eb00a13e0 msgr2=0x7f0eb00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0eb00a13e0 0x7f0eb00a3890 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ec019e730 tx=0x7f0eb8009920 comp rx=0 tx=0).stop 2023-07-21T22:34:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 msgr2=0x7f0ec019da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 0x7f0ec019da30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0eb400e9a0 tx=0x7f0eb400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 shutdown_connections 2023-07-21T22:34:07.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ec0100aa0 0x7f0ec019da30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0eb00a13e0 0x7f0eb00a3890 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0ec0109fc0 0x7f0ec0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 --2- 172.21.15.66:0/31876428 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0ec01013b0 0x7f0ec019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 >> 172.21.15.66:0/31876428 conn(0x7f0ec00745d0 msgr2=0x7f0ec00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 shutdown_connections 2023-07-21T22:34:07.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:06.988+0000 7f0ec5d9d700 1 -- 172.21.15.66:0/31876428 wait complete. 2023-07-21T22:34:07.055 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:34:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 -- 172.21.15.66:0/3472319011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c073140 msgr2=0x7efc8c10be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 --2- 172.21.15.66:0/3472319011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c073140 0x7efc8c10be80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efc800098f0 tx=0x7efc80009c00 comp rx=0 tx=0).stop 2023-07-21T22:34:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 -- 172.21.15.66:0/3472319011 shutdown_connections 2023-07-21T22:34:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 --2- 172.21.15.66:0/3472319011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c10c3c0 0x7efc8c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 --2- 172.21.15.66:0/3472319011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc8c072830 0x7efc8c072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 --2- 172.21.15.66:0/3472319011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c073140 0x7efc8c10be80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 -- 172.21.15.66:0/3472319011 >> 172.21.15.66:0/3472319011 conn(0x7efc8c0fb600 msgr2=0x7efc8c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 -- 172.21.15.66:0/3472319011 shutdown_connections 2023-07-21T22:34:07.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.251+0000 7efc913ff700 1 -- 172.21.15.66:0/3472319011 wait complete. 2023-07-21T22:34:07.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.252+0000 7efc913ff700 1 Processor -- start 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.252+0000 7efc913ff700 1 -- start start 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc913ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c072830 0x7efc8c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc913ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c073140 0x7efc8c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc913ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc8c10c3c0 0x7efc8c19c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc913ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c19ce50 con 0x7efc8c10c3c0 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc913ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c19cfc0 con 0x7efc8c073140 2023-07-21T22:34:07.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc913ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8c19d180 con 0x7efc8c072830 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c072830 0x7efc8c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c073140 0x7efc8c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c072830 0x7efc8c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35504/0 (socket says 172.21.15.66:35504) 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8affd700 1 -- 172.21.15.66:0/1677951105 learned_addr learned my addr 172.21.15.66:0/1677951105 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc8c10c3c0 0x7efc8c19c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8a7fc700 1 -- 172.21.15.66:0/1677951105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c072830 msgr2=0x7efc8c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.302 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c072830 0x7efc8c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8a7fc700 1 -- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc8c10c3c0 msgr2=0x7efc8c19c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc8c10c3c0 0x7efc8c19c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.253+0000 7efc8a7fc700 1 -- 172.21.15.66:0/1677951105 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc80009520 con 0x7efc8c073140 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.254+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c073140 0x7efc8c1983b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc80009810 tx=0x7efc8000fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.254+0000 7efc73fff700 1 -- 172.21.15.66:0/1677951105 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8001c070 con 0x7efc8c073140 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.254+0000 7efc73fff700 1 -- 172.21.15.66:0/1677951105 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8000bba0 con 0x7efc8c073140 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.254+0000 7efc73fff700 1 -- 172.21.15.66:0/1677951105 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80003970 con 0x7efc8c073140 2023-07-21T22:34:07.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.254+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8c19d400 con 0x7efc8c073140 2023-07-21T22:34:07.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.254+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8c19d650 con 0x7efc8c073140 2023-07-21T22:34:07.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.259+0000 7efc73fff700 1 -- 172.21.15.66:0/1677951105 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7efc80020030 con 0x7efc8c073140 2023-07-21T22:34:07.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.260+0000 7efca4cc5b80 1 Processor -- start 2023-07-21T22:34:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 -- start start 2023-07-21T22:34:07.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e84350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e84da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1d89cb0 con 0x55e7e1e84930 2023-07-21T22:34:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1ddca40 con 0x55e7e1e919c0 2023-07-21T22:34:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1dd5ed0 con 0x55e7e1e922d0 2023-07-21T22:34:07.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efc73fff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efc740a1340 0x7efc740a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efc8affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e84350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efc8affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e84350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35514/0 (socket says 172.21.15.66:35514) 2023-07-21T22:34:07.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.261+0000 7efc8affd700 1 -- 172.21.15.66:0/4194386763 learned_addr learned my addr 172.21.15.66:0/4194386763 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e84da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efc740a1340 0x7efc740a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8affd700 1 -- 172.21.15.66:0/4194386763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 msgr2=0x55e7e1e91d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8affd700 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e91d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8affd700 1 -- 172.21.15.66:0/4194386763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 msgr2=0x55e7e1e84da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.310 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8affd700 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e84da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8affd700 1 -- 172.21.15.66:0/4194386763 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7e1c03de0 con 0x55e7e1e922d0 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc73fff700 1 -- 172.21.15.66:0/1677951105 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efc800cdee0 con 0x7efc8c073140 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e91d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.262+0000 7efc8affd700 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e84350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00bea0 tx=0x7efc7c0106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595151356e113c2e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.263+0000 7efc717fa700 1 -- 172.21.15.66:0/4194386763 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c011440 con 0x55e7e1e922d0 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.263+0000 7efc717fa700 1 -- 172.21.15.66:0/4194386763 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc7c015070 con 0x55e7e1e922d0 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.263+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efc740a1340 0x7efc740a37f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc84007760 tx=0x7efc8400b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.311 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.263+0000 7efc717fa700 1 -- 172.21.15.66:0/4194386763 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c019900 con 0x55e7e1e922d0 2023-07-21T22:34:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.263+0000 7efca4cc5b80 1 -- 172.21.15.66:0/4194386763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 msgr2=0x55e7e1e84350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.263+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e84350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00bea0 tx=0x7efc7c0106b0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 -- 172.21.15.66:0/4194386763 shutdown_connections 2023-07-21T22:34:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e91d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e84da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/4194386763 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e84350 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 -- 172.21.15.66:0/4194386763 >> 172.21.15.66:0/4194386763 conn(0x55e7e1e79ef0 msgr2=0x55e7e1e7f830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 -- 172.21.15.66:0/4194386763 shutdown_connections 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 -- 172.21.15.66:0/4194386763 wait complete. 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.264+0000 7efca4cc5b80 1 -- start start 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 Processor -- start 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 0x55e7e1e99340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e96630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e922d0 0x55e7e1e96c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1d75ac0 con 0x55e7e1e922d0 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1d757c0 con 0x55e7e1e919c0 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1c59920 con 0x55e7e1e84930 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efca4cc5b80 10 client.0 _ll_register_callbacks cb 0x55e7e1d73760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efc8affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 0x55e7e1e99340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efc8affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 0x55e7e1e99340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35516/0 (socket says 172.21.15.66:35516) 2023-07-21T22:34:07.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efc8affd700 1 -- 172.21.15.66:0/907000237 learned_addr learned my addr 172.21.15.66:0/907000237 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e96630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.265+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e922d0 0x55e7e1e96c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc8affd700 1 -- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 msgr2=0x55e7e1e96630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc8affd700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e96630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc8affd700 1 -- 172.21.15.66:0/907000237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e922d0 msgr2=0x55e7e1e96c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc8affd700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e922d0 0x55e7e1e96c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc8affd700 1 -- 172.21.15.66:0/907000237 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c00bad0 con 0x55e7e1e84930 2023-07-21T22:34:07.317 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc8affd700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 0x55e7e1e99340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b7f0 tx=0x7efc7c010e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d17d9283d29594 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc6eff5700 1 -- 172.21.15.66:0/907000237 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7c005340 con 0x55e7e1e84930 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efc6eff5700 1 -- 172.21.15.66:0/907000237 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efc7c0054a0 con 0x55e7e1e84930 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.266+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7e1d2f4f0 con 0x55e7e1e84930 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efc6eff5700 1 -- 172.21.15.66:0/907000237 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc7c005660 con 0x55e7e1e84930 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 10 client.27262 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55e7e1becc60 con 0x55e7e1e84930 2023-07-21T22:34:07.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efc6eff5700 1 -- 172.21.15.66:0/907000237 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7e1d2f4f0 con 0x55e7e1e84930 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efc6eff5700 1 -- 172.21.15.66:0/907000237 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efc7c011840 con 0x55e7e1e84930 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 10 client.27262 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 10 client.27262 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 20 client.27262 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1ea6b50 0x55e7e1ea8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 4 client.27262 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7e1dceeb0 con 0x55e7e1ea6b50 2023-07-21T22:34:07.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.267+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1ea6b50 0x55e7e1ea8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.268+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1ea6b50 0x55e7e1ea8f60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d03b23d0748851c server_cookie=e24ee93bcc2c97f8 in_seq=0 out_seq=0 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.268+0000 7efc6eff5700 10 client.27262 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efc6eff5700 1 -- 172.21.15.66:0/907000237 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e7e1dceeb0 con 0x55e7e1ea6b50 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efc6eff5700 10 client.27262 handle_command_reply: tid=0 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 client.27262 shutdown 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 msgr2=0x55e7e1e99340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 0x55e7e1e99340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00b7f0 tx=0x7efc7c010e30 comp rx=0 tx=0).stop 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 shutdown_connections 2023-07-21T22:34:07.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e922d0 0x55e7e1e96c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e84930 0x55e7e1e99340 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1ea6b50 0x55e7e1ea8f60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/907000237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1e96630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 >> 172.21.15.66:0/907000237 conn(0x55e7e1e79ef0 msgr2=0x55e7e1e7f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efc6eff5700 0 client.27262 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 shutdown_connections 2023-07-21T22:34:07.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.269+0000 7efca4cc5b80 1 -- 172.21.15.66:0/907000237 wait complete. 2023-07-21T22:34:07.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.270+0000 7efca4cc5b80 20 client.27262 trim_cache size 0 max 16384 2023-07-21T22:34:07.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.284+0000 7efca4cc5b80 1 Processor -- start 2023-07-21T22:34:07.331 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.284+0000 7efca4cc5b80 1 -- start start 2023-07-21T22:34:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e9dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e919c0 0x55e7e1e98ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e922d0 0x55e7e1e83940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1e8df50 con 0x55e7e1e84930 2023-07-21T22:34:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1d2f4f0 con 0x55e7e1e922d0 2023-07-21T22:34:07.332 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1e7c010 con 0x55e7e1e919c0 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e9dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8a7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e919c0 0x55e7e1e98ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8affd700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e9dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56678/0 (socket says 172.21.15.66:56678) 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8affd700 1 -- 172.21.15.66:0/2190689392 learned_addr learned my addr 172.21.15.66:0/2190689392 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e922d0 0x55e7e1e83940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8a7fc700 1 -- 172.21.15.66:0/2190689392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e922d0 msgr2=0x55e7e1e83940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e922d0 0x55e7e1e83940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8a7fc700 1 -- 172.21.15.66:0/2190689392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 msgr2=0x55e7e1e9dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e9dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8a7fc700 1 -- 172.21.15.66:0/2190689392 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7e1c2a250 con 0x55e7e1e919c0 2023-07-21T22:34:07.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.285+0000 7efc8affd700 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e9dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:07.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.286+0000 7efc8a7fc700 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e919c0 0x55e7e1e98ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc800cb230 tx=0x7efc80075040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b89d56f5e121b68 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.286+0000 7efc71ffb700 1 -- 172.21.15.66:0/2190689392 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80005040 con 0x55e7e1e919c0 2023-07-21T22:34:07.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.286+0000 7efc71ffb700 1 -- 172.21.15.66:0/2190689392 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc8009ac40 con 0x55e7e1e919c0 2023-07-21T22:34:07.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.286+0000 7efc71ffb700 1 -- 172.21.15.66:0/2190689392 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc80097350 con 0x55e7e1e919c0 2023-07-21T22:34:07.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2190689392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e919c0 msgr2=0x55e7e1e98ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e919c0 0x55e7e1e98ee0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc800cb230 tx=0x7efc80075040 comp rx=0 tx=0).stop 2023-07-21T22:34:07.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2190689392 shutdown_connections 2023-07-21T22:34:07.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e922d0 0x55e7e1e83940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1e9dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2190689392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e919c0 0x55e7e1e98ee0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2190689392 >> 172.21.15.66:0/2190689392 conn(0x55e7e1e79ef0 msgr2=0x55e7e1ea2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2190689392 shutdown_connections 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.287+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2190689392 wait complete. 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.288+0000 7efca4cc5b80 1 -- start start 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.288+0000 7efca4cc5b80 1 Processor -- start 2023-07-21T22:34:07.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.288+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1ef3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.288+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1ef41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efca4cc5b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e99d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1e9a2a0 con 0x55e7e1e84930 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1e9a410 con 0x55e7e1e919c0 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efca4cc5b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7e1e9a5d0 con 0x55e7e1e922d0 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efca4cc5b80 10 client.0 _ll_register_callbacks cb 0x7efc7c00bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e99d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8a7fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1ef41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e99d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35530/0 (socket says 172.21.15.66:35530) 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 -- 172.21.15.66:0/2920987517 learned_addr learned my addr 172.21.15.66:0/2920987517 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8affd700 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1ef3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 -- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 msgr2=0x55e7e1ef41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1ef41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 -- 172.21.15.66:0/2920987517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 msgr2=0x55e7e1ef3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1ef3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.289+0000 7efc8b7fe700 1 -- 172.21.15.66:0/2920987517 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7e1becc60 con 0x55e7e1e922d0 2023-07-21T22:34:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc8b7fe700 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e99d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efc84018860 tx=0x7efc84004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=197cfac8d275b4e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc6eff5700 1 -- 172.21.15.66:0/2920987517 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8400e070 con 0x55e7e1e922d0 2023-07-21T22:34:07.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc6eff5700 1 -- 172.21.15.66:0/2920987517 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efc840256d0 con 0x55e7e1e922d0 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc7c015070 con 0x55e7e1e922d0 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc6eff5700 1 -- 172.21.15.66:0/2920987517 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc84025890 con 0x55e7e1e922d0 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efca4cc5b80 10 client.27268 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7efc8c19d650 con 0x55e7e1e922d0 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc6eff5700 1 -- 172.21.15.66:0/2920987517 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc84033020 con 0x55e7e1e922d0 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.290+0000 7efc6eff5700 1 -- 172.21.15.66:0/2920987517 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7efc8402db00 con 0x55e7e1e922d0 2023-07-21T22:34:07.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 10 client.27268 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 10 client.27268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 20 client.27268 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1efbb40 0x55e7e1efdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 4 client.27268 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7e1d91050 con 0x55e7e1efbb40 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efc8affd700 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1efbb40 0x55e7e1efdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efc8affd700 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1efbb40 0x55e7e1efdf20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=202eb19132603f1 server_cookie=6212517338fb1115 in_seq=0 out_seq=0 2023-07-21T22:34:07.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efc6eff5700 10 client.27268 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efc6eff5700 1 -- 172.21.15.66:0/2920987517 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7e1d91050 con 0x55e7e1efbb40 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efc6eff5700 10 client.27268 handle_command_reply: tid=0 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.291+0000 7efca4cc5b80 1 client.27268 shutdown 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 msgr2=0x55e7e1e99d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e99d60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efc84018860 tx=0x7efc84004ad0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 shutdown_connections 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e7e1e922d0 0x55e7e1e99d60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55e7e1e84930 0x55e7e1ef3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x55e7e1efbb40 0x55e7e1efdf20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 --2- 172.21.15.66:0/2920987517 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7e1e919c0 0x55e7e1ef41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efc6eff5700 0 client.27268 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 >> 172.21.15.66:0/2920987517 conn(0x55e7e1e79ef0 msgr2=0x55e7e1e946b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 shutdown_connections 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 1 -- 172.21.15.66:0/2920987517 wait complete. 2023-07-21T22:34:07.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.292+0000 7efca4cc5b80 20 client.27268 trim_cache size 0 max 16384 2023-07-21T22:34:07.346 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:34:07.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efc740a1340 msgr2=0x7efc740a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efc740a1340 0x7efc740a37f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efc84007760 tx=0x7efc8400b040 comp rx=0 tx=0).stop 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c073140 msgr2=0x7efc8c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c073140 0x7efc8c1983b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efc80009810 tx=0x7efc8000fa50 comp rx=0 tx=0).stop 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 shutdown_connections 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7efc740a1340 0x7efc740a37f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7efc8c10c3c0 0x7efc8c19c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc8c072830 0x7efc8c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 --2- 172.21.15.66:0/1677951105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc8c073140 0x7efc8c1983b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 >> 172.21.15.66:0/1677951105 conn(0x7efc8c0fb600 msgr2=0x7efc8c10e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 shutdown_connections 2023-07-21T22:34:07.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.293+0000 7efc913ff700 1 -- 172.21.15.66:0/1677951105 wait complete. 2023-07-21T22:34:07.357 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:07 smithi066 ceph-mon[126379]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T22:34:07.357 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:07 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3476756422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:34:07.360 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:34:07.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:07 smithi104 ceph-mon[129021]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T22:34:07.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:07 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3476756422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:34:07.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:07 smithi149 ceph-mon[129268]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T22:34:07.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:07 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3476756422' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:34:07.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 625640 5.47 MB/sec execute 574 sec latency 34.407 ms 2023-07-21T22:34:07.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.550+0000 7febe0fff700 1 -- 172.21.15.66:0/2783365744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc101610 msgr2=0x7febdc109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.550+0000 7febe0fff700 1 --2- 172.21.15.66:0/2783365744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc101610 0x7febdc109b00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7febd0009990 tx=0x7febd0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.597 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.550+0000 7febe0fff700 1 -- 172.21.15.66:0/2783365744 shutdown_connections 2023-07-21T22:34:07.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.550+0000 7febe0fff700 1 --2- 172.21.15.66:0/2783365744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 0x7febdc10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.551+0000 7febe0fff700 1 --2- 172.21.15.66:0/2783365744 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc100d00 0x7febdc1010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.551+0000 7febe0fff700 1 --2- 172.21.15.66:0/2783365744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc101610 0x7febdc109b00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.551+0000 7febe0fff700 1 -- 172.21.15.66:0/2783365744 >> 172.21.15.66:0/2783365744 conn(0x7febdc0fa4b0 msgr2=0x7febdc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.551+0000 7febe0fff700 1 -- 172.21.15.66:0/2783365744 shutdown_connections 2023-07-21T22:34:07.598 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.551+0000 7febe0fff700 1 -- 172.21.15.66:0/2783365744 wait complete. 2023-07-21T22:34:07.599 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.552+0000 7febe0fff700 1 Processor -- start 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.552+0000 7febe0fff700 1 -- start start 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febe0fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc100d00 0x7febdc0fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febe0fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc101610 0x7febdc0fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febe0fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 0x7febdc1a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febe0fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0ff6c0 con 0x7febdc101610 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febe0fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a4300 con 0x7febdc10a040 2023-07-21T22:34:07.600 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febe0fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1a4440 con 0x7febdc100d00 2023-07-21T22:34:07.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febda59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc100d00 0x7febdc0fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febda59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc100d00 0x7febdc0fe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35540/0 (socket says 172.21.15.66:35540) 2023-07-21T22:34:07.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febda59c700 1 -- 172.21.15.66:0/436505827 learned_addr learned my addr 172.21.15.66:0/436505827 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febd9d9b700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc101610 0x7febdc0fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.601 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.553+0000 7febdad9d700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 0x7febdc1a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febd9d9b700 1 -- 172.21.15.66:0/436505827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc100d00 msgr2=0x7febdc0fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febd9d9b700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc100d00 0x7febdc0fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febd9d9b700 1 -- 172.21.15.66:0/436505827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 msgr2=0x7febdc1a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.602 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febd9d9b700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 0x7febdc1a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febd9d9b700 1 -- 172.21.15.66:0/436505827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00095f0 con 0x7febdc101610 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febdad9d700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 0x7febdc1a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febd9d9b700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc101610 0x7febdc0fee90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febcb7fe700 1 -- 172.21.15.66:0/436505827 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd001d070 con 0x7febdc101610 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc1a4640 con 0x7febdc101610 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febcb7fe700 1 -- 172.21.15.66:0/436505827 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febd0021410 con 0x7febdc101610 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febcb7fe700 1 -- 172.21.15.66:0/436505827 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0017650 con 0x7febdc101610 2023-07-21T22:34:07.603 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.554+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc1a4ac0 con 0x7febdc101610 2023-07-21T22:34:07.605 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.558+0000 7febcb7fe700 1 -- 172.21.15.66:0/436505827 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7febd00217f0 con 0x7febdc101610 2023-07-21T22:34:07.606 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 Processor -- start 2023-07-21T22:34:07.607 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 -- start start 2023-07-21T22:34:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febcb7fe700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febc40a12c0 0x7febc40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febcb7fe700 1 -- 172.21.15.66:0/436505827 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7febd00ce400 con 0x7febdc101610 2023-07-21T22:34:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febdad9d700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febc40a12c0 0x7febc40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71df580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.608 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71dfac0 0x5583d71da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71dada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d70dfcb0 con 0x5583d71da930 2023-07-21T22:34:07.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d7132a40 con 0x5583d71df1b0 2023-07-21T22:34:07.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.560+0000 7febf48cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d712bed0 con 0x5583d71dfac0 2023-07-21T22:34:07.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.609 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71df580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49436/0 (socket says 172.21.15.66:49436) 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 -- 172.21.15.66:0/4011364925 learned_addr learned my addr 172.21.15.66:0/4011364925 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febdad9d700 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71dada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febd9d9b700 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71dfac0 0x5583d71da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 -- 172.21.15.66:0/4011364925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71dfac0 msgr2=0x5583d71da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71dfac0 0x5583d71da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 -- 172.21.15.66:0/4011364925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 msgr2=0x5583d71dada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.610 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71dada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 -- 172.21.15.66:0/4011364925 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d6f59de0 con 0x5583d71df1b0 2023-07-21T22:34:07.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febd9d9b700 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71dfac0 0x5583d71da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:07.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febdad9d700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febc40a12c0 0x7febc40a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febdc100a60 tx=0x7febd40045c0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.611 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.561+0000 7febda59c700 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71df580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7febcc009e50 tx=0x7febcc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100d02814c0b500d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.562+0000 7febc8ff9700 1 -- 172.21.15.66:0/4011364925 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc004a30 con 0x5583d71df1b0 2023-07-21T22:34:07.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.562+0000 7febc8ff9700 1 -- 172.21.15.66:0/4011364925 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febcc0054e0 con 0x5583d71df1b0 2023-07-21T22:34:07.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.562+0000 7febc8ff9700 1 -- 172.21.15.66:0/4011364925 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc019970 con 0x5583d71df1b0 2023-07-21T22:34:07.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.562+0000 7febf48cdb80 1 -- 172.21.15.66:0/4011364925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 msgr2=0x5583d71df580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.562+0000 7febf48cdb80 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71df580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7febcc009e50 tx=0x7febcc011720 comp rx=0 tx=0).stop 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 -- 172.21.15.66:0/4011364925 shutdown_connections 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71df580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71dada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 --2- 172.21.15.66:0/4011364925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71dfac0 0x5583d71da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 -- 172.21.15.66:0/4011364925 >> 172.21.15.66:0/4011364925 conn(0x5583d71cfef0 msgr2=0x5583d71d5830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 -- 172.21.15.66:0/4011364925 shutdown_connections 2023-07-21T22:34:07.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 -- 172.21.15.66:0/4011364925 wait complete. 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 -- start start 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.563+0000 7febf48cdb80 1 Processor -- start 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71e6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71df1b0 0x5583d71e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d70cbac0 con 0x5583d71da930 2023-07-21T22:34:07.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d70cb7c0 con 0x5583d71dfac0 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d6faf920 con 0x5583d71df1b0 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febf48cdb80 10 client.0 _ll_register_callbacks cb 0x5583d70c9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febda59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71e6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febdad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febd9d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71df1b0 0x5583d71e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febdad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49452/0 (socket says 172.21.15.66:49452) 2023-07-21T22:34:07.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.564+0000 7febdad9d700 1 -- 172.21.15.66:0/1232782338 learned_addr learned my addr 172.21.15.66:0/1232782338 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febd9d9b700 1 -- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 msgr2=0x5583d71e4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febd9d9b700 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febd9d9b700 1 -- 172.21.15.66:0/1232782338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 msgr2=0x5583d71e6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febd9d9b700 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71e6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febd9d9b700 1 -- 172.21.15.66:0/1232782338 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc009a80 con 0x5583d71df1b0 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febd9d9b700 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71df1b0 0x5583d71e3df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7febd00cb6f0 tx=0x7febd009ad30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93cc74f658f655e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:07.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febc1ffb700 1 -- 172.21.15.66:0/1232782338 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd009f130 con 0x5583d71df1b0 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febc1ffb700 1 -- 172.21.15.66:0/1232782338 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7febd00cbe40 con 0x5583d71df1b0 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583d70854f0 con 0x5583d71df1b0 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.565+0000 7febc1ffb700 1 -- 172.21.15.66:0/1232782338 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd00a34c0 con 0x5583d71df1b0 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 10 client.27277 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5583d6f42c60 con 0x5583d71df1b0 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febc1ffb700 1 -- 172.21.15.66:0/1232782338 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5583d70854f0 con 0x5583d71df1b0 2023-07-21T22:34:07.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febc1ffb700 1 -- 172.21.15.66:0/1232782338 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7febd000fad0 con 0x5583d71df1b0 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 10 client.27277 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 10 client.27277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 20 client.27277 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d71f3fc0 0x5583d71f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 4 client.27277 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583d7124eb0 con 0x5583d71f3fc0 2023-07-21T22:34:07.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.566+0000 7febda59c700 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d71f3fc0 0x5583d71f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.567+0000 7febda59c700 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d71f3fc0 0x5583d71f63d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972e852e019b6fea server_cookie=70b5d19b9ed2e54e in_seq=0 out_seq=0 2023-07-21T22:34:07.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.567+0000 7febc1ffb700 10 client.27277 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:07.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.568+0000 7febc1ffb700 1 -- 172.21.15.66:0/1232782338 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5583d7124eb0 con 0x5583d71f3fc0 2023-07-21T22:34:07.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.568+0000 7febc1ffb700 10 client.27277 handle_command_reply: tid=0 2023-07-21T22:34:07.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 client.27277 shutdown 2023-07-21T22:34:07.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71df1b0 msgr2=0x5583d71e3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71df1b0 0x5583d71e3df0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7febd00cb6f0 tx=0x7febd009ad30 comp rx=0 tx=0).stop 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 shutdown_connections 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d71f3fc0 0x5583d71f63d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71da930 0x5583d71e6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 --2- 172.21.15.66:0/1232782338 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71df1b0 0x5583d71e3df0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febc1ffb700 0 client.27277 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:07.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 >> 172.21.15.66:0/1232782338 conn(0x5583d71cfef0 msgr2=0x5583d71d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 shutdown_connections 2023-07-21T22:34:07.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 1 -- 172.21.15.66:0/1232782338 wait complete. 2023-07-21T22:34:07.624 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.569+0000 7febf48cdb80 20 client.27277 trim_cache size 0 max 16384 2023-07-21T22:34:07.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.583+0000 7febf48cdb80 1 Processor -- start 2023-07-21T22:34:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.583+0000 7febf48cdb80 1 -- start start 2023-07-21T22:34:07.630 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71dfac0 0x5583d71e6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febf48cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d71cfa00 con 0x5583d71dfac0 2023-07-21T22:34:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febf48cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d71db700 con 0x5583d71df1b0 2023-07-21T22:34:07.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febf48cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d71d6320 con 0x5583d71da930 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febd9d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35560/0 (socket says 172.21.15.66:35560) 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 -- 172.21.15.66:0/3618379429 learned_addr learned my addr 172.21.15.66:0/3618379429 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febdad9d700 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71dfac0 0x5583d71e6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 -- 172.21.15.66:0/3618379429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 msgr2=0x5583d71f0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71f0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 -- 172.21.15.66:0/3618379429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71dfac0 msgr2=0x5583d71e6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71dfac0 0x5583d71e6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febda59c700 1 -- 172.21.15.66:0/3618379429 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583d6f80250 con 0x5583d71da930 2023-07-21T22:34:07.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.584+0000 7febd9d9b700 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.585+0000 7febda59c700 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febcc01b040 tx=0x7febcc009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc02b1be39f2faf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.585+0000 7febc97fa700 1 -- 172.21.15.66:0/3618379429 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc029070 con 0x5583d71da930 2023-07-21T22:34:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.585+0000 7febc97fa700 1 -- 172.21.15.66:0/3618379429 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febcc02e070 con 0x5583d71da930 2023-07-21T22:34:07.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.585+0000 7febc97fa700 1 -- 172.21.15.66:0/3618379429 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc02ab70 con 0x5583d71da930 2023-07-21T22:34:07.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 -- 172.21.15.66:0/3618379429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 msgr2=0x5583d71eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7febcc01b040 tx=0x7febcc009790 comp rx=0 tx=0).stop 2023-07-21T22:34:07.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 -- 172.21.15.66:0/3618379429 shutdown_connections 2023-07-21T22:34:07.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71dfac0 0x5583d71e6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 --2- 172.21.15.66:0/3618379429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71df1b0 0x5583d71f0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 -- 172.21.15.66:0/3618379429 >> 172.21.15.66:0/3618379429 conn(0x5583d71cfef0 msgr2=0x5583d71d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 -- 172.21.15.66:0/3618379429 shutdown_connections 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.586+0000 7febf48cdb80 1 -- 172.21.15.66:0/3618379429 wait complete. 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.587+0000 7febf48cdb80 1 -- start start 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.587+0000 7febf48cdb80 1 Processor -- start 2023-07-21T22:34:07.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71df1b0 0x5583d71e95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d71ea070 con 0x5583d71df1b0 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d71ea1b0 con 0x5583d71dfac0 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583d71ea370 con 0x5583d71da930 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febf48cdb80 10 client.0 _ll_register_callbacks cb 0x7febd0097530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febdad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febd9d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71df1b0 0x5583d71e95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febdad9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49474/0 (socket says 172.21.15.66:49474) 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febdad9d700 1 -- 172.21.15.66:0/2065378506 learned_addr learned my addr 172.21.15.66:0/2065378506 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.588+0000 7febda59c700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febda59c700 1 -- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 msgr2=0x5583d71e9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febda59c700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febda59c700 1 -- 172.21.15.66:0/2065378506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71df1b0 msgr2=0x5583d71e95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:07.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febda59c700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71df1b0 0x5583d71e95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febda59c700 1 -- 172.21.15.66:0/2065378506 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583d6f42c60 con 0x5583d71da930 2023-07-21T22:34:07.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febd9d9b700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71df1b0 0x5583d71e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:07.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febdad9d700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:07.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febda59c700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb290 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7febcc01b040 tx=0x7febcc005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f1ef54d967b9d30 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:07.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febc1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:07.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febc1ffb700 1 -- 172.21.15.66:0/2065378506 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc029040 con 0x5583d71da930 2023-07-21T22:34:07.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febc1ffb700 1 -- 172.21.15.66:0/2065378506 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7febcc02a050 con 0x5583d71da930 2023-07-21T22:34:07.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febcc0054e0 con 0x5583d71da930 2023-07-21T22:34:07.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febc1ffb700 1 -- 172.21.15.66:0/2065378506 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7febcc02a210 con 0x5583d71da930 2023-07-21T22:34:07.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febf48cdb80 10 client.27283 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:07.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.589+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5583d6f59de0 con 0x5583d71da930 2023-07-21T22:34:07.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febc1ffb700 1 -- 172.21.15.66:0/2065378506 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febcc0054e0 con 0x5583d71da930 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febc1ffb700 1 -- 172.21.15.66:0/2065378506 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7febcc011440 con 0x5583d71da930 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febf48cdb80 10 client.27283 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febf48cdb80 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febf48cdb80 20 client.27283 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febf48cdb80 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d7249170 0x5583d724b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febf48cdb80 4 client.27283 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:34:07.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5583d70e7050 con 0x5583d7249170 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.590+0000 7febd9d9b700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d7249170 0x5583d724b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.591+0000 7febc1ffb700 10 client.27283 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.591+0000 7febd9d9b700 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d7249170 0x5583d724b550 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61b0aebc267e62cb server_cookie=c511ebff22f50f58 in_seq=0 out_seq=0 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febc1ffb700 1 -- 172.21.15.66:0/2065378506 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5583d70e7050 con 0x5583d7249170 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febc1ffb700 10 client.27283 handle_command_reply: tid=0 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 client.27283 shutdown 2023-07-21T22:34:07.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 msgr2=0x5583d71eb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb290 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7febcc01b040 tx=0x7febcc005020 comp rx=0 tx=0).stop 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 shutdown_connections 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5583d71dfac0 0x5583d71e9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5583d7249170 0x5583d724b550 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5583d71da930 0x5583d71eb290 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 --2- 172.21.15.66:0/2065378506 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5583d71df1b0 0x5583d71e95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febc1ffb700 0 client.27283 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 >> 172.21.15.66:0/2065378506 conn(0x5583d71cfef0 msgr2=0x5583d71f0730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 shutdown_connections 2023-07-21T22:34:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.592+0000 7febf48cdb80 1 -- 172.21.15.66:0/2065378506 wait complete. 2023-07-21T22:34:07.648 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febf48cdb80 20 client.27283 trim_cache size 0 max 16384 2023-07-21T22:34:07.648 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:34:07.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febc40a12c0 msgr2=0x7febc40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febc40a12c0 0x7febc40a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7febdc100a60 tx=0x7febd40045c0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc101610 msgr2=0x7febdc0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:07.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc101610 0x7febdc0fee90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7febd0000c00 tx=0x7febd0005740 comp rx=0 tx=0).stop 2023-07-21T22:34:07.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 shutdown_connections 2023-07-21T22:34:07.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febdc10a040 0x7febdc1a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7febc40a12c0 0x7febc40a3770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7febdc100d00 0x7febdc0fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 --2- 172.21.15.66:0/436505827 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7febdc101610 0x7febdc0fee90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:07.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 >> 172.21.15.66:0/436505827 conn(0x7febdc0fa4b0 msgr2=0x7febdc10c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:07.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.593+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 shutdown_connections 2023-07-21T22:34:07.651 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:07.594+0000 7febe0fff700 1 -- 172.21.15.66:0/436505827 wait complete. 2023-07-21T22:34:08.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:08 smithi104 ceph-mon[129021]: pgmap v395: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:34:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:08 smithi066 ceph-mon[126379]: pgmap v395: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:34:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:08 smithi149 ceph-mon[129268]: pgmap v395: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:34:08.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 626725 5.47 MB/sec execute 575 sec latency 17.202 ms 2023-07-21T22:34:08.659 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:34:08.659 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:34:08.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 -- 172.21.15.66:0/181429693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 msgr2=0x7f195c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 --2- 172.21.15.66:0/181429693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c0fff50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 -- 172.21.15.66:0/181429693 shutdown_connections 2023-07-21T22:34:08.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 --2- 172.21.15.66:0/181429693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c100490 0x7f195c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 --2- 172.21.15.66:0/181429693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f195c105af0 0x7f195c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 --2- 172.21.15.66:0/181429693 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c0fff50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.850+0000 7f1963fff700 1 -- 172.21.15.66:0/181429693 >> 172.21.15.66:0/181429693 conn(0x7f195c0fb5e0 msgr2=0x7f195c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:08.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.851+0000 7f1963fff700 1 -- 172.21.15.66:0/181429693 shutdown_connections 2023-07-21T22:34:08.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.851+0000 7f1963fff700 1 -- 172.21.15.66:0/181429693 wait complete. 2023-07-21T22:34:08.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 Processor -- start 2023-07-21T22:34:08.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 -- start start 2023-07-21T22:34:08.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.902 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f195c100490 0x7f195c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 0x7f195c1959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c196050 con 0x7f195c0ffae0 2023-07-21T22:34:08.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c1961c0 con 0x7f195c105af0 2023-07-21T22:34:08.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1963fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c196380 con 0x7f195c100490 2023-07-21T22:34:08.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.903 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.852+0000 7f196259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 0x7f195c1959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 0x7f195c1959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49490/0 (socket says 172.21.15.66:49490) 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 -- 172.21.15.66:0/1840359891 learned_addr learned my addr 172.21.15.66:0/1840359891 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56746/0 (socket says 172.21.15.66:56746) 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196159a700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f195c100490 0x7f195c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 -- 172.21.15.66:0/1840359891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f195c100490 msgr2=0x7f195c19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f195c100490 0x7f195c19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.904 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 -- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 msgr2=0x7f195c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 -- 172.21.15.66:0/1840359891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0095f0 con 0x7f195c105af0 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f196259c700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 0x7f195c1959e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f195400ef30 tx=0x7f195400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.853+0000 7f1952ffd700 1 -- 172.21.15.66:0/1840359891 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954018070 con 0x7f195c105af0 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.854+0000 7f1952ffd700 1 -- 172.21.15.66:0/1840359891 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f195400f040 con 0x7f195c105af0 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.854+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c196600 con 0x7f195c105af0 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.854+0000 7f1952ffd700 1 -- 172.21.15.66:0/1840359891 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400cd00 con 0x7f195c105af0 2023-07-21T22:34:08.905 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.854+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c1a2960 con 0x7f195c105af0 2023-07-21T22:34:08.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.857+0000 7f1952ffd700 1 -- 172.21.15.66:0/1840359891 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1954003a00 con 0x7f195c105af0 2023-07-21T22:34:08.907 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.859+0000 7f1952ffd700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19480a12c0 0x7f19480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.859+0000 7f19779ecb80 1 Processor -- start 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.859+0000 7f1952ffd700 1 -- 172.21.15.66:0/1840359891 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f19540ccca0 con 0x7f195c105af0 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.859+0000 7f19779ecb80 1 -- start start 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.859+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19480a12c0 0x7f19480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242e9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f19779ecb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3241eed30 con 0x55c3242f2f50 2023-07-21T22:34:08.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f19779ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324241ac0 con 0x55c3242e99b0 2023-07-21T22:34:08.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f19779ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c32423af50 con 0x55c3242f2640 2023-07-21T22:34:08.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242e9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.909 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242e9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49498/0 (socket says 172.21.15.66:49498) 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242e93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:56762/0 (socket says 172.21.15.66:56762) 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 -- 172.21.15.66:0/946943799 learned_addr learned my addr 172.21.15.66:0/946943799 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19480a12c0 0x7f19480a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1958009c30 tx=0x7f1958009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 -- 172.21.15.66:0/946943799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 msgr2=0x55c3242f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 -- 172.21.15.66:0/946943799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 msgr2=0x55c3242e9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.910 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242e9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.860+0000 7f196159a700 1 -- 172.21.15.66:0/946943799 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c324068e60 con 0x55c3242f2f50 2023-07-21T22:34:08.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.861+0000 7f196159a700 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242e93d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5631c28c63b819 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:08.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.861+0000 7f1947fff700 1 -- 172.21.15.66:0/946943799 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c01d070 con 0x55c3242f2f50 2023-07-21T22:34:08.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.861+0000 7f1947fff700 1 -- 172.21.15.66:0/946943799 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c01e040 con 0x55c3242f2f50 2023-07-21T22:34:08.911 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.861+0000 7f1947fff700 1 -- 172.21.15.66:0/946943799 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0048c0 con 0x55c3242f2f50 2023-07-21T22:34:08.914 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 -- 172.21.15.66:0/946943799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 msgr2=0x55c3242e93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242e93d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c005f00 comp rx=0 tx=0).stop 2023-07-21T22:34:08.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 -- 172.21.15.66:0/946943799 shutdown_connections 2023-07-21T22:34:08.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242e9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 --2- 172.21.15.66:0/946943799 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242e93d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.915 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 -- 172.21.15.66:0/946943799 >> 172.21.15.66:0/946943799 conn(0x55c3242def70 msgr2=0x55c3242e48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 -- 172.21.15.66:0/946943799 shutdown_connections 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.862+0000 7f19779ecb80 1 -- 172.21.15.66:0/946943799 wait complete. 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 -- start start 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 Processor -- start 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.916 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3241dab40 con 0x55c3242f2f50 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3241da840 con 0x55c3242e99b0 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.863+0000 7f19779ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3240be9a0 con 0x55c3242f2640 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f19779ecb80 10 client.0 _ll_register_callbacks cb 0x55c3241d87e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49502/0 (socket says 172.21.15.66:49502) 2023-07-21T22:34:08.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 -- 172.21.15.66:0/1643918646 learned_addr learned my addr 172.21.15.66:0/1643918646 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f196259c700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f196159a700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 -- 172.21.15.66:0/1643918646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 msgr2=0x55c3242f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 -- 172.21.15.66:0/1643918646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 msgr2=0x55c3242f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 -- 172.21.15.66:0/1643918646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c10d220 con 0x55c3242e99b0 2023-07-21T22:34:08.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fa050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f195801c540 tx=0x7f19580047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d7f60409518030 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.864+0000 7f19457fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19457fa700 1 -- 172.21.15.66:0/1643918646 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958004830 con 0x55c3242e99b0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19457fa700 1 -- 172.21.15.66:0/1643918646 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1958004990 con 0x55c3242e99b0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c324194570 con 0x55c3242e99b0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19457fa700 1 -- 172.21.15.66:0/1643918646 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1958004b50 con 0x55c3242e99b0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19779ecb80 10 client.27335 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c324051ce0 con 0x55c3242e99b0 2023-07-21T22:34:08.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.865+0000 7f19457fa700 1 -- 172.21.15.66:0/1643918646 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c324194570 con 0x55c3242e99b0 2023-07-21T22:34:08.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19457fa700 1 -- 172.21.15.66:0/1643918646 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1958025430 con 0x55c3242e99b0 2023-07-21T22:34:08.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19779ecb80 10 client.27335 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:08.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19779ecb80 10 client.27335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:08.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19779ecb80 20 client.27335 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:08.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c3243079c0 0x55c324309dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19779ecb80 4 client.27335 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:08.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c324233f30 con 0x55c3243079c0 2023-07-21T22:34:08.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.866+0000 7f196159a700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c3243079c0 0x55c324309dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.867+0000 7f196159a700 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c3243079c0 0x55c324309dd0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d39ddce473ae311b server_cookie=4892c6cc96296da7 in_seq=0 out_seq=0 2023-07-21T22:34:08.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.867+0000 7f19457fa700 10 client.27335 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:08.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19457fa700 1 -- 172.21.15.66:0/1643918646 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c324233f30 con 0x55c3243079c0 2023-07-21T22:34:08.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19457fa700 10 client.27335 handle_command_reply: tid=0 2023-07-21T22:34:08.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 client.27335 shutdown 2023-07-21T22:34:08.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 msgr2=0x55c3242fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fa050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f195801c540 tx=0x7f19580047c0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 shutdown_connections 2023-07-21T22:34:08.924 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fa050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c3243079c0 0x55c324309dd0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1643918646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19457fa700 0 client.27335 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:08.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 >> 172.21.15.66:0/1643918646 conn(0x55c3242def70 msgr2=0x55c3242e44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:08.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 shutdown_connections 2023-07-21T22:34:08.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 1 -- 172.21.15.66:0/1643918646 wait complete. 2023-07-21T22:34:08.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.868+0000 7f19779ecb80 20 client.27335 trim_cache size 0 max 16384 2023-07-21T22:34:08.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 Processor -- start 2023-07-21T22:34:08.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 -- start start 2023-07-21T22:34:08.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3242eeb60 con 0x55c3242f2f50 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3242f88b0 con 0x55c3242e99b0 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.890+0000 7f19779ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c324305020 con 0x55c3242f2640 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.938 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49504/0 (socket says 172.21.15.66:49504) 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 -- 172.21.15.66:0/3569208298 learned_addr learned my addr 172.21.15.66:0/3569208298 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f196159a700 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f196259c700 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 -- 172.21.15.66:0/3569208298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 msgr2=0x55c3242f9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 -- 172.21.15.66:0/3569208298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 msgr2=0x55c3242f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.939 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 -- 172.21.15.66:0/3569208298 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c32408f2d0 con 0x55c3242e99b0 2023-07-21T22:34:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f196259c700 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:08.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.891+0000 7f1961d9b700 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fe9b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1958013010 tx=0x7f195801cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2919263fbfd83a4f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:08.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.892+0000 7f1950ff9700 1 -- 172.21.15.66:0/3569208298 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580092d0 con 0x55c3242e99b0 2023-07-21T22:34:08.941 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.892+0000 7f1950ff9700 1 -- 172.21.15.66:0/3569208298 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1958009970 con 0x55c3242e99b0 2023-07-21T22:34:08.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.892+0000 7f1950ff9700 1 -- 172.21.15.66:0/3569208298 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580051b0 con 0x55c3242e99b0 2023-07-21T22:34:08.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.892+0000 7f19779ecb80 1 -- 172.21.15.66:0/3569208298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 msgr2=0x55c3242fe9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.892+0000 7f19779ecb80 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fe9b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1958013010 tx=0x7f195801cc50 comp rx=0 tx=0).stop 2023-07-21T22:34:08.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 -- 172.21.15.66:0/3569208298 shutdown_connections 2023-07-21T22:34:08.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242e99b0 0x55c3242fe9b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 --2- 172.21.15.66:0/3569208298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242f2640 0x55c3242f9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 -- 172.21.15.66:0/3569208298 >> 172.21.15.66:0/3569208298 conn(0x55c3242def70 msgr2=0x55c3242e7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 -- 172.21.15.66:0/3569208298 shutdown_connections 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 -- 172.21.15.66:0/3569208298 wait complete. 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 -- start start 2023-07-21T22:34:08.943 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.893+0000 7f19779ecb80 1 Processor -- start 2023-07-21T22:34:08.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242e99b0 0x55c3243550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 0x55c324355610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3242fe590 con 0x55c3242f2f50 2023-07-21T22:34:08.944 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3242fe700 con 0x55c3242f2640 2023-07-21T22:34:08.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3242fe8c0 con 0x55c3242e99b0 2023-07-21T22:34:08.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.894+0000 7f19779ecb80 10 client.0 _ll_register_callbacks cb 0x7f1958011ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:08.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 0x55c324355610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f1961d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242e99b0 0x55c3243550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 0x55c324355610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49510/0 (socket says 172.21.15.66:49510) 2023-07-21T22:34:08.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 -- 172.21.15.66:0/1459897237 learned_addr learned my addr 172.21.15.66:0/1459897237 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:08.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196259c700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 -- 172.21.15.66:0/1459897237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242e99b0 msgr2=0x55c3243550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242e99b0 0x55c3243550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 -- 172.21.15.66:0/1459897237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 msgr2=0x55c3242fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:08.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 -- 172.21.15.66:0/1459897237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c324051ce0 con 0x55c3242f2640 2023-07-21T22:34:08.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196259c700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:08.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f196159a700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 0x55c324355610 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90444bda2690da8a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:08.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.895+0000 7f19457fa700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:08.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19457fa700 1 -- 172.21.15.66:0/1459897237 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c033070 con 0x55c3242f2640 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f194c01e040 con 0x55c3242f2640 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19457fa700 1 -- 172.21.15.66:0/1459897237 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f194c0333d0 con 0x55c3242f2640 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19457fa700 1 -- 172.21.15.66:0/1459897237 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f194c01d040 con 0x55c3242f2640 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19779ecb80 10 client.27341 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c324067330 con 0x55c3242f2640 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.896+0000 7f19457fa700 1 -- 172.21.15.66:0/1459897237 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f194c01e040 con 0x55c3242f2640 2023-07-21T22:34:08.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19457fa700 1 -- 172.21.15.66:0/1459897237 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f194c033c00 con 0x55c3242f2640 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19779ecb80 10 client.27341 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19779ecb80 10 client.27341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19779ecb80 20 client.27341 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c32435dc30 0x55c324360040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19779ecb80 4 client.27341 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c3241f60d0 con 0x55c32435dc30 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.897+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c32435dc30 0x55c324360040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:08.951 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.898+0000 7f1961d9b700 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c32435dc30 0x55c324360040 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82bb832abe54fb6d server_cookie=14d73c362a0760a3 in_seq=0 out_seq=0 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.898+0000 7f19457fa700 10 client.27341 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19457fa700 1 -- 172.21.15.66:0/1459897237 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c3241f60d0 con 0x55c32435dc30 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19457fa700 10 client.27341 handle_command_reply: tid=0 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 client.27341 shutdown 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 msgr2=0x55c324355610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 0x55c324355610 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f194c009990 tx=0x7f194c009310 comp rx=0 tx=0).stop 2023-07-21T22:34:08.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 shutdown_connections 2023-07-21T22:34:08.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c32435dc30 0x55c324360040 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c3242f2f50 0x55c3242fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c3242e99b0 0x55c3243550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 --2- 172.21.15.66:0/1459897237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3242f2640 0x55c324355610 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19457fa700 0 client.27341 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:08.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 >> 172.21.15.66:0/1459897237 conn(0x55c3242def70 msgr2=0x55c3242e8a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:08.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 shutdown_connections 2023-07-21T22:34:08.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 1 -- 172.21.15.66:0/1459897237 wait complete. 2023-07-21T22:34:08.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.899+0000 7f19779ecb80 20 client.27341 trim_cache size 0 max 16384 2023-07-21T22:34:08.955 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:34:08.955 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:34:08.955 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "bdfcffea-65aa-405f-8215-607153e0da7a", 2023-07-21T22:34:08.955 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:34:08.956 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:34:08.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19480a12c0 msgr2=0x7f19480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19480a12c0 0x7f19480a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1958009c30 tx=0x7f1958009260 comp rx=0 tx=0).stop 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 msgr2=0x7f195c1959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 0x7f195c1959e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f195400ef30 tx=0x7f195400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 shutdown_connections 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f195c105af0 0x7f195c1959e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f19480a12c0 0x7f19480a3770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f195c0ffae0 0x7f195c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.957 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 --2- 172.21.15.66:0/1840359891 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f195c100490 0x7f195c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:08.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 >> 172.21.15.66:0/1840359891 conn(0x7f195c0fb5e0 msgr2=0x7f195c0fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:08.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 shutdown_connections 2023-07-21T22:34:08.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:08.900+0000 7f1963fff700 1 -- 172.21.15.66:0/1840359891 wait complete. 2023-07-21T22:34:08.969 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:34:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 -- 172.21.15.66:0/1540605440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8804105ae0 msgr2=0x7f8804105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 --2- 172.21.15.66:0/1540605440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8804105ae0 0x7f8804105eb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f87f40098f0 tx=0x7f87f4009c00 comp rx=0 tx=0).stop 2023-07-21T22:34:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 -- 172.21.15.66:0/1540605440 shutdown_connections 2023-07-21T22:34:09.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 --2- 172.21.15.66:0/1540605440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8804105ae0 0x7f8804105eb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 --2- 172.21.15.66:0/1540605440 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8804100480 0x7f880410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 --2- 172.21.15.66:0/1540605440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88040ffad0 0x7f88040fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 -- 172.21.15.66:0/1540605440 >> 172.21.15.66:0/1540605440 conn(0x7f88040fb5f0 msgr2=0x7f88040fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 -- 172.21.15.66:0/1540605440 shutdown_connections 2023-07-21T22:34:09.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.148+0000 7f88089ff700 1 -- 172.21.15.66:0/1540605440 wait complete. 2023-07-21T22:34:09.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.149+0000 7f88089ff700 1 Processor -- start 2023-07-21T22:34:09.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 -- start start 2023-07-21T22:34:09.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 0x7f8804197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.198 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8804100480 0x7f8804198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8804105ae0 0x7f880419c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880419ce30 con 0x7f8804105ae0 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880419cfa0 con 0x7f88040ffad0 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f88089ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880419d160 con 0x7f8804100480 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f880259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 0x7f8804197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f8801d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8804100480 0x7f8804198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f880259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 0x7f8804197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49536/0 (socket says 172.21.15.66:49536) 2023-07-21T22:34:09.199 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f880259c700 1 -- 172.21.15.66:0/666939077 learned_addr learned my addr 172.21.15.66:0/666939077 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.150+0000 7f8802d9d700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8804105ae0 0x7f880419c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f880259c700 1 -- 172.21.15.66:0/666939077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8804100480 msgr2=0x7f8804198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f880259c700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8804100480 0x7f8804198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f880259c700 1 -- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8804105ae0 msgr2=0x7f880419c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f880259c700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8804105ae0 0x7f880419c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f880259c700 1 -- 172.21.15.66:0/666939077 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009520 con 0x7f88040ffad0 2023-07-21T22:34:09.200 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f8801d9b700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8804100480 0x7f8804198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f880259c700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 0x7f8804197e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009810 tx=0x7f87f400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f87f37fe700 1 -- 172.21.15.66:0/666939077 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f401c070 con 0x7f88040ffad0 2023-07-21T22:34:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880419d3e0 con 0x7f88040ffad0 2023-07-21T22:34:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f87f37fe700 1 -- 172.21.15.66:0/666939077 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f400bbd0 con 0x7f88040ffad0 2023-07-21T22:34:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f87f37fe700 1 -- 172.21.15.66:0/666939077 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4004930 con 0x7f88040ffad0 2023-07-21T22:34:09.201 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.151+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88041a4c80 con 0x7f88040ffad0 2023-07-21T22:34:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.155+0000 7f87f37fe700 1 -- 172.21.15.66:0/666939077 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f87f4020070 con 0x7f88040ffad0 2023-07-21T22:34:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 Processor -- start 2023-07-21T22:34:09.202 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 -- start start 2023-07-21T22:34:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba231b0 0x555b8ba23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba1eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8b923cb0 con 0x555b8ba231b0 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8b976a40 con 0x555b8ba23ac0 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.156+0000 7f881c272b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8b96fed0 con 0x555b8ba1e930 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f880259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba1eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba1eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:35648/0 (socket says 172.21.15.66:35648) 2023-07-21T22:34:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f880259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba1e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49544/0 (socket says 172.21.15.66:49544) 2023-07-21T22:34:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 -- 172.21.15.66:0/3876824585 learned_addr learned my addr 172.21.15.66:0/3876824585 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba231b0 0x555b8ba23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 -- 172.21.15.66:0/3876824585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 msgr2=0x555b8ba1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.206 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 -- 172.21.15.66:0/3876824585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba231b0 msgr2=0x555b8ba23580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba231b0 0x555b8ba23580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 -- 172.21.15.66:0/3876824585 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b8b79dde0 con 0x555b8ba1e930 2023-07-21T22:34:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f87f37fe700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f87ec0a1340 0x7f87ec0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f87f37fe700 1 -- 172.21.15.66:0/666939077 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f87f40cdd10 con 0x7f88040ffad0 2023-07-21T22:34:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.157+0000 7f8801d9b700 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba1eda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87f80115f0 tx=0x7f87f8011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2f4291b2bd69a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f8802d9d700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f87ec0a1340 0x7f87ec0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f87f0ff9700 1 -- 172.21.15.66:0/3876824585 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8004a30 con 0x555b8ba1e930 2023-07-21T22:34:09.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f87f0ff9700 1 -- 172.21.15.66:0/3876824585 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87f80054e0 con 0x555b8ba1e930 2023-07-21T22:34:09.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f87f0ff9700 1 -- 172.21.15.66:0/3876824585 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f800e850 con 0x555b8ba1e930 2023-07-21T22:34:09.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f881c272b80 1 -- 172.21.15.66:0/3876824585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 msgr2=0x555b8ba1eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f8802d9d700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f87ec0a1340 0x7f87ec0a37f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0127f0 tx=0x7f87fc00b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:09.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.158+0000 7f881c272b80 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba1eda0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f87f80115f0 tx=0x7f87f8011900 comp rx=0 tx=0).stop 2023-07-21T22:34:09.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 -- 172.21.15.66:0/3876824585 shutdown_connections 2023-07-21T22:34:09.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba1eda0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba231b0 0x555b8ba23580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 --2- 172.21.15.66:0/3876824585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 -- 172.21.15.66:0/3876824585 >> 172.21.15.66:0/3876824585 conn(0x555b8ba13ef0 msgr2=0x555b8ba19830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:09.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 -- 172.21.15.66:0/3876824585 shutdown_connections 2023-07-21T22:34:09.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 -- 172.21.15.66:0/3876824585 wait complete. 2023-07-21T22:34:09.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.159+0000 7f881c272b80 1 -- start start 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 Processor -- start 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba1e930 0x555b8ba2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba231b0 0x555b8ba27df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8b90fac0 con 0x555b8ba1e930 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8b90f7c0 con 0x555b8ba23ac0 2023-07-21T22:34:09.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8b7f3920 con 0x555b8ba231b0 2023-07-21T22:34:09.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.160+0000 7f881c272b80 10 client.0 _ll_register_callbacks cb 0x555b8b90d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:09.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba283d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49546/0 (socket says 172.21.15.66:49546) 2023-07-21T22:34:09.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f880259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba1e930 0x555b8ba2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8801d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba231b0 0x555b8ba27df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 -- 172.21.15.66:0/3334753980 learned_addr learned my addr 172.21.15.66:0/3334753980 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 -- 172.21.15.66:0/3334753980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba231b0 msgr2=0x555b8ba27df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba231b0 0x555b8ba27df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 -- 172.21.15.66:0/3334753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba1e930 msgr2=0x555b8ba2ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba1e930 0x555b8ba2ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 -- 172.21.15.66:0/3334753980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f8009d50 con 0x555b8ba23ac0 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f880259c700 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba1e930 0x555b8ba2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:09.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba283d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87fc004680 tx=0x7f87fc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c99c1533a5fadaad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:09.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.161+0000 7f87e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:09.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3334753980 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00e040 con 0x555b8ba23ac0 2023-07-21T22:34:09.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b8b8c94f0 con 0x555b8ba23ac0 2023-07-21T22:34:09.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3334753980 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f87fc004db0 con 0x555b8ba23ac0 2023-07-21T22:34:09.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3334753980 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87fc003e80 con 0x555b8ba23ac0 2023-07-21T22:34:09.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f881c272b80 10 client.27353 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555b8b786c60 con 0x555b8ba23ac0 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3334753980 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b8b8c94f0 con 0x555b8ba23ac0 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.162+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3334753980 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f87fc023bd0 con 0x555b8ba23ac0 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f881c272b80 10 client.27353 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f881c272b80 10 client.27353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f881c272b80 20 client.27353 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:09.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f881c272b80 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba38420 0x555b8ba3a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f881c272b80 4 client.27353 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b8b968eb0 con 0x555b8ba38420 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f880259c700 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba38420 0x555b8ba3a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f880259c700 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba38420 0x555b8ba3a830 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87091d393298f388 server_cookie=919b7d33a805ec49 in_seq=0 out_seq=0 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.163+0000 7f87e9ffb700 10 client.27353 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3334753980 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555b8b968eb0 con 0x555b8ba38420 2023-07-21T22:34:09.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f87e9ffb700 10 client.27353 handle_command_reply: tid=0 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 client.27353 shutdown 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 msgr2=0x555b8ba283d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba283d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f87fc004680 tx=0x7f87fc004b40 comp rx=0 tx=0).stop 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 shutdown_connections 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba23ac0 0x555b8ba283d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba38420 0x555b8ba3a830 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba1e930 0x555b8ba2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 --2- 172.21.15.66:0/3334753980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba231b0 0x555b8ba27df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f87e9ffb700 0 client.27353 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:09.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.164+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 >> 172.21.15.66:0/3334753980 conn(0x555b8ba13ef0 msgr2=0x555b8ba19420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:09.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.165+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 shutdown_connections 2023-07-21T22:34:09.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.165+0000 7f881c272b80 1 -- 172.21.15.66:0/3334753980 wait complete. 2023-07-21T22:34:09.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.165+0000 7f881c272b80 20 client.27353 trim_cache size 0 max 16384 2023-07-21T22:34:09.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.182+0000 7f881c272b80 1 Processor -- start 2023-07-21T22:34:09.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.182+0000 7f881c272b80 1 -- start start 2023-07-21T22:34:09.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f881c272b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f881c272b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba2a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f881c272b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f881c272b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8ba22e40 con 0x555b8ba13ef0 2023-07-21T22:34:09.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f881c272b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8ba29140 con 0x555b8ba182c0 2023-07-21T22:34:09.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f881c272b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8ba1ae20 con 0x555b8ba1e930 2023-07-21T22:34:09.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f880259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f880259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba2f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49560/0 (socket says 172.21.15.66:49560) 2023-07-21T22:34:09.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f8802d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f880259c700 1 -- 172.21.15.66:0/2501796027 learned_addr learned my addr 172.21.15.66:0/2501796027 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:09.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.183+0000 7f8801d9b700 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba2a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f8802d9d700 1 -- 172.21.15.66:0/2501796027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 msgr2=0x555b8ba2f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f8802d9d700 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba2f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f8802d9d700 1 -- 172.21.15.66:0/2501796027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 msgr2=0x555b8ba2a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f8802d9d700 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba2a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f8802d9d700 1 -- 172.21.15.66:0/2501796027 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b8b7c4250 con 0x555b8ba1e930 2023-07-21T22:34:09.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f880259c700 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba2f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:09.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f8802d9d700 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba192d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f87fc012570 tx=0x7f87fc0045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c57aa1ac56311ac5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f87f17fa700 1 -- 172.21.15.66:0/2501796027 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00e070 con 0x555b8ba1e930 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f87f17fa700 1 -- 172.21.15.66:0/2501796027 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87fc00f040 con 0x555b8ba1e930 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.184+0000 7f87f17fa700 1 -- 172.21.15.66:0/2501796027 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0092a0 con 0x555b8ba1e930 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 -- 172.21.15.66:0/2501796027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 msgr2=0x555b8ba192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba192d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f87fc012570 tx=0x7f87fc0045d0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 -- 172.21.15.66:0/2501796027 shutdown_connections 2023-07-21T22:34:09.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba2f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba192d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 --2- 172.21.15.66:0/2501796027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba2a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 -- 172.21.15.66:0/2501796027 >> 172.21.15.66:0/2501796027 conn(0x555b8ba34a50 msgr2=0x555b8ba1c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 -- 172.21.15.66:0/2501796027 shutdown_connections 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 -- 172.21.15.66:0/2501796027 wait complete. 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.185+0000 7f881c272b80 1 -- start start 2023-07-21T22:34:09.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 Processor -- start 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba85fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba80310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8ba808f0 con 0x555b8ba13ef0 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8ba80a60 con 0x555b8ba182c0 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b8ba80c20 con 0x555b8ba1e930 2023-07-21T22:34:09.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f881c272b80 10 client.0 _ll_register_callbacks cb 0x7f87fc003dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f8801d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f8801d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba864f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49564/0 (socket says 172.21.15.66:49564) 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f880259c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba85fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.186+0000 7f8801d9b700 1 -- 172.21.15.66:0/3824181085 learned_addr learned my addr 172.21.15.66:0/3824181085 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba80310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 -- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 msgr2=0x555b8ba864f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba864f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 -- 172.21.15.66:0/3824181085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 msgr2=0x555b8ba85fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba85fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 -- 172.21.15.66:0/3824181085 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88041a4c80 con 0x555b8ba1e930 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f880259c700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba85fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f8802d9d700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba80310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f87fc000f80 tx=0x7f87fc016760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e272edde461075 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.187+0000 7f87e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3824181085 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00e040 con 0x555b8ba1e930 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3824181085 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f87fc016df0 con 0x555b8ba1e930 2023-07-21T22:34:09.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b8ba34e20 con 0x555b8ba1e930 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3824181085 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87fc009040 con 0x555b8ba1e930 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f881c272b80 10 client.27319 fetch_fsmap learned FSMap version 43 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x555b8b786c60 con 0x555b8ba1e930 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3824181085 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555b8ba34e20 con 0x555b8ba1e930 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3824181085 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 43) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f87fc03c020 con 0x555b8ba1e930 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f881c272b80 10 client.27319 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T22:34:09.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.188+0000 7f881c272b80 10 client.27319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:09.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f881c272b80 20 client.27319 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:09.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f881c272b80 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba231b0 0x555b8ba8f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:09.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f881c272b80 4 client.27319 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:34:09.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f87f80054e0 con 0x555b8ba231b0 2023-07-21T22:34:09.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f880259c700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba231b0 0x555b8ba8f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:09.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f880259c700 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba231b0 0x555b8ba8f960 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193a7a7c37957905 server_cookie=f8920fb29844b234 in_seq=0 out_seq=0 2023-07-21T22:34:09.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f87e9ffb700 10 client.27319 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:09.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f87e9ffb700 1 -- 172.21.15.66:0/3824181085 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f87f80054e0 con 0x555b8ba231b0 2023-07-21T22:34:09.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f87e9ffb700 10 client.27319 handle_command_reply: tid=0 2023-07-21T22:34:09.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.189+0000 7f881c272b80 1 client.27319 shutdown 2023-07-21T22:34:09.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 msgr2=0x555b8ba80310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba80310 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f87fc000f80 tx=0x7f87fc016760 comp rx=0 tx=0).stop 2023-07-21T22:34:09.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 shutdown_connections 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555b8ba231b0 0x555b8ba8f960 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555b8ba1e930 0x555b8ba80310 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555b8ba13ef0 0x555b8ba85fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f87e9ffb700 0 client.27319 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 --2- 172.21.15.66:0/3824181085 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b8ba182c0 0x555b8ba864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 >> 172.21.15.66:0/3824181085 conn(0x555b8ba34a50 msgr2=0x555b8ba192b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:09.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 shutdown_connections 2023-07-21T22:34:09.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 1 -- 172.21.15.66:0/3824181085 wait complete. 2023-07-21T22:34:09.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.190+0000 7f881c272b80 20 client.27319 trim_cache size 0 max 16384 2023-07-21T22:34:09.249 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:34:09.249 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: "bdfcffea-65aa-405f-8215-607153e0da7a": { 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "bdfcffea-65aa-405f-8215-607153e0da7a", 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:34:09.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:34:09.251 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:34:09.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f87ec0a1340 msgr2=0x7f87ec0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f87ec0a1340 0x7f87ec0a37f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0127f0 tx=0x7f87fc00b040 comp rx=0 tx=0).stop 2023-07-21T22:34:09.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 msgr2=0x7f8804197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 0x7f8804197e50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009810 tx=0x7f87f400fa00 comp rx=0 tx=0).stop 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 shutdown_connections 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88040ffad0 0x7f8804197e50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f87ec0a1340 0x7f87ec0a37f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f8804105ae0 0x7f880419c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 --2- 172.21.15.66:0/666939077 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8804100480 0x7f8804198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:09.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 >> 172.21.15.66:0/666939077 conn(0x7f88040fb5f0 msgr2=0x7f880410ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:09.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 shutdown_connections 2023-07-21T22:34:09.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:09.191+0000 7f88089ff700 1 -- 172.21.15.66:0/666939077 wait complete. 2023-07-21T22:34:09.256 INFO:tasks.cephfs.filesystem:scrub status for tag:bdfcffea-65aa-405f-8215-607153e0da7a - {'path': '/', 'tag': 'bdfcffea-65aa-405f-8215-607153e0da7a', 'options': 'recursive,force'} 2023-07-21T22:34:09.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 628450 5.47 MB/sec execute 576 sec latency 19.803 ms 2023-07-21T22:34:10.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:34:10.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:10.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:10.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: pgmap v396: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:34:10.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:10 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:34:10.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:10.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:10.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:10.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:10.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:10.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:10.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.451 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: pgmap v396: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:34:10.451 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:10 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:34:10.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:10.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:10.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:10.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: pgmap v396: 129 pgs: 129 active+clean; 638 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 261 op/s 2023-07-21T22:34:10.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:10 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:10.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 629480 5.47 MB/sec execute 577 sec latency 24.934 ms 2023-07-21T22:34:11.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 631013 5.47 MB/sec execute 578 sec latency 42.224 ms 2023-07-21T22:34:12.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:12 smithi104 ceph-mon[129021]: pgmap v397: 129 pgs: 129 active+clean; 645 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2023-07-21T22:34:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:12 smithi066 ceph-mon[126379]: pgmap v397: 129 pgs: 129 active+clean; 645 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2023-07-21T22:34:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:12 smithi149 ceph-mon[129268]: pgmap v397: 129 pgs: 129 active+clean; 645 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2023-07-21T22:34:12.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 632471 5.48 MB/sec execute 579 sec latency 19.884 ms 2023-07-21T22:34:13.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 633425 5.48 MB/sec execute 580 sec latency 19.881 ms 2023-07-21T22:34:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:13 smithi104 ceph-mon[129021]: pgmap v398: 129 pgs: 129 active+clean; 645 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2023-07-21T22:34:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:13 smithi104 ceph-mon[129021]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T22:34:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:13 smithi066 ceph-mon[126379]: pgmap v398: 129 pgs: 129 active+clean; 645 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2023-07-21T22:34:14.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:13 smithi066 ceph-mon[126379]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T22:34:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:13 smithi149 ceph-mon[129268]: pgmap v398: 129 pgs: 129 active+clean; 645 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 260 op/s 2023-07-21T22:34:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:13 smithi149 ceph-mon[129268]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T22:34:14.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 634691 5.48 MB/sec execute 581 sec latency 94.674 ms 2023-07-21T22:34:15.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 636099 5.49 MB/sec execute 582 sec latency 26.973 ms 2023-07-21T22:34:16.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:15 smithi104 ceph-mon[129021]: pgmap v400: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2023-07-21T22:34:16.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:15 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:34:16.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:15 smithi066 ceph-mon[126379]: pgmap v400: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2023-07-21T22:34:16.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:15 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:34:16.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:15 smithi149 ceph-mon[129268]: pgmap v400: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.3 MiB/s wr, 250 op/s 2023-07-21T22:34:16.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:15 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:34:16.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 637134 5.49 MB/sec execute 583 sec latency 33.218 ms 2023-07-21T22:34:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:17 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:34:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:17 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:34:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:17 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:17.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:17 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:34:17.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:17 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:34:17.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:17 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:17.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:17 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:34:17.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:17 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:34:17.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:17 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:17.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 638637 5.49 MB/sec execute 584 sec latency 83.423 ms 2023-07-21T22:34:18.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:18 smithi104 ceph-mon[129021]: pgmap v401: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2023-07-21T22:34:18.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:18 smithi066 ceph-mon[126379]: pgmap v401: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2023-07-21T22:34:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:18 smithi149 ceph-mon[129268]: pgmap v401: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2023-07-21T22:34:18.575 INFO:tasks.workunit.client.0.smithi066.stdout: 1 639850 5.49 MB/sec execute 585 sec latency 13.117 ms 2023-07-21T22:34:19.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 640948 5.49 MB/sec execute 586 sec latency 16.671 ms 2023-07-21T22:34:20.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:20 smithi104 ceph-mon[129021]: pgmap v402: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2023-07-21T22:34:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:20 smithi066 ceph-mon[126379]: pgmap v402: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2023-07-21T22:34:20.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:20 smithi149 ceph-mon[129268]: pgmap v402: 129 pgs: 129 active+clean; 655 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 223 op/s 2023-07-21T22:34:20.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 642889 5.51 MB/sec execute 587 sec latency 19.448 ms 2023-07-21T22:34:21.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 643747 5.50 MB/sec execute 588 sec latency 48.449 ms 2023-07-21T22:34:22.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:22 smithi104 ceph-mon[129021]: pgmap v403: 129 pgs: 129 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-07-21T22:34:22.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:22 smithi066 ceph-mon[126379]: pgmap v403: 129 pgs: 129 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-07-21T22:34:22.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:22 smithi149 ceph-mon[129268]: pgmap v403: 129 pgs: 129 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-07-21T22:34:22.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 645304 5.50 MB/sec execute 589 sec latency 35.750 ms 2023-07-21T22:34:23.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 646756 5.51 MB/sec execute 590 sec latency 48.826 ms 2023-07-21T22:34:24.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:24 smithi104 ceph-mon[129021]: pgmap v404: 129 pgs: 129 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-07-21T22:34:24.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:24 smithi066 ceph-mon[126379]: pgmap v404: 129 pgs: 129 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-07-21T22:34:24.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:24 smithi149 ceph-mon[129268]: pgmap v404: 129 pgs: 129 active+clean; 668 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.2 MiB/s wr, 232 op/s 2023-07-21T22:34:24.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 647844 5.51 MB/sec execute 591 sec latency 20.832 ms 2023-07-21T22:34:25.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 648959 5.51 MB/sec execute 592 sec latency 189.413 ms 2023-07-21T22:34:26.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:26 smithi104 ceph-mon[129021]: pgmap v405: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 299 op/s 2023-07-21T22:34:26.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:26 smithi066 ceph-mon[126379]: pgmap v405: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 299 op/s 2023-07-21T22:34:26.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:26 smithi149 ceph-mon[129268]: pgmap v405: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 299 op/s 2023-07-21T22:34:26.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 650311 5.52 MB/sec execute 593 sec latency 53.472 ms 2023-07-21T22:34:27.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 651344 5.52 MB/sec execute 594 sec latency 118.341 ms 2023-07-21T22:34:28.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:28 smithi104 ceph-mon[129021]: pgmap v406: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2023-07-21T22:34:28.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:28 smithi066 ceph-mon[126379]: pgmap v406: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2023-07-21T22:34:28.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:28 smithi149 ceph-mon[129268]: pgmap v406: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2023-07-21T22:34:28.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 653231 5.52 MB/sec execute 595 sec latency 15.768 ms 2023-07-21T22:34:29.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 654248 5.52 MB/sec execute 596 sec latency 15.671 ms 2023-07-21T22:34:30.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:30 smithi104 ceph-mon[129021]: pgmap v407: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2023-07-21T22:34:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:30 smithi066 ceph-mon[126379]: pgmap v407: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2023-07-21T22:34:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:30 smithi149 ceph-mon[129268]: pgmap v407: 129 pgs: 129 active+clean; 678 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.2 MiB/s wr, 207 op/s 2023-07-21T22:34:30.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 655700 5.52 MB/sec execute 597 sec latency 18.801 ms 2023-07-21T22:34:31.577 INFO:tasks.workunit.client.0.smithi066.stdout: 1 657193 5.53 MB/sec execute 598 sec latency 47.710 ms 2023-07-21T22:34:32.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:32 smithi104 ceph-mon[129021]: pgmap v408: 129 pgs: 129 active+clean; 688 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.7 MiB/s wr, 320 op/s 2023-07-21T22:34:32.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:32 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:32.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:32 smithi066 ceph-mon[126379]: pgmap v408: 129 pgs: 129 active+clean; 688 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.7 MiB/s wr, 320 op/s 2023-07-21T22:34:32.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:32 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:32.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:32 smithi149 ceph-mon[129268]: pgmap v408: 129 pgs: 129 active+clean; 688 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.7 MiB/s wr, 320 op/s 2023-07-21T22:34:32.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:32 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:32.576 INFO:tasks.workunit.client.0.smithi066.stdout: 1 657972 5.53 MB/sec execute 599 sec latency 29.126 ms 2023-07-21T22:34:33.577 INFO:tasks.workunit.client.0.smithi066.stdout: 1 cleanup 600 sec 2023-07-21T22:34:33.818 INFO:tasks.workunit.client.0.smithi066.stdout: 0 cleanup 600 sec 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: Operation Count AvgLat MaxLat 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: ---------------------------------------- 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: NTCreateX 105920 2.232 4990.120 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: Close 77835 0.005 0.167 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: Rename 4486 19.376 4974.731 2023-07-21T22:34:33.819 INFO:tasks.workunit.client.0.smithi066.stdout: Unlink 21366 3.059 12867.995 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Deltree 2 95.726 120.822 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Mkdir 1 0.015 0.015 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Qpathinfo 96053 1.348 4919.615 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Qfileinfo 16816 0.002 0.041 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Qfsinfo 17584 0.297 11.003 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Sfileinfo 8646 0.082 14.414 2023-07-21T22:34:33.820 INFO:tasks.workunit.client.0.smithi066.stdout: Find 37116 0.653 14333.558 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout: WriteX 52714 0.159 96.825 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout: ReadX 165966 0.054 482.075 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout: LockX 344 0.775 10.850 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout: UnlockX 344 0.828 11.015 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout: Flush 7424 4.287 201.934 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-07-21T22:34:33.821 INFO:tasks.workunit.client.0.smithi066.stdout:Throughput 5.52684 MB/sec 1 clients 1 procs max_latency=14333.569 ms 2023-07-21T22:34:33.822 INFO:tasks.workunit.client.0.smithi066.stderr:+ dbench 10 2023-07-21T22:34:33.822 INFO:tasks.workunit.client.0.smithi066.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2023-07-21T22:34:33.822 INFO:tasks.workunit.client.0.smithi066.stdout: 2023-07-21T22:34:33.822 INFO:tasks.workunit.client.0.smithi066.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2023-07-21T22:34:33.823 INFO:tasks.workunit.client.0.smithi066.stdout:9 of 10 processes prepared for launch 0 sec 2023-07-21T22:34:33.923 INFO:tasks.workunit.client.0.smithi066.stdout:10 of 10 processes prepared for launch 0 sec 2023-07-21T22:34:33.924 INFO:tasks.workunit.client.0.smithi066.stdout:releasing clients 2023-07-21T22:34:34.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:34 smithi104 ceph-mon[129021]: pgmap v409: 129 pgs: 129 active+clean; 688 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.0 MiB/s wr, 217 op/s 2023-07-21T22:34:34.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:34 smithi066 ceph-mon[126379]: pgmap v409: 129 pgs: 129 active+clean; 688 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.0 MiB/s wr, 217 op/s 2023-07-21T22:34:34.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:34 smithi149 ceph-mon[129268]: pgmap v409: 129 pgs: 129 active+clean; 688 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.0 MiB/s wr, 217 op/s 2023-07-21T22:34:34.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 402 142.89 MB/sec warmup 1 sec latency 514.855 ms 2023-07-21T22:34:35.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 892 102.01 MB/sec warmup 2 sec latency 1514.941 ms 2023-07-21T22:34:36.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:36 smithi104 ceph-mon[129021]: pgmap v410: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2023-07-21T22:34:36.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:36 smithi066 ceph-mon[126379]: pgmap v410: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2023-07-21T22:34:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:36 smithi149 ceph-mon[129268]: pgmap v410: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 314 op/s 2023-07-21T22:34:36.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 1544 80.89 MB/sec warmup 3 sec latency 2515.006 ms 2023-07-21T22:34:37.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:37 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:37.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:37 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:37.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:37 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:37.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 2327 67.36 MB/sec warmup 4 sec latency 3515.049 ms 2023-07-21T22:34:38.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:38 smithi104 ceph-mon[129021]: pgmap v411: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-21T22:34:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:38 smithi066 ceph-mon[126379]: pgmap v411: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-21T22:34:38.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:38 smithi149 ceph-mon[129268]: pgmap v411: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-21T22:34:38.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 3060 65.67 MB/sec warmup 5 sec latency 4515.117 ms 2023-07-21T22:34:39.257 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:34:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 -- 172.21.15.66:0/3248399757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f00ffad0 msgr2=0x7f66f00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3248399757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f00ffad0 0x7f66f00fff40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 -- 172.21.15.66:0/3248399757 shutdown_connections 2023-07-21T22:34:39.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3248399757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f0100480 0x7f66f010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3248399757 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0105ae0 0x7f66f0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3248399757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f00ffad0 0x7f66f00fff40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 -- 172.21.15.66:0/3248399757 >> 172.21.15.66:0/3248399757 conn(0x7f66f00fb5f0 msgr2=0x7f66f00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.487+0000 7f66f5bff700 1 -- 172.21.15.66:0/3248399757 shutdown_connections 2023-07-21T22:34:39.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.488+0000 7f66f5bff700 1 -- 172.21.15.66:0/3248399757 wait complete. 2023-07-21T22:34:39.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.488+0000 7f66f5bff700 1 Processor -- start 2023-07-21T22:34:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 -- start start 2023-07-21T22:34:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 0x7f66f0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 0x7f66f0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f0105ae0 0x7f66f01a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0073580 con 0x7f66f0100480 2023-07-21T22:34:39.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01a42c0 con 0x7f66f0105ae0 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66f5bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01a4400 con 0x7f66f00ffad0 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 0x7f66f0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 0x7f66f0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 0x7f66f0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:60984/0 (socket says 172.21.15.66:60984) 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66effff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f0105ae0 0x7f66f01a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66ef7fe700 1 -- 172.21.15.66:0/3162428900 learned_addr learned my addr 172.21.15.66:0/3162428900 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.489+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 0x7f66f0072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48618/0 (socket says 172.21.15.66:48618) 2023-07-21T22:34:39.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66ef7fe700 1 -- 172.21.15.66:0/3162428900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f0105ae0 msgr2=0x7f66f01a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f0105ae0 0x7f66f01a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66ef7fe700 1 -- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 msgr2=0x7f66f0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 0x7f66f0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66ef7fe700 1 -- 172.21.15.66:0/3162428900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80095f0 con 0x7f66f00ffad0 2023-07-21T22:34:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66e7fff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 0x7f66f0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:39.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.490+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 0x7f66f0072810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e9a0 tx=0x7f66e000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.491+0000 7f66ed7fa700 1 -- 172.21.15.66:0/3162428900 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0009790 con 0x7f66f00ffad0 2023-07-21T22:34:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.491+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f01a4660 con 0x7f66f00ffad0 2023-07-21T22:34:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.491+0000 7f66ed7fa700 1 -- 172.21.15.66:0/3162428900 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e0010430 con 0x7f66f00ffad0 2023-07-21T22:34:39.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.491+0000 7f66ed7fa700 1 -- 172.21.15.66:0/3162428900 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0018680 con 0x7f66f00ffad0 2023-07-21T22:34:39.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.491+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f01a4b90 con 0x7f66f00ffad0 2023-07-21T22:34:39.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.494+0000 7f66ed7fa700 1 -- 172.21.15.66:0/3162428900 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f66e00105a0 con 0x7f66f00ffad0 2023-07-21T22:34:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 Processor -- start 2023-07-21T22:34:39.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 -- start start 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c41b0 0x55d04d1c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1bf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d0c4cb0 con 0x55d04d1c4ac0 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d117a40 con 0x55d04d1bf930 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f67095a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d110ed0 con 0x55d04d1c41b0 2023-07-21T22:34:39.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f66ed7fa700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f66dc0a1340 0x7f66dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.496+0000 7f66ed7fa700 1 -- 172.21.15.66:0/3162428900 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f66e0014070 con 0x7f66f00ffad0 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66effff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c41b0 0x55d04d1c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47510/0 (socket says 172.21.15.66:47510) 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1bf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 -- 172.21.15.66:0/1585661920 learned_addr learned my addr 172.21.15.66:0/1585661920 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:39.546 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66effff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f66dc0a1340 0x7f66dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 -- 172.21.15.66:0/1585661920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c41b0 msgr2=0x55d04d1c4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c41b0 0x55d04d1c4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 -- 172.21.15.66:0/1585661920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 msgr2=0x55d04d1bf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1bf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 -- 172.21.15.66:0/1585661920 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d04cf3ede0 con 0x55d04d1bf930 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1bf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66e7fff700 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1bfda0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d800b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57bfdce20b7edfef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.547 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.497+0000 7f66effff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f66dc0a1340 0x7f66dc0a37f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009ec0 tx=0x7f66e800b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.498+0000 7f66e5ffb700 1 -- 172.21.15.66:0/1585661920 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d801d070 con 0x55d04d1bf930 2023-07-21T22:34:39.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.498+0000 7f66e5ffb700 1 -- 172.21.15.66:0/1585661920 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d801e040 con 0x55d04d1bf930 2023-07-21T22:34:39.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.498+0000 7f66e5ffb700 1 -- 172.21.15.66:0/1585661920 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d8004d50 con 0x55d04d1bf930 2023-07-21T22:34:39.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.498+0000 7f67095a8b80 1 -- 172.21.15.66:0/1585661920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 msgr2=0x55d04d1bfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.548 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.498+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1bfda0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d800b610 comp rx=0 tx=0).stop 2023-07-21T22:34:39.550 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 -- 172.21.15.66:0/1585661920 shutdown_connections 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1bfda0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c41b0 0x55d04d1c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1585661920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1bf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 -- 172.21.15.66:0/1585661920 >> 172.21.15.66:0/1585661920 conn(0x55d04d1b4ef0 msgr2=0x55d04d1ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 -- 172.21.15.66:0/1585661920 shutdown_connections 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 -- 172.21.15.66:0/1585661920 wait complete. 2023-07-21T22:34:39.551 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.499+0000 7f67095a8b80 1 -- start start 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 Processor -- start 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d0b0ac0 con 0x55d04d1c4ac0 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d0b07c0 con 0x55d04d1c41b0 2023-07-21T22:34:39.552 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04cf94920 con 0x55d04d1bf930 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f67095a8b80 10 client.0 _ll_register_callbacks cb 0x55d04d0ae760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32770/0 (socket says 172.21.15.66:32770) 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f66ef7fe700 1 -- 172.21.15.66:0/950872176 learned_addr learned my addr 172.21.15.66:0/950872176 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.500+0000 7f66effff700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ef7fe700 1 -- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 msgr2=0x55d04d1c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.553 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ef7fe700 1 -- 172.21.15.66:0/950872176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 msgr2=0x55d04d1c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ef7fe700 1 -- 172.21.15.66:0/950872176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f010d210 con 0x55d04d1bf930 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66e7fff700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1cbbb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ca5f0 tx=0x7f66e001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af18e956dc9db8f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ceffd700 1 -- 172.21.15.66:0/950872176 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0069070 con 0x55d04d1bf930 2023-07-21T22:34:39.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ceffd700 1 -- 172.21.15.66:0/950872176 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f66e0099f10 con 0x55d04d1bf930 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d04d06a4f0 con 0x55d04d1bf930 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f66ceffd700 1 -- 172.21.15.66:0/950872176 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66e009df40 con 0x55d04d1bf930 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f67095a8b80 10 client.27331 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.501+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d04cf27c60 con 0x55d04d1bf930 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f66ceffd700 1 -- 172.21.15.66:0/950872176 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e0093030 con 0x55d04d1bf930 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f66ceffd700 1 -- 172.21.15.66:0/950872176 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f66e00a1ed0 con 0x55d04d1bf930 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f67095a8b80 10 client.27331 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:39.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f67095a8b80 10 client.27331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f67095a8b80 20 client.27331 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f67095a8b80 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d1d9310 0x55d04d1db720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f67095a8b80 4 client.27331 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d04d109eb0 con 0x55d04d1d9310 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.502+0000 7f66e7fff700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d1d9310 0x55d04d1db720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.503+0000 7f66e7fff700 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d1d9310 0x55d04d1db720 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33553dba753aff5d server_cookie=8eb92f3b04b535ff in_seq=0 out_seq=0 2023-07-21T22:34:39.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.503+0000 7f66ceffd700 10 client.27331 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f66ceffd700 1 -- 172.21.15.66:0/950872176 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d04d109eb0 con 0x55d04d1d9310 2023-07-21T22:34:39.557 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f66ceffd700 10 client.27331 handle_command_reply: tid=0 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 client.27331 shutdown 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 msgr2=0x55d04d1cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1cbbb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ca5f0 tx=0x7f66e001f000 comp rx=0 tx=0).stop 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 shutdown_connections 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d1d9310 0x55d04d1db720 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.558 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1cbbb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f66ceffd700 0 client.27331 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 --2- 172.21.15.66:0/950872176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 >> 172.21.15.66:0/950872176 conn(0x55d04d1b4ef0 msgr2=0x55d04d1ba420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 shutdown_connections 2023-07-21T22:34:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 1 -- 172.21.15.66:0/950872176 wait complete. 2023-07-21T22:34:39.559 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.505+0000 7f67095a8b80 20 client.27331 trim_cache size 0 max 16384 2023-07-21T22:34:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.523+0000 7f67095a8b80 1 Processor -- start 2023-07-21T22:34:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.523+0000 7f67095a8b80 1 -- start start 2023-07-21T22:34:39.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f67095a8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d1d5dd0 con 0x55d04d1c41b0 2023-07-21T22:34:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f67095a8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d1bb320 con 0x55d04d1bf930 2023-07-21T22:34:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f67095a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d1c90e0 con 0x55d04d1c4ac0 2023-07-21T22:34:39.572 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47534/0 (socket says 172.21.15.66:47534) 2023-07-21T22:34:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 -- 172.21.15.66:0/2913532125 learned_addr learned my addr 172.21.15.66:0/2913532125 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66effff700 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66e7fff700 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 -- 172.21.15.66:0/2913532125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c4ac0 msgr2=0x55d04d1c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 -- 172.21.15.66:0/2913532125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c41b0 msgr2=0x55d04d1c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.524+0000 7f66ef7fe700 1 -- 172.21.15.66:0/2913532125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d04cf65250 con 0x55d04d1bf930 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.525+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1cb6e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e970 tx=0x7f66e009a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a6a0903c57c5e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.525+0000 7f66e67fc700 1 -- 172.21.15.66:0/2913532125 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0069070 con 0x55d04d1bf930 2023-07-21T22:34:39.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.525+0000 7f66e67fc700 1 -- 172.21.15.66:0/2913532125 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e0093070 con 0x55d04d1bf930 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.525+0000 7f66e67fc700 1 -- 172.21.15.66:0/2913532125 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00a2130 con 0x55d04d1bf930 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.525+0000 7f67095a8b80 1 -- 172.21.15.66:0/2913532125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 msgr2=0x55d04d1cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.525+0000 7f67095a8b80 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1cb6e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e970 tx=0x7f66e009a4d0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 -- 172.21.15.66:0/2913532125 shutdown_connections 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1bf930 0x55d04d1cb6e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1c4ac0 0x55d04d1c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 --2- 172.21.15.66:0/2913532125 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c41b0 0x55d04d1c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 -- 172.21.15.66:0/2913532125 >> 172.21.15.66:0/2913532125 conn(0x55d04d1b4ef0 msgr2=0x55d04d1bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 -- 172.21.15.66:0/2913532125 shutdown_connections 2023-07-21T22:34:39.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 -- 172.21.15.66:0/2913532125 wait complete. 2023-07-21T22:34:39.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 -- start start 2023-07-21T22:34:39.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.526+0000 7f67095a8b80 1 Processor -- start 2023-07-21T22:34:39.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d1cf810 con 0x55d04d1c4ac0 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d1cf980 con 0x55d04d1c41b0 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d04d22d200 con 0x55d04d1bf930 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f67095a8b80 10 client.0 _ll_register_callbacks cb 0x7f66e0099ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32786/0 (socket says 172.21.15.66:32786) 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66e7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47538/0 (socket says 172.21.15.66:47538) 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66ef7fe700 1 -- 172.21.15.66:0/1652249994 learned_addr learned my addr 172.21.15.66:0/1652249994 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66effff700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66ef7fe700 1 -- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 msgr2=0x55d04d1ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.527+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ef7fe700 1 -- 172.21.15.66:0/1652249994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 msgr2=0x55d04d1cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:39.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ef7fe700 1 -- 172.21.15.66:0/1652249994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d04cf27c60 con 0x55d04d1bf930 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66e7fff700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ef7fe700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1d0640 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66e0096980 tx=0x7f66e00a2100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8108c6f4ca5af02 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ceffd700 1 -- 172.21.15.66:0/1652249994 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0069070 con 0x55d04d1bf930 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ceffd700 1 -- 172.21.15.66:0/1652249994 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f66e009df40 con 0x55d04d1bf930 2023-07-21T22:34:39.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66d801e040 con 0x55d04d1bf930 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f66ceffd700 1 -- 172.21.15.66:0/1652249994 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66e001c1c0 con 0x55d04d1bf930 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f67095a8b80 10 client.27337 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.528+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55d04cf3d2b0 con 0x55d04d1bf930 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f66ceffd700 1 -- 172.21.15.66:0/1652249994 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66d801e040 con 0x55d04d1bf930 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f66ceffd700 1 -- 172.21.15.66:0/1652249994 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f66e0098f00 con 0x55d04d1bf930 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f67095a8b80 10 client.27337 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:39.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f67095a8b80 10 client.27337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f67095a8b80 20 client.27337 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d22e430 0x55d04d230810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f67095a8b80 4 client.27337 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d04d0cc050 con 0x55d04d22e430 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f66e7fff700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d22e430 0x55d04d230810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.529+0000 7f66e7fff700 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d22e430 0x55d04d230810 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3385b7905342b9 server_cookie=83c3ae6338d0295 in_seq=0 out_seq=0 2023-07-21T22:34:39.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.530+0000 7f66ceffd700 10 client.27337 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f66ceffd700 1 -- 172.21.15.66:0/1652249994 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d04d0cc050 con 0x55d04d22e430 2023-07-21T22:34:39.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f66ceffd700 10 client.27337 handle_command_reply: tid=0 2023-07-21T22:34:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 client.27337 shutdown 2023-07-21T22:34:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 msgr2=0x55d04d1d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1d0640 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66e0096980 tx=0x7f66e00a2100 comp rx=0 tx=0).stop 2023-07-21T22:34:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 shutdown_connections 2023-07-21T22:34:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d04d22e430 0x55d04d230810 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d04d1c4ac0 0x55d04d1cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f66ceffd700 0 client.27337 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d04d1bf930 0x55d04d1d0640 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f66e0096980 tx=0x7f66e00a2100 comp rx=0 tx=0).stop 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 --2- 172.21.15.66:0/1652249994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d04d1c41b0 0x55d04d1ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 >> 172.21.15.66:0/1652249994 conn(0x55d04d1b4ef0 msgr2=0x55d04d1bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 shutdown_connections 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 1 -- 172.21.15.66:0/1652249994 wait complete. 2023-07-21T22:34:39.587 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.535+0000 7f67095a8b80 20 client.27337 trim_cache size 0 max 16384 2023-07-21T22:34:39.588 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:34:39.588 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:34:39.588 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:34:39.588 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:34:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f66dc0a1340 msgr2=0x7f66dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f66dc0a1340 0x7f66dc0a37f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009ec0 tx=0x7f66e800b040 comp rx=0 tx=0).stop 2023-07-21T22:34:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 msgr2=0x7f66f0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.589 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 0x7f66f0072810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e9a0 tx=0x7f66e000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 shutdown_connections 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66f0105ae0 0x7f66f01a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f66dc0a1340 0x7f66dc0a37f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f00ffad0 0x7f66f0072810 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 --2- 172.21.15.66:0/3162428900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f66f0100480 0x7f66f0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 >> 172.21.15.66:0/3162428900 conn(0x7f66f00fb5f0 msgr2=0x7f66f00fbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.590 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 shutdown_connections 2023-07-21T22:34:39.591 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.536+0000 7f66f5bff700 1 -- 172.21.15.66:0/3162428900 wait complete. 2023-07-21T22:34:39.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:34:39.603 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:34:39.815 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1892012175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a410a070 msgr2=0x7f70a410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1892012175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a410a070 0x7f70a410c4b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b0a0 tx=0x7f709c00b3b0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1892012175 shutdown_connections 2023-07-21T22:34:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1892012175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a410a070 0x7f70a410c4b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1892012175 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.816 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1892012175 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a41015b0 0x7f70a4109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1892012175 >> 172.21.15.66:0/1892012175 conn(0x7f70a40fa490 msgr2=0x7f70a40fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1892012175 shutdown_connections 2023-07-21T22:34:39.817 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.768+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1892012175 wait complete. 2023-07-21T22:34:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.770+0000 7f70ab7ff700 1 Processor -- start 2023-07-21T22:34:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.770+0000 7f70ab7ff700 1 -- start start 2023-07-21T22:34:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.770+0000 7f70ab7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a40fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.818 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.770+0000 7f70ab7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a41015b0 0x7f70a40fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.770+0000 7f70ab7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a410a070 0x7f70a41a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70ab7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40ff660 con 0x7f70a410a070 2023-07-21T22:34:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70ab7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a43c0 con 0x7f70a41015b0 2023-07-21T22:34:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70ab7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a4500 con 0x7f70a4100ca0 2023-07-21T22:34:39.819 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a40fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a40fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32810/0 (socket says 172.21.15.66:32810) 2023-07-21T22:34:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 -- 172.21.15.66:0/1201313495 learned_addr learned my addr 172.21.15.66:0/1201313495 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a8d9a700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a41015b0 0x7f70a40fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.820 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a9d9c700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a410a070 0x7f70a41a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 -- 172.21.15.66:0/1201313495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a41015b0 msgr2=0x7f70a40fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a41015b0 0x7f70a40fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 -- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a410a070 msgr2=0x7f70a41a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:39.821 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a410a070 0x7f70a41a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 -- 172.21.15.66:0/1201313495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009d00 con 0x7f70a4100ca0 2023-07-21T22:34:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.771+0000 7f70a959b700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a40fe8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e9a0 tx=0x7f70a000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.772+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0009790 con 0x7f70a4100ca0 2023-07-21T22:34:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.772+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70a0010430 con 0x7f70a4100ca0 2023-07-21T22:34:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.772+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0018680 con 0x7f70a4100ca0 2023-07-21T22:34:39.822 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.772+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a41a4760 con 0x7f70a4100ca0 2023-07-21T22:34:39.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.772+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a41a4c40 con 0x7f70a4100ca0 2023-07-21T22:34:39.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.773+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4104900 con 0x7f70a4100ca0 2023-07-21T22:34:39.823 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.775+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f70a00105a0 con 0x7f70a4100ca0 2023-07-21T22:34:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.776+0000 7f709a7fc700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70900a1380 0x7f70900a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.777+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f70a0014070 con 0x7f70a4100ca0 2023-07-21T22:34:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.777+0000 7f70a8d9a700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70900a1380 0x7f70900a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:39.825 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.777+0000 7f70a8d9a700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70900a1380 0x7f70900a3830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70a41000e0 tx=0x7f7094009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:39.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:39.778+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f70a00962b0 con 0x7f70a4100ca0 2023-07-21T22:34:39.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 3448 60.73 MB/sec warmup 6 sec latency 5515.166 ms 2023-07-21T22:34:40.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.053+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70a4067300 con 0x7f70a4100ca0 2023-07-21T22:34:40.101 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.054+0000 7f709a7fc700 1 -- 172.21.15.66:0/1201313495 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+10824 (secure 0 0 0) 0x7f70a00998d0 con 0x7f70a4100ca0 2023-07-21T22:34:40.101 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:34:40.102 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:34:36.030104+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:34:40.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70900a1380 msgr2=0x7f70900a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70900a1380 0x7f70900a3830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f70a41000e0 tx=0x7f7094009260 comp rx=0 tx=0).stop 2023-07-21T22:34:40.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 msgr2=0x7f70a40fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a40fe8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f70a000e9a0 tx=0x7f70a000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 shutdown_connections 2023-07-21T22:34:40.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70900a1380 0x7f70900a3830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f70a410a070 0x7f70a41a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70a4100ca0 0x7f70a40fe8f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 --2- 172.21.15.66:0/1201313495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a41015b0 0x7f70a40fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 >> 172.21.15.66:0/1201313495 conn(0x7f70a40fa490 msgr2=0x7f70a4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 shutdown_connections 2023-07-21T22:34:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.056+0000 7f70ab7ff700 1 -- 172.21.15.66:0/1201313495 wait complete. 2023-07-21T22:34:40.105 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 46 2023-07-21T22:34:40.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:34:40.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.320+0000 7fb03e5ff700 1 -- 172.21.15.66:0/475894005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380731c0 msgr2=0x7fb0381105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.320+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/475894005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380731c0 0x7fb0381105c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb0200098f0 tx=0x7fb020009c00 comp rx=0 tx=0).stop 2023-07-21T22:34:40.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.321+0000 7fb03e5ff700 1 -- 172.21.15.66:0/475894005 shutdown_connections 2023-07-21T22:34:40.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.321+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/475894005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb038110b00 0x7fb038112ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.321+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/475894005 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0380728b0 0x7fb038072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.321+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/475894005 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380731c0 0x7fb0381105c0 secure :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb0200098f0 tx=0x7fb020009c00 comp rx=0 tx=0).stop 2023-07-21T22:34:40.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.321+0000 7fb03e5ff700 1 -- 172.21.15.66:0/475894005 >> 172.21.15.66:0/475894005 conn(0x7fb0380fb5f0 msgr2=0x7fb0380fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.321+0000 7fb03e5ff700 1 -- 172.21.15.66:0/475894005 shutdown_connections 2023-07-21T22:34:40.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:40 smithi104 ceph-mon[129021]: pgmap v412: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-21T22:34:40.372 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.322+0000 7fb03e5ff700 1 -- 172.21.15.66:0/475894005 wait complete. 2023-07-21T22:34:40.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.323+0000 7fb03e5ff700 1 Processor -- start 2023-07-21T22:34:40.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 -- start start 2023-07-21T22:34:40.373 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380728b0 0x7fb038071b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb038110b00 0x7fb038072090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 0x7fb03806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03806b7e0 con 0x7fb0380728b0 2023-07-21T22:34:40.374 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03806b950 con 0x7fb03806ab20 2023-07-21T22:34:40.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.324+0000 7fb03e5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03806f6a0 con 0x7fb038110b00 2023-07-21T22:34:40.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 0x7fb03806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 0x7fb03806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47572/0 (socket says 172.21.15.66:47572) 2023-07-21T22:34:40.375 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 -- 172.21.15.66:0/3328509272 learned_addr learned my addr 172.21.15.66:0/3328509272 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.376 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 -- 172.21.15.66:0/3328509272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb038110b00 msgr2=0x7fb038072090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T22:34:40.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb038110b00 0x7fb038072090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 -- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380728b0 msgr2=0x7fb038071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380728b0 0x7fb038071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.377 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.325+0000 7fb03cb9c700 1 -- 172.21.15.66:0/3328509272 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb020009520 con 0x7fb03806ab20 2023-07-21T22:34:40.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.326+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 0x7fb03806afc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb03000d770 tx=0x7fb03000db30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.327+0000 7fb035ffb700 1 -- 172.21.15.66:0/3328509272 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030004a30 con 0x7fb03806ab20 2023-07-21T22:34:40.378 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.327+0000 7fb035ffb700 1 -- 172.21.15.66:0/3328509272 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0300054e0 con 0x7fb03806ab20 2023-07-21T22:34:40.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.327+0000 7fb035ffb700 1 -- 172.21.15.66:0/3328509272 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030009c30 con 0x7fb03806ab20 2023-07-21T22:34:40.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.327+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03806f920 con 0x7fb03806ab20 2023-07-21T22:34:40.379 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.327+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03806fde0 con 0x7fb03806ab20 2023-07-21T22:34:40.380 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.333+0000 7fb035ffb700 1 -- 172.21.15.66:0/3328509272 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb030005020 con 0x7fb03806ab20 2023-07-21T22:34:40.381 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.334+0000 7fb051f94b80 1 Processor -- start 2023-07-21T22:34:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.334+0000 7fb051f94b80 1 -- start start 2023-07-21T22:34:40.382 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.335+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 0x5592bf42c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.335+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42cac0 0x5592bf427350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.335+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf427930 0x5592bf427da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.335+0000 7fb051f94b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf32ccb0 con 0x5592bf427930 2023-07-21T22:34:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.335+0000 7fb051f94b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf37fa40 con 0x5592bf42cac0 2023-07-21T22:34:40.383 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.335+0000 7fb051f94b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf378ed0 con 0x5592bf42c1b0 2023-07-21T22:34:40.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 0x5592bf42c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 0x5592bf42c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32838/0 (socket says 172.21.15.66:32838) 2023-07-21T22:34:40.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 -- 172.21.15.66:0/4091511377 learned_addr learned my addr 172.21.15.66:0/4091511377 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf427930 0x5592bf427da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 -- 172.21.15.66:0/4091511377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42cac0 msgr2=0x5592bf427350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.384 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42cac0 0x5592bf427350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 -- 172.21.15.66:0/4091511377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf427930 msgr2=0x5592bf427da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf427930 0x5592bf427da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.385 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.336+0000 7fb02f7fe700 1 -- 172.21.15.66:0/4091511377 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592bf1a6de0 con 0x5592bf42c1b0 2023-07-21T22:34:40.386 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.337+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 0x5592bf42c580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb038073970 tx=0x7fb02000bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=292328bae202248d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.337+0000 7fb02e7fc700 1 -- 172.21.15.66:0/4091511377 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000f6c0 con 0x5592bf42c1b0 2023-07-21T22:34:40.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.337+0000 7fb035ffb700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb0240a1490 0x7fb0240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.387 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.337+0000 7fb02e7fc700 1 -- 172.21.15.66:0/4091511377 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0200044e0 con 0x5592bf42c1b0 2023-07-21T22:34:40.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.337+0000 7fb02e7fc700 1 -- 172.21.15.66:0/4091511377 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020005650 con 0x5592bf42c1b0 2023-07-21T22:34:40.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.337+0000 7fb035ffb700 1 -- 172.21.15.66:0/3328509272 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb0300cc780 con 0x7fb03806ab20 2023-07-21T22:34:40.388 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.338+0000 7fb051f94b80 1 -- 172.21.15.66:0/4091511377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 msgr2=0x5592bf42c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.338+0000 7fb051f94b80 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 0x5592bf42c580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb038073970 tx=0x7fb02000bef0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.389 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.338+0000 7fb037fff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb0240a1490 0x7fb0240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.340+0000 7fb051f94b80 1 -- 172.21.15.66:0/4091511377 shutdown_connections 2023-07-21T22:34:40.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.340+0000 7fb051f94b80 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf427930 0x5592bf427da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.340+0000 7fb051f94b80 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf42c1b0 0x5592bf42c580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.340+0000 7fb051f94b80 1 --2- 172.21.15.66:0/4091511377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42cac0 0x5592bf427350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.340+0000 7fb051f94b80 1 -- 172.21.15.66:0/4091511377 >> 172.21.15.66:0/4091511377 conn(0x5592bf41cef0 msgr2=0x5592bf422830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.391 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.340+0000 7fb051f94b80 1 -- 172.21.15.66:0/4091511377 shutdown_connections 2023-07-21T22:34:40.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.341+0000 7fb051f94b80 1 -- 172.21.15.66:0/4091511377 wait complete. 2023-07-21T22:34:40.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.341+0000 7fb037fff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb0240a1490 0x7fb0240a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb028004b10 tx=0x7fb02800b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.342+0000 7fb051f94b80 1 -- start start 2023-07-21T22:34:40.392 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.342+0000 7fb051f94b80 1 Processor -- start 2023-07-21T22:34:40.393 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf433c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf430e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf431390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf318ac0 con 0x5592bf42cac0 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf3187c0 con 0x5592bf42c1b0 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf1fc920 con 0x5592bf427930 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.343+0000 7fb051f94b80 10 client.0 _ll_register_callbacks cb 0x5592bf316760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:40.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf430e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf430e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47596/0 (socket says 172.21.15.66:47596) 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 -- 172.21.15.66:0/2181656774 learned_addr learned my addr 172.21.15.66:0/2181656774 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf431390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb037fff700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf433c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 -- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 msgr2=0x5592bf433c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf433c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 -- 172.21.15.66:0/2181656774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 msgr2=0x5592bf431390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf431390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.344+0000 7fb02f7fe700 1 -- 172.21.15.66:0/2181656774 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb020020040 con 0x5592bf42c1b0 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.345+0000 7fb037fff700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf433c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.345+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf430e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb020009c40 tx=0x7fb02000f510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6191cfbc5174ecab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.346+0000 7fb0177fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.346+0000 7fb0177fe700 1 -- 172.21.15.66:0/2181656774 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020005040 con 0x5592bf42c1b0 2023-07-21T22:34:40.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.346+0000 7fb0177fe700 1 -- 172.21.15.66:0/2181656774 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb02002c070 con 0x5592bf42c1b0 2023-07-21T22:34:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.346+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592bf2d24f0 con 0x5592bf42c1b0 2023-07-21T22:34:40.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.346+0000 7fb0177fe700 1 -- 172.21.15.66:0/2181656774 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0200051a0 con 0x5592bf42c1b0 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 10 client.27392 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5592bf18fc60 con 0x5592bf42c1b0 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb0177fe700 1 -- 172.21.15.66:0/2181656774 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592bf2d24f0 con 0x5592bf42c1b0 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb0177fe700 1 -- 172.21.15.66:0/2181656774 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb020024070 con 0x5592bf42c1b0 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 10 client.27392 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 10 client.27392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:34:40.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 20 client.27392 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf440fc0 0x5592bf4433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 4 client.27392 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.347+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592bf371eb0 con 0x5592bf440fc0 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.348+0000 7fb037fff700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf440fc0 0x5592bf4433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.349+0000 7fb0177fe700 10 client.27392 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.349+0000 7fb037fff700 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf440fc0 0x5592bf4433d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97135a1b92b684f5 server_cookie=4e0337d2b28030f7 in_seq=0 out_seq=0 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.350+0000 7fb0177fe700 1 -- 172.21.15.66:0/2181656774 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592bf371eb0 con 0x5592bf440fc0 2023-07-21T22:34:40.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.350+0000 7fb0177fe700 10 client.27392 handle_command_reply: tid=0 2023-07-21T22:34:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.350+0000 7fb051f94b80 1 client.27392 shutdown 2023-07-21T22:34:40.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 msgr2=0x5592bf430e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf430e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb020009c40 tx=0x7fb02000f510 comp rx=0 tx=0).stop 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 shutdown_connections 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf440fc0 0x5592bf4433d0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf431390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf433c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2181656774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf430e10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 >> 172.21.15.66:0/2181656774 conn(0x5592bf41cef0 msgr2=0x5592bf422420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb0177fe700 0 client.27392 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 shutdown_connections 2023-07-21T22:34:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.351+0000 7fb051f94b80 1 -- 172.21.15.66:0/2181656774 wait complete. 2023-07-21T22:34:40.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.352+0000 7fb051f94b80 20 client.27392 trim_cache size 0 max 16384 2023-07-21T22:34:40.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 Processor -- start 2023-07-21T22:34:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 -- start start 2023-07-21T22:34:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf4385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf433730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf426950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf2d24f0 con 0x5592bf42cac0 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf41f010 con 0x5592bf42c1b0 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.380+0000 7fb051f94b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf42eea0 con 0x5592bf427930 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.381+0000 7fb03cb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf426950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.381+0000 7fb03cb9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf426950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48690/0 (socket says 172.21.15.66:48690) 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.381+0000 7fb03cb9c700 1 -- 172.21.15.66:0/651046094 learned_addr learned my addr 172.21.15.66:0/651046094 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.381+0000 7fb037fff700 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf4385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.381+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf433730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.382+0000 7fb03cb9c700 1 -- 172.21.15.66:0/651046094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 msgr2=0x5592bf4385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.382+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf4385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.382+0000 7fb03cb9c700 1 -- 172.21.15.66:0/651046094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 msgr2=0x5592bf433730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.382+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf433730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.382+0000 7fb03cb9c700 1 -- 172.21.15.66:0/651046094 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592bf421230 con 0x5592bf42cac0 2023-07-21T22:34:40.432 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.383+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf426950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0300993e0 tx=0x7fb030098e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567b174c7c8240eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.384+0000 7fb02effd700 1 -- 172.21.15.66:0/651046094 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03009d480 con 0x5592bf42cac0 2023-07-21T22:34:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.384+0000 7fb02effd700 1 -- 172.21.15.66:0/651046094 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0300a18a0 con 0x5592bf42cac0 2023-07-21T22:34:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.384+0000 7fb02effd700 1 -- 172.21.15.66:0/651046094 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0300cc360 con 0x5592bf42cac0 2023-07-21T22:34:40.433 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.385+0000 7fb051f94b80 1 -- 172.21.15.66:0/651046094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 msgr2=0x5592bf426950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.385+0000 7fb051f94b80 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf426950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0300993e0 tx=0x7fb030098e30 comp rx=0 tx=0).stop 2023-07-21T22:34:40.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.386+0000 7fb051f94b80 1 -- 172.21.15.66:0/651046094 shutdown_connections 2023-07-21T22:34:40.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.386+0000 7fb051f94b80 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf426950 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.386+0000 7fb051f94b80 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf4385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.386+0000 7fb051f94b80 1 --2- 172.21.15.66:0/651046094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf433730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.386+0000 7fb051f94b80 1 -- 172.21.15.66:0/651046094 >> 172.21.15.66:0/651046094 conn(0x5592bf41cef0 msgr2=0x5592bf425a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.387+0000 7fb051f94b80 1 -- 172.21.15.66:0/651046094 shutdown_connections 2023-07-21T22:34:40.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.387+0000 7fb051f94b80 1 -- 172.21.15.66:0/651046094 wait complete. 2023-07-21T22:34:40.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.388+0000 7fb051f94b80 1 -- start start 2023-07-21T22:34:40.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.388+0000 7fb051f94b80 1 Processor -- start 2023-07-21T22:34:40.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf436780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf436cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf493e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf4373e0 con 0x5592bf42cac0 2023-07-21T22:34:40.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf494380 con 0x5592bf42c1b0 2023-07-21T22:34:40.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592bf4a5500 con 0x5592bf427930 2023-07-21T22:34:40.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.389+0000 7fb051f94b80 10 client.0 _ll_register_callbacks cb 0x7fb0280097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:40.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.390+0000 7fb037fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf436780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.390+0000 7fb037fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf436780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32868/0 (socket says 172.21.15.66:32868) 2023-07-21T22:34:40.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.390+0000 7fb037fff700 1 -- 172.21.15.66:0/2152614753 learned_addr learned my addr 172.21.15.66:0/2152614753 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.390+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf436cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf493e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb037fff700 1 -- 172.21.15.66:0/2152614753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 msgr2=0x5592bf436cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb037fff700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf436cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb037fff700 1 -- 172.21.15.66:0/2152614753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 msgr2=0x5592bf493e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb037fff700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf493e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb037fff700 1 -- 172.21.15.66:0/2152614753 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03806fde0 con 0x5592bf427930 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.391+0000 7fb03cb9c700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf493e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.392+0000 7fb037fff700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf436780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5592bf4285b0 tx=0x7fb028006040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe4af00fda856b9a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.392+0000 7fb0177fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:40.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.392+0000 7fb0177fe700 1 -- 172.21.15.66:0/2152614753 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb028029070 con 0x5592bf427930 2023-07-21T22:34:40.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.392+0000 7fb0177fe700 1 -- 172.21.15.66:0/2152614753 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb02802e470 con 0x5592bf427930 2023-07-21T22:34:40.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.393+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0200044e0 con 0x5592bf427930 2023-07-21T22:34:40.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.393+0000 7fb0177fe700 1 -- 172.21.15.66:0/2152614753 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb02802e630 con 0x5592bf427930 2023-07-21T22:34:40.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.393+0000 7fb051f94b80 10 client.27355 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:40.442 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.393+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5592bf1cd250 con 0x5592bf427930 2023-07-21T22:34:40.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb0177fe700 1 -- 172.21.15.66:0/2152614753 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0200044e0 con 0x5592bf427930 2023-07-21T22:34:40.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb0177fe700 1 -- 172.21.15.66:0/2152614753 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fb028036070 con 0x5592bf427930 2023-07-21T22:34:40.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb051f94b80 10 client.27355 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:40.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb051f94b80 10 client.27355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:34:40.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb051f94b80 20 client.27355 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:40.443 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf4a6c30 0x5592bf4a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb051f94b80 4 client.27355 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:34:40.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.394+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5592bf334050 con 0x5592bf4a6c30 2023-07-21T22:34:40.444 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.395+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf4a6c30 0x5592bf4a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb02f7fe700 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf4a6c30 0x5592bf4a9040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1682ec6b80d5bf9 server_cookie=de16f4ebf1fb0484 in_seq=0 out_seq=0 2023-07-21T22:34:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb0177fe700 10 client.27355 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb0177fe700 1 -- 172.21.15.66:0/2152614753 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5592bf334050 con 0x5592bf4a6c30 2023-07-21T22:34:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb0177fe700 10 client.27355 handle_command_reply: tid=0 2023-07-21T22:34:40.446 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb051f94b80 1 client.27355 shutdown 2023-07-21T22:34:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 msgr2=0x5592bf436780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf436780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x5592bf4285b0 tx=0x7fb028006040 comp rx=0 tx=0).stop 2023-07-21T22:34:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 shutdown_connections 2023-07-21T22:34:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.396+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5592bf4a6c30 0x5592bf4a9040 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5592bf42cac0 0x5592bf493e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.447 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5592bf427930 0x5592bf436780 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 1 --2- 172.21.15.66:0/2152614753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592bf42c1b0 0x5592bf436cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 >> 172.21.15.66:0/2152614753 conn(0x5592bf41cef0 msgr2=0x5592bf426950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb0177fe700 0 client.27355 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:34:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 shutdown_connections 2023-07-21T22:34:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 1 -- 172.21.15.66:0/2152614753 wait complete. 2023-07-21T22:34:40.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.397+0000 7fb051f94b80 20 client.27355 trim_cache size 0 max 16384 2023-07-21T22:34:40.449 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:34:40.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb0240a1490 msgr2=0x7fb0240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb0240a1490 0x7fb0240a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb028004b10 tx=0x7fb02800b040 comp rx=0 tx=0).stop 2023-07-21T22:34:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 msgr2=0x7fb03806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 0x7fb03806afc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb03000d770 tx=0x7fb03000db30 comp rx=0 tx=0).stop 2023-07-21T22:34:40.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 shutdown_connections 2023-07-21T22:34:40.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb03806ab20 0x7fb03806afc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb0240a1490 0x7fb0240a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb0380728b0 0x7fb038071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 --2- 172.21.15.66:0/3328509272 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb038110b00 0x7fb038072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.398+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 >> 172.21.15.66:0/3328509272 conn(0x7fb0380fb5f0 msgr2=0x7fb0380fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.452 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.399+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 shutdown_connections 2023-07-21T22:34:40.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.399+0000 7fb03e5ff700 1 -- 172.21.15.66:0/3328509272 wait complete. 2023-07-21T22:34:40.453 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:40 smithi066 ceph-mon[126379]: pgmap v412: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-21T22:34:40.465 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:34:40.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:40 smithi149 ceph-mon[129268]: pgmap v412: 129 pgs: 129 active+clean; 698 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2023-07-21T22:34:40.684 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 -- 172.21.15.66:0/1280043374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 msgr2=0x7ff0880fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/1280043374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff0880fe3c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff0780099c0 tx=0x7ff078009cd0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 -- 172.21.15.66:0/1280043374 shutdown_connections 2023-07-21T22:34:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/1280043374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 0x7ff08810abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/1280043374 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff0880fe3c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/1280043374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 0x7ff088102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.685 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 -- 172.21.15.66:0/1280043374 >> 172.21.15.66:0/1280043374 conn(0x7ff088075b70 msgr2=0x7ff088075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 -- 172.21.15.66:0/1280043374 shutdown_connections 2023-07-21T22:34:40.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.637+0000 7ff08e7ff700 1 -- 172.21.15.66:0/1280043374 wait complete. 2023-07-21T22:34:40.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 Processor -- start 2023-07-21T22:34:40.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 -- start start 2023-07-21T22:34:40.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff08819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 0x7ff08819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 0x7ff088195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088196150 con 0x7ff0880fdff0 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0881962c0 con 0x7ff0880fe990 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.638+0000 7ff08e7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff088196480 con 0x7ff088102fe0 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff08cd9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 0x7ff088195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff08819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 0x7ff08819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff08cd9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 0x7ff088195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32876/0 (socket says 172.21.15.66:32876) 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 0x7ff08819bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47640/0 (socket says 172.21.15.66:47640) 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff08cd9c700 1 -- 172.21.15.66:0/3438045902 learned_addr learned my addr 172.21.15.66:0/3438045902 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff08819ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48720/0 (socket says 172.21.15.66:48720) 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 -- 172.21.15.66:0/3438045902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 msgr2=0x7ff088195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 0x7ff088195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 -- 172.21.15.66:0/3438045902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 msgr2=0x7ff08819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 0x7ff08819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 -- 172.21.15.66:0/3438045902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c009520 con 0x7ff0880fdff0 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.639+0000 7ff087fff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff08819ba70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff0780098e0 tx=0x7ff07800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.640+0000 7ff0857fa700 1 -- 172.21.15.66:0/3438045902 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07801d070 con 0x7ff0880fdff0 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.640+0000 7ff0857fa700 1 -- 172.21.15.66:0/3438045902 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff078005050 con 0x7ff0880fdff0 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.640+0000 7ff0857fa700 1 -- 172.21.15.66:0/3438045902 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078004970 con 0x7ff0880fdff0 2023-07-21T22:34:40.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.640+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0780095f0 con 0x7ff0880fdff0 2023-07-21T22:34:40.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.640+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff088196940 con 0x7ff0880fdff0 2023-07-21T22:34:40.691 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.644+0000 7ff0857fa700 1 -- 172.21.15.66:0/3438045902 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff078005470 con 0x7ff0880fdff0 2023-07-21T22:34:40.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.645+0000 7ff0a2107b80 1 Processor -- start 2023-07-21T22:34:40.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.645+0000 7ff0a2107b80 1 -- start start 2023-07-21T22:34:40.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.645+0000 7ff0857fa700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0700a1340 0x7ff0700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af935c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0857fa700 1 -- 172.21.15.66:0/3438045902 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0780cdcb0 con 0x7ff0880fdff0 2023-07-21T22:34:40.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9353350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9353da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9258cb0 con 0x560af935ced0 2023-07-21T22:34:40.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af92aba40 con 0x560af9353930 2023-07-21T22:34:40.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af92a4ed0 con 0x560af935c5c0 2023-07-21T22:34:40.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0700a1340 0x7ff0700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af935c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af935c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32888/0 (socket says 172.21.15.66:32888) 2023-07-21T22:34:40.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 -- 172.21.15.66:0/2489286109 learned_addr learned my addr 172.21.15.66:0/2489286109 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9353da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.696 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff0877fe700 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9353350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 -- 172.21.15.66:0/2489286109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 msgr2=0x560af9353da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9353da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 -- 172.21.15.66:0/2489286109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 msgr2=0x560af9353350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9353350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff087fff700 1 -- 172.21.15.66:0/2489286109 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af90d2de0 con 0x560af935c5c0 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.646+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9353da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.647+0000 7ff087fff700 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af935c990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff078097030 tx=0x7ff07809ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd62679cb979a70 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.647+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0700a1340 0x7ff0700a37f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff080009c10 tx=0x7ff080013250 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.647+0000 7ff06e7fc700 1 -- 172.21.15.66:0/2489286109 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780051c0 con 0x560af935c5c0 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.647+0000 7ff06e7fc700 1 -- 172.21.15.66:0/2489286109 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff07809ad90 con 0x560af935c5c0 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.647+0000 7ff06e7fc700 1 -- 172.21.15.66:0/2489286109 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780a2cc0 con 0x560af935c5c0 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 -- 172.21.15.66:0/2489286109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 msgr2=0x560af935c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af935c990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff078097030 tx=0x7ff07809ab30 comp rx=0 tx=0).stop 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 -- 172.21.15.66:0/2489286109 shutdown_connections 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9353da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af935c990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/2489286109 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9353350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 -- 172.21.15.66:0/2489286109 >> 172.21.15.66:0/2489286109 conn(0x560af9348ef0 msgr2=0x560af934e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 -- 172.21.15.66:0/2489286109 shutdown_connections 2023-07-21T22:34:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 -- 172.21.15.66:0/2489286109 wait complete. 2023-07-21T22:34:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.648+0000 7ff0a2107b80 1 -- start start 2023-07-21T22:34:40.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 Processor -- start 2023-07-21T22:34:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af93611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af93617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9244ac0 con 0x560af935ced0 2023-07-21T22:34:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af92447c0 con 0x560af9353930 2023-07-21T22:34:40.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9128920 con 0x560af935c5c0 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0a2107b80 10 client.0 _ll_register_callbacks cb 0x560af9242760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff087fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff087fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47648/0 (socket says 172.21.15.66:47648) 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff087fff700 1 -- 172.21.15.66:0/198611680 learned_addr learned my addr 172.21.15.66:0/198611680 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.649+0000 7ff0877fe700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af93611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af93617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff087fff700 1 -- 172.21.15.66:0/198611680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 msgr2=0x560af93611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff087fff700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af93611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff087fff700 1 -- 172.21.15.66:0/198611680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 msgr2=0x560af93617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff087fff700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af93617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff087fff700 1 -- 172.21.15.66:0/198611680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff088196940 con 0x560af9353930 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af93617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff0877fe700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af93611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff087fff700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363f60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff0780a29c0 tx=0x7ff07809b110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5bd7bae5f138816 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff0677fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff0677fe700 1 -- 172.21.15.66:0/198611680 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780a2a60 con 0x560af9353930 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff0677fe700 1 -- 172.21.15.66:0/198611680 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff0780a2bc0 con 0x560af9353930 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560af91fe4f0 con 0x560af9353930 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.650+0000 7ff0677fe700 1 -- 172.21.15.66:0/198611680 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff078096920 con 0x560af9353930 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 10 client.27410 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560af90bbc60 con 0x560af9353930 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0677fe700 1 -- 172.21.15.66:0/198611680 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560af91fe4f0 con 0x560af9353930 2023-07-21T22:34:40.705 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0677fe700 1 -- 172.21.15.66:0/198611680 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff0780c8020 con 0x560af9353930 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 10 client.27410 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 10 client.27410 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 20 client.27410 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93710b0 0x560af93734c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 4 client.27410 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560af929deb0 con 0x560af93710b0 2023-07-21T22:34:40.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.651+0000 7ff0877fe700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93710b0 0x560af93734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.652+0000 7ff0877fe700 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93710b0 0x560af93734c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eab0429d090d5b40 server_cookie=6745af0b82b92e00 in_seq=0 out_seq=0 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.652+0000 7ff0677fe700 10 client.27410 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0677fe700 1 -- 172.21.15.66:0/198611680 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560af929deb0 con 0x560af93710b0 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0677fe700 10 client.27410 handle_command_reply: tid=0 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 client.27410 shutdown 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 msgr2=0x560af9363f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363f60 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff0780a29c0 tx=0x7ff07809b110 comp rx=0 tx=0).stop 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 shutdown_connections 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93710b0 0x560af93734c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363f60 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af93617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0677fe700 0 client.27410 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/198611680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af93611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 >> 172.21.15.66:0/198611680 conn(0x560af9348ef0 msgr2=0x560af934e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 shutdown_connections 2023-07-21T22:34:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 1 -- 172.21.15.66:0/198611680 wait complete. 2023-07-21T22:34:40.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.653+0000 7ff0a2107b80 20 client.27410 trim_cache size 0 max 16384 2023-07-21T22:34:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.667+0000 7ff0a2107b80 1 Processor -- start 2023-07-21T22:34:40.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.667+0000 7ff0a2107b80 1 -- start start 2023-07-21T22:34:40.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af9353930 0x560af9368730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af935c5c0 0x560af9360a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9360fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af91fe520 con 0x560af935ced0 2023-07-21T22:34:40.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9364570 con 0x560af935c5c0 2023-07-21T22:34:40.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af934fca0 con 0x560af9353930 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff08cd9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9360fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0877fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af935c5c0 0x560af9360a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff08cd9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9360fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48738/0 (socket says 172.21.15.66:48738) 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff08cd9c700 1 -- 172.21.15.66:0/1460439696 learned_addr learned my addr 172.21.15.66:0/1460439696 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff087fff700 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af9353930 0x560af9368730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0877fe700 1 -- 172.21.15.66:0/1460439696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af9353930 msgr2=0x560af9368730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0877fe700 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af9353930 0x560af9368730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0877fe700 1 -- 172.21.15.66:0/1460439696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 msgr2=0x560af9360fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0877fe700 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9360fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff0877fe700 1 -- 172.21.15.66:0/1460439696 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af90f9250 con 0x560af935c5c0 2023-07-21T22:34:40.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.668+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9360fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:40.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.669+0000 7ff0877fe700 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af935c5c0 0x560af9360a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x560af93589c0 tx=0x7ff07c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba48fea7338605b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.669+0000 7ff06effd700 1 -- 172.21.15.66:0/1460439696 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c016030 con 0x560af935c5c0 2023-07-21T22:34:40.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.669+0000 7ff06effd700 1 -- 172.21.15.66:0/1460439696 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff07c017040 con 0x560af935c5c0 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.669+0000 7ff06effd700 1 -- 172.21.15.66:0/1460439696 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c003970 con 0x560af935c5c0 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.669+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1460439696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af935c5c0 msgr2=0x560af9360a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.669+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af935c5c0 0x560af9360a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x560af93589c0 tx=0x7ff07c009260 comp rx=0 tx=0).stop 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1460439696 shutdown_connections 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9360fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af9353930 0x560af9368730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1460439696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af935c5c0 0x560af9360a80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1460439696 >> 172.21.15.66:0/1460439696 conn(0x560af9348ef0 msgr2=0x560af9351a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1460439696 shutdown_connections 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1460439696 wait complete. 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 -- start start 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.670+0000 7ff0a2107b80 1 Processor -- start 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af9362a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9362fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9363500 con 0x560af935ced0 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9363670 con 0x560af9353930 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9363830 con 0x560af935c5c0 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0a2107b80 10 client.0 _ll_register_callbacks cb 0x7ff07c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47662/0 (socket says 172.21.15.66:47662) 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 -- 172.21.15.66:0/1617611653 learned_addr learned my addr 172.21.15.66:0/1617611653 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9362fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff0877fe700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af9362a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 -- 172.21.15.66:0/1617611653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 msgr2=0x560af9362a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af9362a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 -- 172.21.15.66:0/1617611653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 msgr2=0x560af9362fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9362fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff087fff700 1 -- 172.21.15.66:0/1617611653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff080003a00 con 0x560af9353930 2023-07-21T22:34:40.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.671+0000 7ff08cd9c700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9362fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff087fff700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363b00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff078005cb0 tx=0x7ff078009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7efc1e5eb86d2ef6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0677fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0677fe700 1 -- 172.21.15.66:0/1617611653 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07807d070 con 0x560af9353930 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff07809ad90 con 0x560af9353930 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0677fe700 1 -- 172.21.15.66:0/1617611653 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff078098f90 con 0x560af9353930 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0677fe700 1 -- 172.21.15.66:0/1617611653 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff07809ab30 con 0x560af9353930 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0a2107b80 10 client.27416 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:40.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7ff07c004040 con 0x560af9353930 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0677fe700 1 -- 172.21.15.66:0/1617611653 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff07809ad90 con 0x560af9353930 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0677fe700 1 -- 172.21.15.66:0/1617611653 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff0780a7020 con 0x560af9353930 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.672+0000 7ff0a2107b80 10 client.27416 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0a2107b80 10 client.27416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0a2107b80 20 client.27416 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93c5ef0 0x560af93c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0a2107b80 4 client.27416 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:34:40.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560af9260050 con 0x560af93c5ef0 2023-07-21T22:34:40.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0877fe700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93c5ef0 0x560af93c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0877fe700 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93c5ef0 0x560af93c8300 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1c0e1a9753e6f8d server_cookie=cb93f3d78e90efb in_seq=0 out_seq=0 2023-07-21T22:34:40.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.673+0000 7ff0677fe700 10 client.27416 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:40.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0677fe700 1 -- 172.21.15.66:0/1617611653 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560af9260050 con 0x560af93c5ef0 2023-07-21T22:34:40.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0677fe700 10 client.27416 handle_command_reply: tid=0 2023-07-21T22:34:40.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 client.27416 shutdown 2023-07-21T22:34:40.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 msgr2=0x560af9363b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363b00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff078005cb0 tx=0x7ff078009310 comp rx=0 tx=0).stop 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 shutdown_connections 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x560af93c5ef0 0x560af93c8300 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af9353930 0x560af9363b00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0677fe700 0 client.27416 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560af935ced0 0x560af9362fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 --2- 172.21.15.66:0/1617611653 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560af935c5c0 0x560af9362a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 >> 172.21.15.66:0/1617611653 conn(0x560af9348ef0 msgr2=0x560af9360a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 shutdown_connections 2023-07-21T22:34:40.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 1 -- 172.21.15.66:0/1617611653 wait complete. 2023-07-21T22:34:40.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.674+0000 7ff0a2107b80 20 client.27416 trim_cache size 0 max 16384 2023-07-21T22:34:40.730 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:34:40.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0700a1340 msgr2=0x7ff0700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0700a1340 0x7ff0700a37f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff080009c10 tx=0x7ff080013250 comp rx=0 tx=0).stop 2023-07-21T22:34:40.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 msgr2=0x7ff08819ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff08819ba70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff0780098e0 tx=0x7ff07800f690 comp rx=0 tx=0).stop 2023-07-21T22:34:40.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 shutdown_connections 2023-07-21T22:34:40.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff0700a1340 0x7ff0700a37f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff088102fe0 0x7ff088195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff0880fdff0 0x7ff08819ba70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 --2- 172.21.15.66:0/3438045902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0880fe990 0x7ff08819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 >> 172.21.15.66:0/3438045902 conn(0x7ff088075b70 msgr2=0x7ff088101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 shutdown_connections 2023-07-21T22:34:40.732 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.675+0000 7ff08e7ff700 1 -- 172.21.15.66:0/3438045902 wait complete. 2023-07-21T22:34:40.740 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:34:40.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 3605 52.44 MB/sec warmup 7 sec latency 6515.227 ms 2023-07-21T22:34:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 -- 172.21.15.66:0/4007943565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0370105af0 msgr2=0x7f0370105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.979 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 --2- 172.21.15.66:0/4007943565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0370105af0 0x7f0370105ec0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0360009990 tx=0x7f0360009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 -- 172.21.15.66:0/4007943565 shutdown_connections 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 --2- 172.21.15.66:0/4007943565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0370105af0 0x7f0370105ec0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 --2- 172.21.15.66:0/4007943565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0370100490 0x7f037010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 --2- 172.21.15.66:0/4007943565 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f03700ffae0 0x7f03700fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 -- 172.21.15.66:0/4007943565 >> 172.21.15.66:0/4007943565 conn(0x7f03700fb5e0 msgr2=0x7f03700fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.931+0000 7f0374fff700 1 -- 172.21.15.66:0/4007943565 shutdown_connections 2023-07-21T22:34:40.980 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.932+0000 7f0374fff700 1 -- 172.21.15.66:0/4007943565 wait complete. 2023-07-21T22:34:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 Processor -- start 2023-07-21T22:34:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 -- start start 2023-07-21T22:34:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 0x7f0370197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0370100490 0x7f0370198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.982 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0370105af0 0x7f037019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019ce00 con 0x7f0370100490 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019cf70 con 0x7f03700ffae0 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f0374fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037019d130 con 0x7f0370105af0 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f036e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 0x7f0370197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f036e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 0x7f0370197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47680/0 (socket says 172.21.15.66:47680) 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f036e59c700 1 -- 172.21.15.66:0/1083446594 learned_addr learned my addr 172.21.15.66:0/1083446594 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f036ed9d700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0370105af0 0x7f037019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.933+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0370100490 0x7f0370198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f036e59c700 1 -- 172.21.15.66:0/1083446594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0370105af0 msgr2=0x7f037019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f036e59c700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0370105af0 0x7f037019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f036e59c700 1 -- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0370100490 msgr2=0x7f0370198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f036e59c700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0370100490 0x7f0370198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f036e59c700 1 -- 172.21.15.66:0/1083446594 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03600095f0 con 0x7f03700ffae0 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f036e59c700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 0x7f0370197e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0360005910 tx=0x7f0360005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f035f7fe700 1 -- 172.21.15.66:0/1083446594 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036001d070 con 0x7f03700ffae0 2023-07-21T22:34:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037019d3b0 con 0x7f03700ffae0 2023-07-21T22:34:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f035f7fe700 1 -- 172.21.15.66:0/1083446594 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0360022470 con 0x7f03700ffae0 2023-07-21T22:34:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.934+0000 7f035f7fe700 1 -- 172.21.15.66:0/1083446594 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360017650 con 0x7f03700ffae0 2023-07-21T22:34:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.935+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03701a4c50 con 0x7f03700ffae0 2023-07-21T22:34:40.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.938+0000 7f035f7fe700 1 -- 172.21.15.66:0/1083446594 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f036000fad0 con 0x7f03700ffae0 2023-07-21T22:34:40.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.939+0000 7f03889a4b80 1 Processor -- start 2023-07-21T22:34:40.986 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.939+0000 7f03889a4b80 1 -- start start 2023-07-21T22:34:40.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd152580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd14d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f03889a4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd052cb0 con 0x560fcd14d930 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f03889a4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd0a5a40 con 0x560fcd152ac0 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f03889a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd09eed0 con 0x560fcd1521b0 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd14d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036ed9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd14d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47688/0 (socket says 172.21.15.66:47688) 2023-07-21T22:34:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036e59c700 1 -- 172.21.15.66:0/2370828350 learned_addr learned my addr 172.21.15.66:0/2370828350 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036dd9b700 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f035f7fe700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036dd9b700 1 -- 172.21.15.66:0/2370828350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 msgr2=0x560fcd152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036dd9b700 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036dd9b700 1 -- 172.21.15.66:0/2370828350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 msgr2=0x560fcd14d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036dd9b700 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd14d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.940+0000 7f036dd9b700 1 -- 172.21.15.66:0/2370828350 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fcceccde0 con 0x560fcd14d930 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.941+0000 7f036ed9d700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.941+0000 7f035f7fe700 1 -- 172.21.15.66:0/1083446594 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f03600ce510 con 0x7f03700ffae0 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.941+0000 7f036dd9b700 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd14dda0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0364009c80 tx=0x7f0364009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1c02fcd3a1bc4a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.941+0000 7f035cff9700 1 -- 172.21.15.66:0/2370828350 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364011cb0 con 0x560fcd14d930 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.941+0000 7f035cff9700 1 -- 172.21.15.66:0/2370828350 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0364011e10 con 0x560fcd14d930 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.941+0000 7f035cff9700 1 -- 172.21.15.66:0/2370828350 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364005280 con 0x560fcd14d930 2023-07-21T22:34:40.991 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 -- 172.21.15.66:0/2370828350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 msgr2=0x560fcd14dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f036ed9d700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0368009710 tx=0x7f0368004ad0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd14dda0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0364009c80 tx=0x7f0364009f90 comp rx=0 tx=0).stop 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 -- 172.21.15.66:0/2370828350 shutdown_connections 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd14dda0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2370828350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd14d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.992 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 -- 172.21.15.66:0/2370828350 >> 172.21.15.66:0/2370828350 conn(0x560fcd142ef0 msgr2=0x560fcd148830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 -- 172.21.15.66:0/2370828350 shutdown_connections 2023-07-21T22:34:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.942+0000 7f03889a4b80 1 -- 172.21.15.66:0/2370828350 wait complete. 2023-07-21T22:34:40.993 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.943+0000 7f03889a4b80 1 -- start start 2023-07-21T22:34:40.994 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.943+0000 7f03889a4b80 1 Processor -- start 2023-07-21T22:34:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 0x560fcd159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd156d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd152ac0 0x560fcd157350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd03eac0 con 0x560fcd152ac0 2023-07-21T22:34:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd03e7c0 con 0x560fcd14d930 2023-07-21T22:34:40.995 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fccf22920 con 0x560fcd1521b0 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f03889a4b80 10 client.0 _ll_register_callbacks cb 0x560fcd03c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 0x560fcd159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 0x560fcd159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47692/0 (socket says 172.21.15.66:47692) 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 -- 172.21.15.66:0/2108038135 learned_addr learned my addr 172.21.15.66:0/2108038135 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32968/0 (socket says 172.21.15.66:32968) 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036ed9d700 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd152ac0 0x560fcd157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.996 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 -- 172.21.15.66:0/2108038135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 msgr2=0x560fcd156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 -- 172.21.15.66:0/2108038135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd152ac0 msgr2=0x560fcd157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd152ac0 0x560fcd157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.944+0000 7f036e59c700 1 -- 172.21.15.66:0/2108038135 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03640098e0 con 0x560fcd14d930 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f036e59c700 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 0x560fcd159af0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0360005f50 tx=0x7f0360005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec9913cab96a447 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f0355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:40.997 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f0355ffb700 1 -- 172.21.15.66:0/2108038135 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360097630 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f0355ffb700 1 -- 172.21.15.66:0/2108038135 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f03600a3690 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fccff84f0 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f0355ffb700 1 -- 172.21.15.66:0/2108038135 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f036009b0a0 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f03889a4b80 10 client.27428 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560fcceb5c60 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.945+0000 7f0355ffb700 1 -- 172.21.15.66:0/2108038135 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560fccff84f0 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f0355ffb700 1 -- 172.21.15.66:0/2108038135 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f03600ce110 con 0x560fcd14d930 2023-07-21T22:34:40.998 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f03889a4b80 10 client.27428 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f03889a4b80 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f03889a4b80 20 client.27428 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd166f90 0x560fcd169370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f03889a4b80 4 client.27428 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fcd097eb0 con 0x560fcd166f90 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.946+0000 7f036dd9b700 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd166f90 0x560fcd169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.947+0000 7f036dd9b700 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd166f90 0x560fcd169370 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=733cb5bce08dfe15 server_cookie=39f48ca0e4aadc2 in_seq=0 out_seq=0 2023-07-21T22:34:40.999 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.947+0000 7f0355ffb700 10 client.27428 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:41.000 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f0355ffb700 1 -- 172.21.15.66:0/2108038135 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560fcd097eb0 con 0x560fcd166f90 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f0355ffb700 10 client.27428 handle_command_reply: tid=0 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 client.27428 shutdown 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 msgr2=0x560fcd159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 0x560fcd159af0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0360005f50 tx=0x7f0360005c00 comp rx=0 tx=0).stop 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 shutdown_connections 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd14d930 0x560fcd159af0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.001 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd166f90 0x560fcd169370 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd152ac0 0x560fcd157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 --2- 172.21.15.66:0/2108038135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f0355ffb700 0 client.27428 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.951+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 >> 172.21.15.66:0/2108038135 conn(0x560fcd142ef0 msgr2=0x560fcd148420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.952+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 shutdown_connections 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.952+0000 7f03889a4b80 1 -- 172.21.15.66:0/2108038135 wait complete. 2023-07-21T22:34:41.002 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.952+0000 7f03889a4b80 20 client.27428 trim_cache size 0 max 16384 2023-07-21T22:34:41.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.967+0000 7f03889a4b80 1 Processor -- start 2023-07-21T22:34:41.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.967+0000 7f03889a4b80 1 -- start start 2023-07-21T22:34:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.967+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd15e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f03889a4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd149b20 con 0x560fcd14d930 2023-07-21T22:34:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f03889a4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd14e1a0 con 0x560fcd152ac0 2023-07-21T22:34:41.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f03889a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd163dd0 con 0x560fcd1521b0 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd168fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd168fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32984/0 (socket says 172.21.15.66:32984) 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 -- 172.21.15.66:0/1748225140 learned_addr learned my addr 172.21.15.66:0/1748225140 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036e59c700 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd15e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036ed9d700 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 -- 172.21.15.66:0/1748225140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 msgr2=0x560fcd159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 -- 172.21.15.66:0/1748225140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 msgr2=0x560fcd15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd15e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 -- 172.21.15.66:0/1748225140 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fccef3250 con 0x560fcd1521b0 2023-07-21T22:34:41.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.968+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd168fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0364004c80 tx=0x7f0364011bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47eb842bceb7c771 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.969+0000 7f035d7fa700 1 -- 172.21.15.66:0/1748225140 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364027b50 con 0x560fcd1521b0 2023-07-21T22:34:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.969+0000 7f035d7fa700 1 -- 172.21.15.66:0/1748225140 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0364027cb0 con 0x560fcd1521b0 2023-07-21T22:34:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.969+0000 7f035d7fa700 1 -- 172.21.15.66:0/1748225140 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364003da0 con 0x560fcd1521b0 2023-07-21T22:34:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.969+0000 7f03889a4b80 1 -- 172.21.15.66:0/1748225140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 msgr2=0x560fcd168fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.020 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd168fa0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0364004c80 tx=0x7f0364011bd0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 -- 172.21.15.66:0/1748225140 shutdown_connections 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd15e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1748225140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd168fa0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 -- 172.21.15.66:0/1748225140 >> 172.21.15.66:0/1748225140 conn(0x560fcd142ef0 msgr2=0x560fcd146f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 -- 172.21.15.66:0/1748225140 shutdown_connections 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 -- 172.21.15.66:0/1748225140 wait complete. 2023-07-21T22:34:41.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 -- start start 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.970+0000 7f03889a4b80 1 Processor -- start 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd1b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd1b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd15a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd15ac50 con 0x560fcd14d930 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd15adc0 con 0x560fcd152ac0 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fcd15af80 con 0x560fcd1521b0 2023-07-21T22:34:41.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f03889a4b80 10 client.0 _ll_register_callbacks cb 0x560fcd03c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd1b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f036dd9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd1b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:32992/0 (socket says 172.21.15.66:32992) 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f036e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd1b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f036dd9b700 1 -- 172.21.15.66:0/1959237613 learned_addr learned my addr 172.21.15.66:0/1959237613 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.971+0000 7f036ed9d700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036dd9b700 1 -- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 msgr2=0x560fcd15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036dd9b700 1 -- 172.21.15.66:0/1959237613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 msgr2=0x560fcd1b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd1b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036dd9b700 1 -- 172.21.15.66:0/1959237613 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036401d040 con 0x560fcd1521b0 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036e59c700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd1b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f036dd9b700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd1b46c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0364011cc0 tx=0x7f0364005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e15e6166f123546f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f0355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f0355ffb700 1 -- 172.21.15.66:0/1959237613 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364027ea0 con 0x560fcd1521b0 2023-07-21T22:34:41.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0364011e10 con 0x560fcd1521b0 2023-07-21T22:34:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f0355ffb700 1 -- 172.21.15.66:0/1959237613 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0364027420 con 0x560fcd1521b0 2023-07-21T22:34:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.972+0000 7f0355ffb700 1 -- 172.21.15.66:0/1959237613 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0364003680 con 0x560fcd1521b0 2023-07-21T22:34:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 10 client.27391 fetch_fsmap learned FSMap version 46 2023-07-21T22:34:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x560fccecb2b0 con 0x560fcd1521b0 2023-07-21T22:34:41.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f0355ffb700 1 -- 172.21.15.66:0/1959237613 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0364011e10 con 0x560fcd1521b0 2023-07-21T22:34:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f0355ffb700 1 -- 172.21.15.66:0/1959237613 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 46) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f0364005060 con 0x560fcd1521b0 2023-07-21T22:34:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 10 client.27391 fetch_fsmap finished waiting for FSMap version 46 2023-07-21T22:34:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 10 client.27391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:41.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 20 client.27391 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd1bbb90 0x560fcd1bdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 4 client.27391 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560fcd05a050 con 0x560fcd1bbb90 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.973+0000 7f036e59c700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd1bbb90 0x560fcd1bdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.974+0000 7f036e59c700 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd1bbb90 0x560fcd1bdfa0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a52c890589451f server_cookie=126aeadc0916cc23 in_seq=0 out_seq=0 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.974+0000 7f0355ffb700 10 client.27391 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f0355ffb700 1 -- 172.21.15.66:0/1959237613 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fcd05a050 con 0x560fcd1bbb90 2023-07-21T22:34:41.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f0355ffb700 10 client.27391 handle_command_reply: tid=0 2023-07-21T22:34:41.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 client.27391 shutdown 2023-07-21T22:34:41.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 msgr2=0x560fcd1b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd1b46c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0364011cc0 tx=0x7f0364005da0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 shutdown_connections 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560fcd152ac0 0x560fcd15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560fcd1bbb90 0x560fcd1bdfa0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560fcd14d930 0x560fcd1b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 --2- 172.21.15.66:0/1959237613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560fcd1521b0 0x560fcd1b46c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f0355ffb700 0 client.27391 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:41.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 >> 172.21.15.66:0/1959237613 conn(0x560fcd142ef0 msgr2=0x560fcd162d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 shutdown_connections 2023-07-21T22:34:41.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.979+0000 7f03889a4b80 1 -- 172.21.15.66:0/1959237613 wait complete. 2023-07-21T22:34:41.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f03889a4b80 20 client.27391 trim_cache size 0 max 16384 2023-07-21T22:34:41.030 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:34:41.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 msgr2=0x7f03580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0368009710 tx=0x7f0368004ad0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 msgr2=0x7f0370197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:41.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 0x7f0370197e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0360005910 tx=0x7f0360005740 comp rx=0 tx=0).stop 2023-07-21T22:34:41.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 shutdown_connections 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03700ffae0 0x7f0370197e20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f03580a12c0 0x7f03580a3770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0370105af0 0x7f037019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 --2- 172.21.15.66:0/1083446594 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0370100490 0x7f0370198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.980+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 >> 172.21.15.66:0/1083446594 conn(0x7f03700fb5e0 msgr2=0x7f037010adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.981+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 shutdown_connections 2023-07-21T22:34:41.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:40.981+0000 7f0374fff700 1 -- 172.21.15.66:0/1083446594 wait complete. 2023-07-21T22:34:41.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:41 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1201313495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:34:41.456 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:41 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1201313495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:34:41.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:41 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1201313495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:34:41.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 3801 46.03 MB/sec warmup 8 sec latency 7515.366 ms 2023-07-21T22:34:42.047 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:34:42.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:34:42.334 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/3999254176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b140fe880 msgr2=0x7f0b140fec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/3999254176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b140fe880 0x7f0b140fec50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009990 tx=0x7f0b10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/3999254176 shutdown_connections 2023-07-21T22:34:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/3999254176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b1410c430 0x7f0b1410e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/3999254176 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b140fe880 0x7f0b140fec50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/3999254176 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140ff190 0x7f0b1410bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.287+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/3999254176 >> 172.21.15.66:0/3999254176 conn(0x7f0b140fa4b0 msgr2=0x7f0b140fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.288+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/3999254176 shutdown_connections 2023-07-21T22:34:42.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.288+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/3999254176 wait complete. 2023-07-21T22:34:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 Processor -- start 2023-07-21T22:34:42.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 -- start start 2023-07-21T22:34:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 0x7f0b14100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b140ff190 0x7f0b141010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b1410c430 0x7f0b141015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140708e0 con 0x7f0b1410c430 2023-07-21T22:34:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14070a50 con 0x7f0b140ff190 2023-07-21T22:34:42.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.290+0000 7f0b1b9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14070c10 con 0x7f0b140fe880 2023-07-21T22:34:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.291+0000 7f0b18f9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b140ff190 0x7f0b141010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.291+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 0x7f0b14100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.291+0000 7f0b19f9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b1410c430 0x7f0b141015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.291+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 0x7f0b14100b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43434/0 (socket says 172.21.15.66:43434) 2023-07-21T22:34:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.291+0000 7f0b1979b700 1 -- 172.21.15.66:0/1531085199 learned_addr learned my addr 172.21.15.66:0/1531085199 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.292+0000 7f0b1979b700 1 -- 172.21.15.66:0/1531085199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b140ff190 msgr2=0x7f0b141010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.292+0000 7f0b1979b700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b140ff190 0x7f0b141010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.292+0000 7f0b1979b700 1 -- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b1410c430 msgr2=0x7f0b141015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.292+0000 7f0b1979b700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b1410c430 0x7f0b141015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.292+0000 7f0b1979b700 1 -- 172.21.15.66:0/1531085199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100095f0 con 0x7f0b140fe880 2023-07-21T22:34:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.293+0000 7f0b1979b700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 0x7f0b14100b60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005910 tx=0x7f0b10005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.293+0000 7f0b0a7fc700 1 -- 172.21.15.66:0/1531085199 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1001d070 con 0x7f0b140fe880 2023-07-21T22:34:42.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.293+0000 7f0b0a7fc700 1 -- 172.21.15.66:0/1531085199 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b10022470 con 0x7f0b140fe880 2023-07-21T22:34:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.293+0000 7f0b0a7fc700 1 -- 172.21.15.66:0/1531085199 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10017650 con 0x7f0b140fe880 2023-07-21T22:34:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.293+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b14070e90 con 0x7f0b140fe880 2023-07-21T22:34:42.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.293+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b140712f0 con 0x7f0b140fe880 2023-07-21T22:34:42.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.299+0000 7f0b0a7fc700 1 -- 172.21.15.66:0/1531085199 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0b1000fad0 con 0x7f0b140fe880 2023-07-21T22:34:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.301+0000 7f0b2f31bb80 1 Processor -- start 2023-07-21T22:34:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.302+0000 7f0b0a7fc700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0b000a1340 0x7f0b000a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.302+0000 7f0b0a7fc700 1 -- 172.21.15.66:0/1531085199 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0b100ce560 con 0x7f0b140fe880 2023-07-21T22:34:42.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.302+0000 7f0b2f31bb80 1 -- start start 2023-07-21T22:34:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.302+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7e7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.302+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e29b0 0x55c30f7e2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f6e7d30 con 0x55c30f7e7230 2023-07-21T22:34:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f73aac0 con 0x55c30f7e7b40 2023-07-21T22:34:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f733f50 con 0x55c30f7e29b0 2023-07-21T22:34:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7e7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34432/0 (socket says 172.21.15.66:34432) 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b1979b700 1 -- 172.21.15.66:0/3467729173 learned_addr learned my addr 172.21.15.66:0/3467729173 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e29b0 0x55c30f7e2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0b000a1340 0x7f0b000a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.303+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.304+0000 7f0b1979b700 1 -- 172.21.15.66:0/3467729173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e29b0 msgr2=0x55c30f7e2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.304+0000 7f0b1979b700 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e29b0 0x55c30f7e2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.304+0000 7f0b1979b700 1 -- 172.21.15.66:0/3467729173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 msgr2=0x55c30f7e23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.304+0000 7f0b1979b700 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7e23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.304+0000 7f0b1979b700 1 -- 172.21.15.66:0/3467729173 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c30f561e60 con 0x55c30f7e7230 2023-07-21T22:34:42.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.304+0000 7f0b1979b700 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7e7600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0b1009b020 tx=0x7f0b10009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790a45c63eae8348 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.305+0000 7f0aff7fe700 1 -- 172.21.15.66:0/3467729173 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1009f120 con 0x55c30f7e7230 2023-07-21T22:34:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.305+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0b000a1340 0x7f0b000a37f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009550 tx=0x7f0b0c009100 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.305+0000 7f0aff7fe700 1 -- 172.21.15.66:0/3467729173 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b1009f350 con 0x55c30f7e7230 2023-07-21T22:34:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.305+0000 7f0aff7fe700 1 -- 172.21.15.66:0/3467729173 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100a10b0 con 0x55c30f7e7230 2023-07-21T22:34:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3467729173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 msgr2=0x55c30f7e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.354 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7e7600 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0b1009b020 tx=0x7f0b10009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3467729173 shutdown_connections 2023-07-21T22:34:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e29b0 0x55c30f7e2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7e7600 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3467729173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.306+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3467729173 >> 172.21.15.66:0/3467729173 conn(0x55c30f7d7f70 msgr2=0x55c30f7dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.307+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3467729173 shutdown_connections 2023-07-21T22:34:42.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.307+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3467729173 wait complete. 2023-07-21T22:34:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.308+0000 7f0b2f31bb80 1 -- start start 2023-07-21T22:34:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.308+0000 7f0b2f31bb80 1 Processor -- start 2023-07-21T22:34:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7ebe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f6d3b40 con 0x55c30f7e7230 2023-07-21T22:34:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f6d3840 con 0x55c30f7e29b0 2023-07-21T22:34:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f5b79a0 con 0x55c30f7e7b40 2023-07-21T22:34:42.358 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.309+0000 7f0b2f31bb80 10 client.0 _ll_register_callbacks cb 0x55c30f6d17e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7eeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49088/0 (socket says 172.21.15.66:49088) 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b1979b700 1 -- 172.21.15.66:0/3207756236 learned_addr learned my addr 172.21.15.66:0/3207756236 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7ebe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b18f9a700 1 -- 172.21.15.66:0/3207756236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 msgr2=0x55c30f7ec3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.359 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ec3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b18f9a700 1 -- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 msgr2=0x55c30f7eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b18f9a700 1 -- 172.21.15.66:0/3207756236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b140712f0 con 0x55c30f7e7230 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.310+0000 7f0b1979b700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.311+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.311+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7ebe00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55c30f7deef0 tx=0x7f0b040129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedf293d5ea03bd9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.360 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.311+0000 7f0afcff9700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:34:42.361 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.312+0000 7f0afcff9700 1 -- 172.21.15.66:0/3207756236 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017070 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.312+0000 7f0afcff9700 1 -- 172.21.15.66:0/3207756236 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0b04004030 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.312+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c30f68d570 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.312+0000 7f0afcff9700 1 -- 172.21.15.66:0/3207756236 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b040041f0 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.312+0000 7f0b2f31bb80 10 client.17379 fetch_fsmap learned FSMap version 47 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.312+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55c30f54ace0 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0afcff9700 1 -- 172.21.15.66:0/3207756236 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c30f68d570 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0afcff9700 1 -- 172.21.15.66:0/3207756236 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 47) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0b04003820 con 0x55c30f7e7230 2023-07-21T22:34:42.362 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0b2f31bb80 10 client.17379 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:34:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0b2f31bb80 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0b2f31bb80 20 client.17379 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f7fc290 0x55c30f7fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0b2f31bb80 4 client.17379 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.313+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c30f72cf30 con 0x55c30f7fc290 2023-07-21T22:34:42.363 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.314+0000 7f0b1979b700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f7fc290 0x55c30f7fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.314+0000 7f0b1979b700 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f7fc290 0x55c30f7fe6a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99dcb1de690e2da server_cookie=d3f758c1d7ee57fe in_seq=0 out_seq=0 2023-07-21T22:34:42.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.314+0000 7f0afcff9700 10 client.17379 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.319+0000 7f0afcff9700 1 -- 172.21.15.66:0/3207756236 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c30f72cf30 con 0x55c30f7fc290 2023-07-21T22:34:42.366 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.319+0000 7f0afcff9700 10 client.17379 handle_command_reply: tid=0 2023-07-21T22:34:42.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.319+0000 7f0b2f31bb80 1 client.17379 shutdown 2023-07-21T22:34:42.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 msgr2=0x55c30f7ebe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7ebe00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55c30f7deef0 tx=0x7f0b040129b0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 shutdown_connections 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f7fc290 0x55c30f7fe6a0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ec3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/3207756236 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7ebe00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0afcff9700 0 client.17379 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 >> 172.21.15.66:0/3207756236 conn(0x55c30f7d7f70 msgr2=0x55c30f7dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 shutdown_connections 2023-07-21T22:34:42.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/3207756236 wait complete. 2023-07-21T22:34:42.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.320+0000 7f0b2f31bb80 20 client.17379 trim_cache size 0 max 16384 2023-07-21T22:34:42.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:42 smithi104 ceph-mon[129021]: pgmap v413: 129 pgs: 129 active+clean; 719 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 354 op/s 2023-07-21T22:34:42.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:42 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:42.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 Processor -- start 2023-07-21T22:34:42.394 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 -- start start 2023-07-21T22:34:42.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7f3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f68d570 con 0x55c30f7e7230 2023-07-21T22:34:42.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f7f9940 con 0x55c30f7e29b0 2023-07-21T22:34:42.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.347+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f7f8b80 con 0x55c30f7e7b40 2023-07-21T22:34:42.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7f3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49098/0 (socket says 172.21.15.66:49098) 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 -- 172.21.15.66:0/2011483066 learned_addr learned my addr 172.21.15.66:0/2011483066 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 -- 172.21.15.66:0/2011483066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 msgr2=0x55c30f7ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 -- 172.21.15.66:0/2011483066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 msgr2=0x55c30f7eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.398 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 -- 172.21.15.66:0/2011483066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c30f5882d0 con 0x55c30f7e29b0 2023-07-21T22:34:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.348+0000 7f0b1979b700 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7f3400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b10099670 tx=0x7f0b100aef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdbfd13301826191 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0afffff700 1 -- 172.21.15.66:0/2011483066 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1009f120 con 0x55c30f7e29b0 2023-07-21T22:34:42.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0afffff700 1 -- 172.21.15.66:0/2011483066 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b100225e0 con 0x55c30f7e29b0 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0afffff700 1 -- 172.21.15.66:0/2011483066 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10022850 con 0x55c30f7e29b0 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/2011483066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 msgr2=0x55c30f7f3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7f3400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b10099670 tx=0x7f0b100aef90 comp rx=0 tx=0).stop 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/2011483066 shutdown_connections 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e29b0 0x55c30f7f3400 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7b40 0x55c30f7ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/2011483066 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e7230 0x55c30f7eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.349+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/2011483066 >> 172.21.15.66:0/2011483066 conn(0x55c30f7d7f70 msgr2=0x55c30f7e0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.350+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/2011483066 shutdown_connections 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.350+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/2011483066 wait complete. 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.350+0000 7f0b2f31bb80 1 -- start start 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 Processor -- start 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e29b0 0x55c30f7f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 0x55c30f7f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7f2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f850120 con 0x55c30f7e29b0 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f850260 con 0x55c30f7e7b40 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c30f8503a0 con 0x55c30f7e7230 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.351+0000 7f0b2f31bb80 10 client.0 _ll_register_callbacks cb 0x7f0b04009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 0x55c30f7f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b19f9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e29b0 0x55c30f7f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 0x55c30f7f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43474/0 (socket says 172.21.15.66:43474) 2023-07-21T22:34:42.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b1979b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e29b0 0x55c30f7f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34458/0 (socket says 172.21.15.66:34458) 2023-07-21T22:34:42.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 -- 172.21.15.66:0/1805396932 learned_addr learned my addr 172.21.15.66:0/1805396932 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 -- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 msgr2=0x55c30f7f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.404 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 -- 172.21.15.66:0/1805396932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e29b0 msgr2=0x55c30f7f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e29b0 0x55c30f7f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b18f9a700 1 -- 172.21.15.66:0/1805396932 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c30f561e60 con 0x55c30f7e7230 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.352+0000 7f0b19f9c700 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0b18f9a700 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 0x55c30f7f1fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009d90 tx=0x7f0b04013f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadbc16fd785fd6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0afcff9700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0afcff9700 1 -- 172.21.15.66:0/1805396932 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b04017040 con 0x55c30f7e7230 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b1009f350 con 0x55c30f7e7230 2023-07-21T22:34:42.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0afcff9700 1 -- 172.21.15.66:0/1805396932 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0b04000e40 con 0x55c30f7e7230 2023-07-21T22:34:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0afcff9700 1 -- 172.21.15.66:0/1805396932 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b040050b0 con 0x55c30f7e7230 2023-07-21T22:34:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0b2f31bb80 10 client.27409 fetch_fsmap learned FSMap version 47 2023-07-21T22:34:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.353+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55c30f560330 con 0x55c30f7e7230 2023-07-21T22:34:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0afcff9700 1 -- 172.21.15.66:0/1805396932 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b1009f350 con 0x55c30f7e7230 2023-07-21T22:34:42.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0afcff9700 1 -- 172.21.15.66:0/1805396932 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 47) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0b0400fc20 con 0x55c30f7e7230 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b2f31bb80 10 client.27409 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b2f31bb80 10 client.27409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b2f31bb80 20 client.27409 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f851b20 0x55c30f853f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b2f31bb80 4 client.27409 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c30f6ef0d0 con 0x55c30f851b20 2023-07-21T22:34:42.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.354+0000 7f0b1979b700 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f851b20 0x55c30f853f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.356+0000 7f0b1979b700 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f851b20 0x55c30f853f30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ada680c2660e6b9 server_cookie=52baa11f6a670d10 in_seq=0 out_seq=0 2023-07-21T22:34:42.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.356+0000 7f0afcff9700 10 client.27409 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0afcff9700 1 -- 172.21.15.66:0/1805396932 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c30f6ef0d0 con 0x55c30f851b20 2023-07-21T22:34:42.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0afcff9700 10 client.27409 handle_command_reply: tid=0 2023-07-21T22:34:42.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 client.27409 shutdown 2023-07-21T22:34:42.411 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 msgr2=0x55c30f7f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 0x55c30f7f1fe0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009d90 tx=0x7f0b04013f20 comp rx=0 tx=0).stop 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 shutdown_connections 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c30f7e7b40 0x55c30f7f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c30f851b20 0x55c30f853f30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c30f7e29b0 0x55c30f7f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 --2- 172.21.15.66:0/1805396932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c30f7e7230 0x55c30f7f1fe0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.412 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 >> 172.21.15.66:0/1805396932 conn(0x55c30f7d7f70 msgr2=0x55c30f7ed710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.363+0000 7f0afcff9700 0 client.27409 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 shutdown_connections 2023-07-21T22:34:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b2f31bb80 1 -- 172.21.15.66:0/1805396932 wait complete. 2023-07-21T22:34:42.413 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b2f31bb80 20 client.27409 trim_cache size 0 max 16384 2023-07-21T22:34:42.413 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:34:42.413 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:34:42.414 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "71169ff2-86c0-46d9-a3b7-78c711d315d8", 2023-07-21T22:34:42.414 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:34:42.414 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:34:42.414 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0b000a1340 msgr2=0x7f0b000a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0b000a1340 0x7f0b000a37f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009550 tx=0x7f0b0c009100 comp rx=0 tx=0).stop 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 msgr2=0x7f0b14100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.364+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 0x7f0b14100b60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005910 tx=0x7f0b10005740 comp rx=0 tx=0).stop 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 shutdown_connections 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0b000a1340 0x7f0b000a37f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0b1410c430 0x7f0b141015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.415 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0b140fe880 0x7f0b14100b60 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 --2- 172.21.15.66:0/1531085199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b140ff190 0x7f0b141010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 >> 172.21.15.66:0/1531085199 conn(0x7f0b140fa4b0 msgr2=0x7f0b1410d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 shutdown_connections 2023-07-21T22:34:42.416 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.365+0000 7f0b1b9ff700 1 -- 172.21.15.66:0/1531085199 wait complete. 2023-07-21T22:34:42.429 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:42 smithi066 ceph-mon[126379]: pgmap v413: 129 pgs: 129 active+clean; 719 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 354 op/s 2023-07-21T22:34:42.429 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:42 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:42.431 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:34:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:42 smithi149 ceph-mon[129268]: pgmap v413: 129 pgs: 129 active+clean; 719 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 354 op/s 2023-07-21T22:34:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:42 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:42.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 -- 172.21.15.66:0/3833995707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c1015b0 msgr2=0x7fb93c109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3833995707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c1015b0 0x7fb93c109aa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb930009990 tx=0x7fb930009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 -- 172.21.15.66:0/3833995707 shutdown_connections 2023-07-21T22:34:42.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3833995707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb93c10a070 0x7fb93c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3833995707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3833995707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c1015b0 0x7fb93c109aa0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb930009990 tx=0x7fb930009ca0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 -- 172.21.15.66:0/3833995707 >> 172.21.15.66:0/3833995707 conn(0x7fb93c0fa4b0 msgr2=0x7fb93c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.649+0000 7fb9415ff700 1 -- 172.21.15.66:0/3833995707 shutdown_connections 2023-07-21T22:34:42.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.650+0000 7fb9415ff700 1 -- 172.21.15.66:0/3833995707 wait complete. 2023-07-21T22:34:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.651+0000 7fb9415ff700 1 Processor -- start 2023-07-21T22:34:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.651+0000 7fb9415ff700 1 -- start start 2023-07-21T22:34:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb9415ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb9415ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb93c10a070 0x7fb93c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb9415ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c19cb60 0x7fb93c19cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb9415ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19d5d0 con 0x7fb93c10a070 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb9415ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19d740 con 0x7fb93c19cb60 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb9415ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c19d900 con 0x7fb93c100ca0 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb93affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb93affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43502/0 (socket says 172.21.15.66:43502) 2023-07-21T22:34:42.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb93affd700 1 -- 172.21.15.66:0/3134655956 learned_addr learned my addr 172.21.15.66:0/3134655956 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb93c10a070 0x7fb93c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c19cb60 0x7fb93c19cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.652+0000 7fb93affd700 1 -- 172.21.15.66:0/3134655956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c19cb60 msgr2=0x7fb93c19cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb93affd700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c19cb60 0x7fb93c19cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb93affd700 1 -- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb93c10a070 msgr2=0x7fb93c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb93affd700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb93c10a070 0x7fb93c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb93affd700 1 -- 172.21.15.66:0/3134655956 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9300095f0 con 0x7fb93c100ca0 2023-07-21T22:34:42.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c19cb60 0x7fb93c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:34:42.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb93affd700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c1981c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d950 tx=0x7fb92c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb923fff700 1 -- 172.21.15.66:0/3134655956 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c009940 con 0x7fb93c100ca0 2023-07-21T22:34:42.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c101aa0 con 0x7fb93c100ca0 2023-07-21T22:34:42.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb923fff700 1 -- 172.21.15.66:0/3134655956 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92c010460 con 0x7fb93c100ca0 2023-07-21T22:34:42.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb923fff700 1 -- 172.21.15.66:0/3134655956 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00f620 con 0x7fb93c100ca0 2023-07-21T22:34:42.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.653+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c101f80 con 0x7fb93c100ca0 2023-07-21T22:34:42.706 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.659+0000 7fb923fff700 1 -- 172.21.15.66:0/3134655956 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb92c0108d0 con 0x7fb93c100ca0 2023-07-21T22:34:42.707 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.660+0000 7fb954e38b80 1 Processor -- start 2023-07-21T22:34:42.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.660+0000 7fb954e38b80 1 -- start start 2023-07-21T22:34:42.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.708 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb954e38b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b230b2cb0 con 0x560b231ad930 2023-07-21T22:34:42.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb954e38b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b23105a40 con 0x560b231b21b0 2023-07-21T22:34:42.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb954e38b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b230feed0 con 0x560b231b2ac0 2023-07-21T22:34:42.709 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb93b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb93affd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb93b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49136/0 (socket says 172.21.15.66:49136) 2023-07-21T22:34:42.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb93b7fe700 1 -- 172.21.15.66:0/4090485364 learned_addr learned my addr 172.21.15.66:0/4090485364 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.661+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb923fff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb9240a1340 0x7fb9240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.710 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb923fff700 1 -- 172.21.15.66:0/3134655956 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb92c0ccec0 con 0x7fb93c100ca0 2023-07-21T22:34:42.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb93affd700 1 -- 172.21.15.66:0/4090485364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 msgr2=0x560b231b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb93affd700 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb93affd700 1 -- 172.21.15.66:0/4090485364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 msgr2=0x560b231adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb93affd700 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.662+0000 7fb93affd700 1 -- 172.21.15.66:0/4090485364 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b22f2cde0 con 0x560b231b2ac0 2023-07-21T22:34:42.711 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb93affd700 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ad350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ca270 tx=0x7fb92c010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a410f3a219a1577 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb9240a1340 0x7fb9240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb9217fa700 1 -- 172.21.15.66:0/4090485364 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c099bf0 con 0x560b231b2ac0 2023-07-21T22:34:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb9217fa700 1 -- 172.21.15.66:0/4090485364 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92c09dc20 con 0x560b231b2ac0 2023-07-21T22:34:42.712 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.663+0000 7fb9217fa700 1 -- 172.21.15.66:0/4090485364 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c09a0b0 con 0x560b231b2ac0 2023-07-21T22:34:42.713 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 -- 172.21.15.66:0/4090485364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 msgr2=0x560b231ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb9240a1340 0x7fb9240a37f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb9340076a0 tx=0x7fb93400b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ad350 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ca270 tx=0x7fb92c010cd0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 -- 172.21.15.66:0/4090485364 shutdown_connections 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 --2- 172.21.15.66:0/4090485364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ad350 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 -- 172.21.15.66:0/4090485364 >> 172.21.15.66:0/4090485364 conn(0x560b231a2ef0 msgr2=0x560b231a8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 -- 172.21.15.66:0/4090485364 shutdown_connections 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.664+0000 7fb954e38b80 1 -- 172.21.15.66:0/4090485364 wait complete. 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.665+0000 7fb954e38b80 1 -- start start 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.665+0000 7fb954e38b80 1 Processor -- start 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231ad930 0x560b231b9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b2ac0 0x560b231b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b2309eac0 con 0x560b231b21b0 2023-07-21T22:34:42.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b2309e7c0 con 0x560b231b2ac0 2023-07-21T22:34:42.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b22f82920 con 0x560b231ad930 2023-07-21T22:34:42.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.666+0000 7fb954e38b80 10 client.0 _ll_register_callbacks cb 0x560b2309c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:42.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93a7fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:34488/0 (socket says 172.21.15.66:34488) 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93a7fc700 1 -- 172.21.15.66:0/2431964679 learned_addr learned my addr 172.21.15.66:0/2431964679 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b2ac0 0x560b231b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93affd700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231ad930 0x560b231b9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93affd700 1 -- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b2ac0 msgr2=0x560b231b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93affd700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b2ac0 0x560b231b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93affd700 1 -- 172.21.15.66:0/2431964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 msgr2=0x560b231b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93affd700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93affd700 1 -- 172.21.15.66:0/2431964679 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c10d170 con 0x560b231ad930 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b2ac0 0x560b231b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.667+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb93affd700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231ad930 0x560b231b9c40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x560b231ba270 tx=0x7fb92c096530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f2eb6912b2313f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb91eff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb91eff5700 1 -- 172.21.15.66:0/2431964679 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0a2040 con 0x560b231ad930 2023-07-21T22:34:42.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb91eff5700 1 -- 172.21.15.66:0/2431964679 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb92c0b1030 con 0x560b231ad930 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b230584f0 con 0x560b231ad930 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb91eff5700 1 -- 172.21.15.66:0/2431964679 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c099bf0 con 0x560b231ad930 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb954e38b80 10 client.27421 fetch_fsmap learned FSMap version 47 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.668+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7fb934009950 con 0x560b231ad930 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb91eff5700 1 -- 172.21.15.66:0/2431964679 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b230584f0 con 0x560b231ad930 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb91eff5700 1 -- 172.21.15.66:0/2431964679 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 47) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb92c099f30 con 0x560b231ad930 2023-07-21T22:34:42.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb954e38b80 10 client.27421 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb954e38b80 10 client.27421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb954e38b80 20 client.27421 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b231c6f60 0x560b231c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb954e38b80 4 client.27421 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b230f7eb0 con 0x560b231c6f60 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.669+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b231c6f60 0x560b231c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.670+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b231c6f60 0x560b231c9340 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afe5916ad1c88b8c server_cookie=4564606bec5c70ff in_seq=0 out_seq=0 2023-07-21T22:34:42.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.670+0000 7fb91eff5700 10 client.27421 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.675+0000 7fb91eff5700 1 -- 172.21.15.66:0/2431964679 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560b230f7eb0 con 0x560b231c6f60 2023-07-21T22:34:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.675+0000 7fb91eff5700 10 client.27421 handle_command_reply: tid=0 2023-07-21T22:34:42.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.675+0000 7fb954e38b80 1 client.27421 shutdown 2023-07-21T22:34:42.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231ad930 msgr2=0x560b231b9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231ad930 0x560b231b9c40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x560b231ba270 tx=0x7fb92c096530 comp rx=0 tx=0).stop 2023-07-21T22:34:42.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 shutdown_connections 2023-07-21T22:34:42.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b2ac0 0x560b231b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b231c6f60 0x560b231c9340 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231ad930 0x560b231b9c40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2431964679 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb91eff5700 0 client.27421 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 >> 172.21.15.66:0/2431964679 conn(0x560b231a2ef0 msgr2=0x560b231a83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 shutdown_connections 2023-07-21T22:34:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.676+0000 7fb954e38b80 1 -- 172.21.15.66:0/2431964679 wait complete. 2023-07-21T22:34:42.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.677+0000 7fb954e38b80 20 client.27421 trim_cache size 0 max 16384 2023-07-21T22:34:42.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 Processor -- start 2023-07-21T22:34:42.756 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 -- start start 2023-07-21T22:34:42.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.757 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b230584f0 con 0x560b231ad930 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b2319fec0 con 0x560b231b21b0 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.709+0000 7fb954e38b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b231a29b0 con 0x560b231b2ac0 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:43540/0 (socket says 172.21.15.66:43540) 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 -- 172.21.15.66:0/2859427987 learned_addr learned my addr 172.21.15.66:0/2859427987 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93affd700 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.758 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.759 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 -- 172.21.15.66:0/2859427987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 msgr2=0x560b231b97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 -- 172.21.15.66:0/2859427987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 msgr2=0x560b231be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231be4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.710+0000 7fb93b7fe700 1 -- 172.21.15.66:0/2859427987 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c101f80 con 0x560b231b2ac0 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.711+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ac9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb9340164c0 tx=0x7fb934014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5fb69c661d040cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.711+0000 7fb921ffb700 1 -- 172.21.15.66:0/2859427987 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934016c60 con 0x560b231b2ac0 2023-07-21T22:34:42.760 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.711+0000 7fb921ffb700 1 -- 172.21.15.66:0/2859427987 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb934016dc0 con 0x560b231b2ac0 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.711+0000 7fb921ffb700 1 -- 172.21.15.66:0/2859427987 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934015770 con 0x560b231b2ac0 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 -- 172.21.15.66:0/2859427987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 msgr2=0x560b231ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ac9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb9340164c0 tx=0x7fb934014000 comp rx=0 tx=0).stop 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 -- 172.21.15.66:0/2859427987 shutdown_connections 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231ac9f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231ad930 0x560b231be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.761 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 --2- 172.21.15.66:0/2859427987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231b21b0 0x560b231b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 -- 172.21.15.66:0/2859427987 >> 172.21.15.66:0/2859427987 conn(0x560b231a2ef0 msgr2=0x560b231c37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 -- 172.21.15.66:0/2859427987 shutdown_connections 2023-07-21T22:34:42.762 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.712+0000 7fb954e38b80 1 -- 172.21.15.66:0/2859427987 wait complete. 2023-07-21T22:34:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.713+0000 7fb954e38b80 1 -- start start 2023-07-21T22:34:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.713+0000 7fb954e38b80 1 Processor -- start 2023-07-21T22:34:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 0x560b231be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.764 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b231b6460 con 0x560b231b21b0 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b231b65d0 con 0x560b231ad930 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b2321aca0 con 0x560b231b2ac0 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb954e38b80 10 client.0 _ll_register_callbacks cb 0x7fb930009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 0x560b231be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 0x560b231be6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:49164/0 (socket says 172.21.15.66:49164) 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 -- 172.21.15.66:0/1835190277 learned_addr learned my addr 172.21.15.66:0/1835190277 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:34:42.765 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93b7fe700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 -- 172.21.15.66:0/1835190277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 msgr2=0x560b231b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231b5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 -- 172.21.15.66:0/1835190277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 msgr2=0x560b231b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93affd700 1 -- 172.21.15.66:0/1835190277 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93401e040 con 0x560b231ad930 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.714+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:34:42.766 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb93affd700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 0x560b231be6b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb92c095970 tx=0x7fb92c010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf30b35f4fe11d1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb91eff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb91eff5700 1 -- 172.21.15.66:0/1835190277 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0a1e40 con 0x560b231ad930 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb91eff5700 1 -- 172.21.15.66:0/1835190277 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb92c0b1040 con 0x560b231ad930 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb92c09dc20 con 0x560b231ad930 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb91eff5700 1 -- 172.21.15.66:0/1835190277 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c0999c0 con 0x560b231ad930 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb954e38b80 10 client.27464 fetch_fsmap learned FSMap version 47 2023-07-21T22:34:42.767 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x560b22f15c60 con 0x560b231ad930 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.715+0000 7fb91eff5700 1 -- 172.21.15.66:0/1835190277 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb92c09dc20 con 0x560b231ad930 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb91eff5700 1 -- 172.21.15.66:0/1835190277 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 47) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb92c0d0af0 con 0x560b231ad930 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb954e38b80 10 client.27464 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb954e38b80 10 client.27464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb954e38b80 20 client.27464 populate_metadata read hostname 'smithi066' 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb954e38b80 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b2321be50 0x560b2321e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:34:42.768 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb954e38b80 4 client.27464 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:34:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b230ba050 con 0x560b2321be50 2023-07-21T22:34:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b2321be50 0x560b2321e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:34:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb93a7fc700 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b2321be50 0x560b2321e230 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7f7aa942a1a530 server_cookie=65f080c4096b3577 in_seq=0 out_seq=0 2023-07-21T22:34:42.769 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.716+0000 7fb91eff5700 10 client.27464 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb91eff5700 1 -- 172.21.15.66:0/1835190277 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x560b230ba050 con 0x560b2321be50 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb91eff5700 10 client.27464 handle_command_reply: tid=0 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 client.27464 shutdown 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 msgr2=0x560b231be6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 0x560b231be6b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb92c095970 tx=0x7fb92c010cd0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 shutdown_connections 2023-07-21T22:34:42.770 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560b231ad930 0x560b231be6b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x560b2321be50 0x560b2321e230 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x560b231b2ac0 0x560b231b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 --2- 172.21.15.66:0/1835190277 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x560b231b21b0 0x560b231b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 >> 172.21.15.66:0/1835190277 conn(0x560b231a2ef0 msgr2=0x560b231ac9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb91eff5700 0 client.27464 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:34:42.771 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.722+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 shutdown_connections 2023-07-21T22:34:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.723+0000 7fb954e38b80 1 -- 172.21.15.66:0/1835190277 wait complete. 2023-07-21T22:34:42.772 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.723+0000 7fb954e38b80 20 client.27464 trim_cache size 0 max 16384 2023-07-21T22:34:42.772 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:34:42.772 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (450 inodes in the stack)", 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: "71169ff2-86c0-46d9-a3b7-78c711d315d8": { 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "71169ff2-86c0-46d9-a3b7-78c711d315d8", 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:34:42.773 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:34:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.723+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb9240a1340 msgr2=0x7fb9240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.723+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb9240a1340 0x7fb9240a37f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb9340076a0 tx=0x7fb93400b040 comp rx=0 tx=0).stop 2023-07-21T22:34:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.723+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 msgr2=0x7fb93c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:34:42.774 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.723+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c1981c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d950 tx=0x7fb92c00dc60 comp rx=0 tx=0).stop 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 shutdown_connections 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb93c19cb60 0x7fb93c19cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb9240a1340 0x7fb9240a37f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb93c100ca0 0x7fb93c1981c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 --2- 172.21.15.66:0/3134655956 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb93c10a070 0x7fb93c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 >> 172.21.15.66:0/3134655956 conn(0x7fb93c0fa4b0 msgr2=0x7fb93c0fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:34:42.775 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 shutdown_connections 2023-07-21T22:34:42.776 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:34:42.724+0000 7fb9415ff700 1 -- 172.21.15.66:0/3134655956 wait complete. 2023-07-21T22:34:42.791 INFO:tasks.cephfs.filesystem:scrub status for tag:71169ff2-86c0-46d9-a3b7-78c711d315d8 - {'path': '/', 'tag': '71169ff2-86c0-46d9-a3b7-78c711d315d8', 'options': 'recursive,force'} 2023-07-21T22:34:42.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 4148 42.35 MB/sec warmup 9 sec latency 8173.630 ms 2023-07-21T22:34:43.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:43 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:34:43.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:43 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:43.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:43 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:43.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:43 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:34:43.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:43 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:43.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:43 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:43.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:43 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:34:43.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:43 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:43.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:43 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:43.924 INFO:tasks.workunit.client.0.smithi066.stdout: 10 4797 42.07 MB/sec warmup 10 sec latency 434.681 ms 2023-07-21T22:34:44.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:44.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:44.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:44.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:44 smithi104 ceph-mon[129021]: pgmap v414: 129 pgs: 129 active+clean; 719 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2023-07-21T22:34:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:44.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:44.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:44.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:44 smithi066 ceph-mon[126379]: pgmap v414: 129 pgs: 129 active+clean; 719 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2023-07-21T22:34:44.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:44.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:44.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:44.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:44.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:44 smithi149 ceph-mon[129268]: pgmap v414: 129 pgs: 129 active+clean; 719 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 241 op/s 2023-07-21T22:34:44.925 INFO:tasks.workunit.client.0.smithi066.stdout: 10 5502 40.27 MB/sec warmup 11 sec latency 1434.784 ms 2023-07-21T22:34:45.925 INFO:tasks.workunit.client.0.smithi066.stdout: 10 6176 41.73 MB/sec warmup 12 sec latency 1733.531 ms 2023-07-21T22:34:46.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:46 smithi104 ceph-mon[129021]: pgmap v415: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2023-07-21T22:34:46.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:46 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:34:46.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:46 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:46.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:46 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:46.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:46 smithi066 ceph-mon[126379]: pgmap v415: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2023-07-21T22:34:46.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:46 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:34:46.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:46 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:46.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:46 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:46 smithi149 ceph-mon[129268]: pgmap v415: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.2 MiB/s wr, 372 op/s 2023-07-21T22:34:46.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:46 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:34:46.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:46 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:46.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:46 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:34:46.925 INFO:tasks.workunit.client.0.smithi066.stdout: 10 6631 41.86 MB/sec warmup 13 sec latency 159.187 ms 2023-07-21T22:34:47.925 INFO:tasks.workunit.client.0.smithi066.stdout: 10 6806 39.57 MB/sec warmup 14 sec latency 119.376 ms 2023-07-21T22:34:48.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:48 smithi104 ceph-mon[129021]: pgmap v416: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2023-07-21T22:34:48.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:48 smithi066 ceph-mon[126379]: pgmap v416: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2023-07-21T22:34:48.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:48 smithi149 ceph-mon[129268]: pgmap v416: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2023-07-21T22:34:48.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 7005 37.08 MB/sec warmup 15 sec latency 108.262 ms 2023-07-21T22:34:49.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 7244 35.23 MB/sec warmup 16 sec latency 347.041 ms 2023-07-21T22:34:50.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:50 smithi104 ceph-mon[129021]: pgmap v417: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2023-07-21T22:34:50.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:50 smithi066 ceph-mon[126379]: pgmap v417: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2023-07-21T22:34:50.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:50 smithi149 ceph-mon[129268]: pgmap v417: 129 pgs: 129 active+clean; 736 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2023-07-21T22:34:50.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 7725 34.35 MB/sec warmup 17 sec latency 125.149 ms 2023-07-21T22:34:51.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:51 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:34:51.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:51 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:34:51.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:51 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:34:51.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 8423 33.98 MB/sec warmup 18 sec latency 243.031 ms 2023-07-21T22:34:52.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:52 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:34:52.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:52 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:34:52.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:52 smithi066 ceph-mon[126379]: pgmap v418: 129 pgs: 129 active+clean; 746 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 389 op/s 2023-07-21T22:34:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:52 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:34:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:52 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:34:52.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:52 smithi149 ceph-mon[129268]: pgmap v418: 129 pgs: 129 active+clean; 746 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 389 op/s 2023-07-21T22:34:52.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:52 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:34:52.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:52 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:34:52.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:52 smithi104 ceph-mon[129021]: pgmap v418: 129 pgs: 129 active+clean; 746 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 389 op/s 2023-07-21T22:34:52.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 8866 33.17 MB/sec warmup 19 sec latency 387.193 ms 2023-07-21T22:34:53.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 9458 33.97 MB/sec warmup 20 sec latency 119.586 ms 2023-07-21T22:34:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:54 smithi066 ceph-mon[126379]: pgmap v419: 129 pgs: 129 active+clean; 746 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 245 op/s 2023-07-21T22:34:54.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:54 smithi149 ceph-mon[129268]: pgmap v419: 129 pgs: 129 active+clean; 746 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 245 op/s 2023-07-21T22:34:54.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:54 smithi104 ceph-mon[129021]: pgmap v419: 129 pgs: 129 active+clean; 746 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 245 op/s 2023-07-21T22:34:54.926 INFO:tasks.workunit.client.0.smithi066.stdout: 10 9978 34.60 MB/sec warmup 21 sec latency 86.965 ms 2023-07-21T22:34:55.927 INFO:tasks.workunit.client.0.smithi066.stdout: 10 10205 33.57 MB/sec warmup 22 sec latency 93.146 ms 2023-07-21T22:34:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:56 smithi066 ceph-mon[126379]: pgmap v420: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 429 op/s 2023-07-21T22:34:56.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:56 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:34:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:56 smithi149 ceph-mon[129268]: pgmap v420: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 429 op/s 2023-07-21T22:34:56.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:56 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:34:56.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:56 smithi104 ceph-mon[129021]: pgmap v420: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 429 op/s 2023-07-21T22:34:56.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:56 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:34:56.927 INFO:tasks.workunit.client.0.smithi066.stdout: 10 10412 32.34 MB/sec warmup 23 sec latency 257.540 ms 2023-07-21T22:34:57.465 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:57 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:57.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:57 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:57.625 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:57 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:34:57.928 INFO:tasks.workunit.client.0.smithi066.stdout: 10 10489 31.03 MB/sec warmup 24 sec latency 542.814 ms 2023-07-21T22:34:58.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:58 smithi066 ceph-mon[126379]: pgmap v421: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 298 op/s 2023-07-21T22:34:58.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:58 smithi149 ceph-mon[129268]: pgmap v421: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 298 op/s 2023-07-21T22:34:58.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:58 smithi104 ceph-mon[129021]: pgmap v421: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 298 op/s 2023-07-21T22:34:58.928 INFO:tasks.workunit.client.0.smithi066.stdout: 10 10890 30.58 MB/sec warmup 25 sec latency 999.318 ms 2023-07-21T22:34:59.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:34:59 smithi066 ceph-mon[126379]: pgmap v422: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 298 op/s 2023-07-21T22:34:59.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:34:59 smithi149 ceph-mon[129268]: pgmap v422: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 298 op/s 2023-07-21T22:34:59.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:34:59 smithi104 ceph-mon[129021]: pgmap v422: 129 pgs: 129 active+clean; 834 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 298 op/s 2023-07-21T22:34:59.928 INFO:tasks.workunit.client.0.smithi066.stdout: 10 11360 30.38 MB/sec warmup 26 sec latency 128.966 ms 2023-07-21T22:35:00.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:00 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:00.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:00 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:00.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:00 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:35:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:00 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:00 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:00.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:00 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:35:00.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:00 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:35:00.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:00 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:35:00.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:00 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:35:00.928 INFO:tasks.workunit.client.0.smithi066.stdout: 10 11702 29.61 MB/sec warmup 27 sec latency 142.155 ms 2023-07-21T22:35:01.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:01 smithi066 ceph-mon[126379]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 531 op/s 2023-07-21T22:35:01.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:01 smithi149 ceph-mon[129268]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 531 op/s 2023-07-21T22:35:01.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:01 smithi104 ceph-mon[129021]: pgmap v423: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 28 MiB/s wr, 531 op/s 2023-07-21T22:35:01.928 INFO:tasks.workunit.client.0.smithi066.stdout: 10 12198 29.46 MB/sec warmup 28 sec latency 747.724 ms 2023-07-21T22:35:02.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:02 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:35:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:02 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:35:02.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:02 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:35:02.928 INFO:tasks.workunit.client.0.smithi066.stdout: 10 12440 28.66 MB/sec warmup 29 sec latency 1748.459 ms 2023-07-21T22:35:03.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 12802 28.32 MB/sec warmup 30 sec latency 2748.514 ms 2023-07-21T22:35:04.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:04 smithi104 ceph-mon[129021]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 26 MiB/s wr, 417 op/s 2023-07-21T22:35:04.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:04 smithi066 ceph-mon[126379]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 26 MiB/s wr, 417 op/s 2023-07-21T22:35:04.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:04 smithi149 ceph-mon[129268]: pgmap v424: 129 pgs: 129 active+clean; 1.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 26 MiB/s wr, 417 op/s 2023-07-21T22:35:04.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 13684 29.62 MB/sec warmup 31 sec latency 3748.559 ms 2023-07-21T22:35:05.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 14090 29.18 MB/sec warmup 32 sec latency 4748.634 ms 2023-07-21T22:35:06.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:06 smithi104 ceph-mon[129021]: pgmap v425: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 28 MiB/s wr, 809 op/s 2023-07-21T22:35:06.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:06 smithi066 ceph-mon[126379]: pgmap v425: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 28 MiB/s wr, 809 op/s 2023-07-21T22:35:06.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:06 smithi149 ceph-mon[129268]: pgmap v425: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 28 MiB/s wr, 809 op/s 2023-07-21T22:35:06.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 14497 28.65 MB/sec warmup 33 sec latency 5748.680 ms 2023-07-21T22:35:07.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:07 smithi104 ceph-mon[129021]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T22:35:07.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:07 smithi066 ceph-mon[126379]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T22:35:07.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:07 smithi149 ceph-mon[129268]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T22:35:07.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 15506 29.30 MB/sec warmup 34 sec latency 6748.723 ms 2023-07-21T22:35:08.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:08 smithi104 ceph-mon[129021]: pgmap v427: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 750 op/s 2023-07-21T22:35:08.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:08 smithi066 ceph-mon[126379]: pgmap v427: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 750 op/s 2023-07-21T22:35:08.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:08 smithi149 ceph-mon[129268]: pgmap v427: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 750 op/s 2023-07-21T22:35:08.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 16425 30.52 MB/sec warmup 35 sec latency 7748.768 ms 2023-07-21T22:35:09.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 16942 30.16 MB/sec warmup 36 sec latency 8748.832 ms 2023-07-21T22:35:10.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:10 smithi104 ceph-mon[129021]: pgmap v428: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 750 op/s 2023-07-21T22:35:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:10 smithi066 ceph-mon[126379]: pgmap v428: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 750 op/s 2023-07-21T22:35:10.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:10 smithi149 ceph-mon[129268]: pgmap v428: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 23 MiB/s wr, 750 op/s 2023-07-21T22:35:10.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 17758 30.45 MB/sec warmup 37 sec latency 9748.929 ms 2023-07-21T22:35:11.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 18520 30.58 MB/sec warmup 38 sec latency 10748.971 ms 2023-07-21T22:35:12.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:12 smithi104 ceph-mon[129021]: pgmap v429: 129 pgs: 129 active+clean; 994 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T22:35:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:12 smithi066 ceph-mon[126379]: pgmap v429: 129 pgs: 129 active+clean; 994 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T22:35:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:12 smithi149 ceph-mon[129268]: pgmap v429: 129 pgs: 129 active+clean; 994 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T22:35:12.792 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:35:12.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 19399 31.41 MB/sec warmup 39 sec latency 11749.020 ms 2023-07-21T22:35:13.087 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.040+0000 7f58147ff700 1 -- 172.21.15.66:0/1984432595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c102fe0 msgr2=0x7f580c10abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.040+0000 7f58147ff700 1 --2- 172.21.15.66:0/1984432595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c102fe0 0x7f580c10abb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5804009f90 tx=0x7f580400b330 comp rx=0 tx=0).stop 2023-07-21T22:35:13.088 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.041+0000 7f58147ff700 1 -- 172.21.15.66:0/1984432595 shutdown_connections 2023-07-21T22:35:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.041+0000 7f58147ff700 1 --2- 172.21.15.66:0/1984432595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c102fe0 0x7f580c10abb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.041+0000 7f58147ff700 1 --2- 172.21.15.66:0/1984432595 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c0fdff0 0x7f580c0fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.041+0000 7f58147ff700 1 --2- 172.21.15.66:0/1984432595 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f580c0fe990 0x7f580c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.089 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.041+0000 7f58147ff700 1 -- 172.21.15.66:0/1984432595 >> 172.21.15.66:0/1984432595 conn(0x7f580c075b70 msgr2=0x7f580c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.093 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.042+0000 7f58147ff700 1 -- 172.21.15.66:0/1984432595 shutdown_connections 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.042+0000 7f58147ff700 1 -- 172.21.15.66:0/1984432595 wait complete. 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.043+0000 7f58147ff700 1 Processor -- start 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.043+0000 7f58147ff700 1 -- start start 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f58147ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c0fdff0 0x7f580c105ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f58147ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f580c0fe990 0x7f580c106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f58147ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 0x7f580c106970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.094 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f58147ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c112890 con 0x7f580c0fe990 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f58147ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c112a00 con 0x7f580c0fdff0 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f58147ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c112bc0 con 0x7f580c102fe0 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f5812d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 0x7f580c106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f581259b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c0fdff0 0x7f580c105ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f5812d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 0x7f580c106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57564/0 (socket says 172.21.15.66:57564) 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.045+0000 7f5812d9c700 1 -- 172.21.15.66:0/1687211237 learned_addr learned my addr 172.21.15.66:0/1687211237 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:13.095 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.046+0000 7f5812d9c700 1 -- 172.21.15.66:0/1687211237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c0fdff0 msgr2=0x7f580c105ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.046+0000 7f5812d9c700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c0fdff0 0x7f580c105ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.046+0000 7f5812d9c700 1 -- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f580c0fe990 msgr2=0x7f580c106430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.046+0000 7f5812d9c700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f580c0fe990 0x7f580c106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.096 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.046+0000 7f5812d9c700 1 -- 172.21.15.66:0/1687211237 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804009c40 con 0x7f580c102fe0 2023-07-21T22:35:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.046+0000 7f5812d9c700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 0x7f580c106970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5804009000 tx=0x7f5804004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.047+0000 7f58037fe700 1 -- 172.21.15.66:0/1687211237 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400e040 con 0x7f580c102fe0 2023-07-21T22:35:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.047+0000 7f58037fe700 1 -- 172.21.15.66:0/1687211237 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f580400bbf0 con 0x7f580c102fe0 2023-07-21T22:35:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.047+0000 7f58037fe700 1 -- 172.21.15.66:0/1687211237 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58040052a0 con 0x7f580c102fe0 2023-07-21T22:35:13.097 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.047+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c112e40 con 0x7f580c102fe0 2023-07-21T22:35:13.098 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.047+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c113350 con 0x7f580c102fe0 2023-07-21T22:35:13.100 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.053+0000 7f5828138b80 1 Processor -- start 2023-07-21T22:35:13.104 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.056+0000 7f5828138b80 1 -- start start 2023-07-21T22:35:13.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.057+0000 7f5828138b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.057+0000 7f5828138b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e99350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.105 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.057+0000 7f5828138b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e99da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.057+0000 7f5828138b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2d9ecb0 con 0x55d2a2e9eac0 2023-07-21T22:35:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.057+0000 7f5828138b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2df1a40 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.057+0000 7f5828138b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2deaed0 con 0x55d2a2e99930 2023-07-21T22:35:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38388/0 (socket says 172.21.15.66:38388) 2023-07-21T22:35:13.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 -- 172.21.15.66:0/1817054798 learned_addr learned my addr 172.21.15.66:0/1817054798 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:13.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 -- 172.21.15.66:0/1817054798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 msgr2=0x55d2a2e99da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e99da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 -- 172.21.15.66:0/1817054798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 msgr2=0x55d2a2e99350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e99350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.058+0000 7f5811d9a700 1 -- 172.21.15.66:0/1817054798 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2a2c18de0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.059+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e9e580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00db80 tx=0x7f57fc00de90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15affbed52c9b30d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.059+0000 7f5800ff9700 1 -- 172.21.15.66:0/1817054798 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00fbf0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.059+0000 7f5800ff9700 1 -- 172.21.15.66:0/1817054798 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57fc00fd50 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.059+0000 7f5800ff9700 1 -- 172.21.15.66:0/1817054798 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc0107b0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.060+0000 7f5828138b80 1 -- 172.21.15.66:0/1817054798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 msgr2=0x55d2a2e9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.060+0000 7f5828138b80 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e9e580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00db80 tx=0x7f57fc00de90 comp rx=0 tx=0).stop 2023-07-21T22:35:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f58037fe700 1 -- 172.21.15.66:0/1687211237 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f580401d470 con 0x7f580c102fe0 2023-07-21T22:35:13.109 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 -- 172.21.15.66:0/1817054798 shutdown_connections 2023-07-21T22:35:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e9e580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e99da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 --2- 172.21.15.66:0/1817054798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e99350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 -- 172.21.15.66:0/1817054798 >> 172.21.15.66:0/1817054798 conn(0x55d2a2e8eef0 msgr2=0x55d2a2e94830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 -- 172.21.15.66:0/1817054798 shutdown_connections 2023-07-21T22:35:13.110 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.061+0000 7f5828138b80 1 -- 172.21.15.66:0/1817054798 wait complete. 2023-07-21T22:35:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.062+0000 7f5828138b80 1 -- start start 2023-07-21T22:35:13.111 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.063+0000 7f5828138b80 1 Processor -- start 2023-07-21T22:35:13.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2ea5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.112 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2d8aac0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2d8a7c0 con 0x55d2a2e9eac0 2023-07-21T22:35:13.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2c6e920 con 0x55d2a2e99930 2023-07-21T22:35:13.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.065+0000 7f5828138b80 10 client.0 _ll_register_callbacks cb 0x55d2a2d88760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:13.113 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.066+0000 7f5811d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2ea5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.066+0000 7f5811d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2ea5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57578/0 (socket says 172.21.15.66:57578) 2023-07-21T22:35:13.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.066+0000 7f5811d9a700 1 -- 172.21.15.66:0/3339743134 learned_addr learned my addr 172.21.15.66:0/3339743134 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:13.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.066+0000 7f5811d9a700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.114 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.066+0000 7f581259b700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.067+0000 7f581259b700 1 -- 172.21.15.66:0/3339743134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 msgr2=0x55d2a2ea5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.067+0000 7f581259b700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2ea5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.067+0000 7f581259b700 1 -- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9eac0 msgr2=0x55d2a2ea3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.116 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.067+0000 7f581259b700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.067+0000 7f581259b700 1 -- 172.21.15.66:0/3339743134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc00d860 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.067+0000 7f581259b700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea2e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55d2a2e95e70 tx=0x7f5808022950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ac57c0ecbc98c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.068+0000 7f58037fe700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f57f80a1230 0x7f57f80a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.117 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.068+0000 7f58037fe700 1 -- 172.21.15.66:0/1687211237 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f58040cdea0 con 0x7f580c102fe0 2023-07-21T22:35:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.069+0000 7f57f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:35:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.069+0000 7f57f5ffb700 1 -- 172.21.15.66:0/3339743134 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808004260 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.069+0000 7f57f5ffb700 1 -- 172.21.15.66:0/3339743134 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f58080043c0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.069+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2a2d444f0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.118 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.069+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f57f80a1230 0x7f57f80a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.070+0000 7f57f5ffb700 1 -- 172.21.15.66:0/3339743134 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5808004580 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.070+0000 7f5828138b80 10 client.17409 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:13.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.070+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d2a2c01c60 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f57f5ffb700 1 -- 172.21.15.66:0/3339743134 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2a2d444f0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f57f5ffb700 1 -- 172.21.15.66:0/3339743134 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f580802b8a0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5828138b80 10 client.17409 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5828138b80 10 client.17409 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:13.120 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5828138b80 20 client.17409 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5828138b80 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2eb3550 0x55d2a2eb5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5828138b80 4 client.17409 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2a2de3eb0 con 0x55d2a2eb3550 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.071+0000 7f5811d9a700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2eb3550 0x55d2a2eb5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.072+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f57f80a1230 0x7f57f80a36e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57fc011f10 tx=0x7f57fc004750 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.073+0000 7f5811d9a700 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2eb3550 0x55d2a2eb5960 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f512a539637f6b server_cookie=4a4eae55bdf57808 in_seq=0 out_seq=0 2023-07-21T22:35:13.121 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.073+0000 7f57f5ffb700 10 client.17409 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:13.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.077+0000 7f57f5ffb700 1 -- 172.21.15.66:0/3339743134 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2a2de3eb0 con 0x55d2a2eb3550 2023-07-21T22:35:13.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.077+0000 7f57f5ffb700 10 client.17409 handle_command_reply: tid=0 2023-07-21T22:35:13.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.077+0000 7f5828138b80 1 client.17409 shutdown 2023-07-21T22:35:13.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9e1b0 msgr2=0x55d2a2ea2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea2e10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55d2a2e95e70 tx=0x7f5808022950 comp rx=0 tx=0).stop 2023-07-21T22:35:13.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 shutdown_connections 2023-07-21T22:35:13.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2eb3550 0x55d2a2eb5960 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.126 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2ea5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 --2- 172.21.15.66:0/3339743134 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea2e10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 >> 172.21.15.66:0/3339743134 conn(0x55d2a2e8eef0 msgr2=0x55d2a2e94420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f57f5ffb700 0 client.17409 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 shutdown_connections 2023-07-21T22:35:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 1 -- 172.21.15.66:0/3339743134 wait complete. 2023-07-21T22:35:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.078+0000 7f5828138b80 20 client.17409 trim_cache size 0 max 16384 2023-07-21T22:35:13.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 Processor -- start 2023-07-21T22:35:13.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 -- start start 2023-07-21T22:35:13.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e97aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2de3eb0 con 0x55d2a2e9eac0 2023-07-21T22:35:13.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2d444f0 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.102+0000 7f5828138b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2ea6360 con 0x55d2a2e99930 2023-07-21T22:35:13.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f5811d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f581259b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e97aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f5811d9a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e942d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38400/0 (socket says 172.21.15.66:38400) 2023-07-21T22:35:13.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f5811d9a700 1 -- 172.21.15.66:0/1653299241 learned_addr learned my addr 172.21.15.66:0/1653299241 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:13.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f5812d9c700 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f581259b700 1 -- 172.21.15.66:0/1653299241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 msgr2=0x55d2a2e942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f581259b700 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f581259b700 1 -- 172.21.15.66:0/1653299241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 msgr2=0x55d2a2e94810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f581259b700 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e94810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.103+0000 7f581259b700 1 -- 172.21.15.66:0/1653299241 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2a2e93230 con 0x55d2a2e99930 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.104+0000 7f581259b700 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e97aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55d2a2e9dec0 tx=0x7f580802bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f970f7aa6967a8c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.104+0000 7f58017fa700 1 -- 172.21.15.66:0/1653299241 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808005960 con 0x55d2a2e99930 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.104+0000 7f58017fa700 1 -- 172.21.15.66:0/1653299241 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f580802b440 con 0x55d2a2e99930 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.104+0000 7f58017fa700 1 -- 172.21.15.66:0/1653299241 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808003680 con 0x55d2a2e99930 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 -- 172.21.15.66:0/1653299241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 msgr2=0x55d2a2e97aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e97aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55d2a2e9dec0 tx=0x7f580802bf70 comp rx=0 tx=0).stop 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 -- 172.21.15.66:0/1653299241 shutdown_connections 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e9eac0 0x55d2a2e94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e99930 0x55d2a2e97aa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 --2- 172.21.15.66:0/1653299241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2e942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 -- 172.21.15.66:0/1653299241 >> 172.21.15.66:0/1653299241 conn(0x55d2a2e8eef0 msgr2=0x55d2a2eaed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 -- 172.21.15.66:0/1653299241 shutdown_connections 2023-07-21T22:35:13.155 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.105+0000 7f5828138b80 1 -- 172.21.15.66:0/1653299241 wait complete. 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.106+0000 7f5828138b80 1 -- start start 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.106+0000 7f5828138b80 1 Processor -- start 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e99930 0x55d2a2eaa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2ea96f0 con 0x55d2a2e99930 2023-07-21T22:35:13.157 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2ea9860 con 0x55d2a2e9e1b0 2023-07-21T22:35:13.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2a2f07450 con 0x55d2a2e9eac0 2023-07-21T22:35:13.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5828138b80 10 client.0 _ll_register_callbacks cb 0x7f580801a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:13.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5812d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5812d9c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57586/0 (socket says 172.21.15.66:57586) 2023-07-21T22:35:13.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5812d9c700 1 -- 172.21.15.66:0/1384921794 learned_addr learned my addr 172.21.15.66:0/1384921794 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:13.158 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f581259b700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e99930 0x55d2a2eaa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.107+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f581259b700 1 -- 172.21.15.66:0/1384921794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e9eac0 msgr2=0x55d2a2ea9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f581259b700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f581259b700 1 -- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 msgr2=0x55d2a2ea89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f581259b700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f581259b700 1 -- 172.21.15.66:0/1384921794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2a2c01c60 con 0x55d2a2e99930 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f581259b700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e99930 0x55d2a2eaa720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f580801afa0 tx=0x7f5808023ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eb734fe613ac80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.159 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.108+0000 7f57f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f57f5ffb700 1 -- 172.21.15.66:0/1384921794 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080059c0 con 0x55d2a2e99930 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57fc00fd50 con 0x55d2a2e99930 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f57f5ffb700 1 -- 172.21.15.66:0/1384921794 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5808000e40 con 0x55d2a2e99930 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f57f5ffb700 1 -- 172.21.15.66:0/1384921794 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5808005300 con 0x55d2a2e99930 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f5828138b80 10 client.17415 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d2a2c3f250 con 0x55d2a2e99930 2023-07-21T22:35:13.160 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.109+0000 7f57f5ffb700 1 -- 172.21.15.66:0/1384921794 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57fc00fd50 con 0x55d2a2e99930 2023-07-21T22:35:13.161 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f57f5ffb700 1 -- 172.21.15.66:0/1384921794 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5808003a30 con 0x55d2a2e99930 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5828138b80 10 client.17415 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5828138b80 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5828138b80 20 client.17415 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5828138b80 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2f08700 0x55d2a2f0aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5828138b80 4 client.17415 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2a2da6050 con 0x55d2a2f08700 2023-07-21T22:35:13.162 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.110+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2f08700 0x55d2a2f0aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.111+0000 7f5811d9a700 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2f08700 0x55d2a2f0aae0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e19e64b745594da server_cookie=286b5c229f31452 in_seq=0 out_seq=0 2023-07-21T22:35:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.111+0000 7f57f5ffb700 10 client.17415 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.115+0000 7f57f5ffb700 1 -- 172.21.15.66:0/1384921794 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d2a2da6050 con 0x55d2a2f08700 2023-07-21T22:35:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.115+0000 7f57f5ffb700 10 client.17415 handle_command_reply: tid=0 2023-07-21T22:35:13.163 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 client.17415 shutdown 2023-07-21T22:35:13.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e99930 msgr2=0x55d2a2eaa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e99930 0x55d2a2eaa720 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f580801afa0 tx=0x7f5808023ea0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 shutdown_connections 2023-07-21T22:35:13.164 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55d2a2f08700 0x55d2a2f0aae0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d2a2e9eac0 0x55d2a2ea9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55d2a2e99930 0x55d2a2eaa720 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 --2- 172.21.15.66:0/1384921794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2a2e9e1b0 0x55d2a2ea89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.116+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 >> 172.21.15.66:0/1384921794 conn(0x55d2a2e8eef0 msgr2=0x55d2a2e97aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.165 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.117+0000 7f57f5ffb700 0 client.17415 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:13.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.117+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 shutdown_connections 2023-07-21T22:35:13.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.117+0000 7f5828138b80 1 -- 172.21.15.66:0/1384921794 wait complete. 2023-07-21T22:35:13.166 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.117+0000 7f5828138b80 20 client.17415 trim_cache size 0 max 16384 2023-07-21T22:35:13.166 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:35:13.166 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:35:13.167 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:35:13.167 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:35:13.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f57f80a1230 msgr2=0x7f57f80a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f57f80a1230 0x7f57f80a36e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f57fc011f10 tx=0x7f57fc004750 comp rx=0 tx=0).stop 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 msgr2=0x7f580c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 0x7f580c106970 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5804009000 tx=0x7f5804004b00 comp rx=0 tx=0).stop 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 shutdown_connections 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f580c0fdff0 0x7f580c105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f57f80a1230 0x7f57f80a36e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f580c102fe0 0x7f580c106970 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 --2- 172.21.15.66:0/1687211237 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f580c0fe990 0x7f580c106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 >> 172.21.15.66:0/1687211237 conn(0x7f580c075b70 msgr2=0x7f580c0fb680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 shutdown_connections 2023-07-21T22:35:13.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.118+0000 7f58147ff700 1 -- 172.21.15.66:0/1687211237 wait complete. 2023-07-21T22:35:13.186 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:35:13.186 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:35:13.448 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.401+0000 7f55ba9ff700 1 -- 172.21.15.66:0/3904091586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 msgr2=0x7f55b4106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.449 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.401+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/3904091586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b4106910 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55a80099c0 tx=0x7f55a8009cd0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.402+0000 7f55ba9ff700 1 -- 172.21.15.66:0/3904091586 shutdown_connections 2023-07-21T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.402+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/3904091586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55b4106ff0 0x7f55b410ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.402+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/3904091586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 0x7f55b4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.450 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.402+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/3904091586 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b4106910 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.402+0000 7f55ba9ff700 1 -- 172.21.15.66:0/3904091586 >> 172.21.15.66:0/3904091586 conn(0x7f55b40fd800 msgr2=0x7f55b40ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.404+0000 7f55ba9ff700 1 -- 172.21.15.66:0/3904091586 shutdown_connections 2023-07-21T22:35:13.451 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.404+0000 7f55ba9ff700 1 -- 172.21.15.66:0/3904091586 wait complete. 2023-07-21T22:35:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.405+0000 7f55ba9ff700 1 Processor -- start 2023-07-21T22:35:13.453 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.406+0000 7f55ba9ff700 1 -- start start 2023-07-21T22:35:13.454 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.406+0000 7f55ba9ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 0x7f55b4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.455 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.408+0000 7f55ba9ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.408+0000 7f55ba9ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55b4106ff0 0x7f55b419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.408+0000 7f55ba9ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b419ce80 con 0x7f55b4102950 2023-07-21T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.408+0000 7f55ba9ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b419cff0 con 0x7f55b4106ff0 2023-07-21T22:35:13.456 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.408+0000 7f55ba9ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b419d1b0 con 0x7f55b4101fb0 2023-07-21T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.409+0000 7f55b3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 0x7f55b4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.409+0000 7f55b3fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 0x7f55b4197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57606/0 (socket says 172.21.15.66:57606) 2023-07-21T22:35:13.457 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.409+0000 7f55b3fff700 1 -- 172.21.15.66:0/825424941 learned_addr learned my addr 172.21.15.66:0/825424941 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:13.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.411+0000 7f55b37fe700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.412+0000 7f55b37fe700 1 -- 172.21.15.66:0/825424941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 msgr2=0x7f55b4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.412+0000 7f55b37fe700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 0x7f55b4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.412+0000 7f55b37fe700 1 -- 172.21.15.66:0/825424941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55b4106ff0 msgr2=0x7f55b419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.412+0000 7f55b37fe700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55b4106ff0 0x7f55b419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.412+0000 7f55b37fe700 1 -- 172.21.15.66:0/825424941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a80095f0 con 0x7f55b4102950 2023-07-21T22:35:13.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.414+0000 7f55b37fe700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b41983e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009fd0 tx=0x7f55a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.415+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a801d070 con 0x7f55b4102950 2023-07-21T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.415+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55a80047d0 con 0x7f55b4102950 2023-07-21T22:35:13.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.415+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8005480 con 0x7f55b4102950 2023-07-21T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.415+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b419d430 con 0x7f55b4102950 2023-07-21T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.415+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b419d680 con 0x7f55b4102950 2023-07-21T22:35:13.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.417+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a0005320 con 0x7f55b4102950 2023-07-21T22:35:13.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.420+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f55a800fa80 con 0x7f55b4102950 2023-07-21T22:35:13.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.421+0000 7f55b17fa700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f559c0a13c0 0x7f559c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.421+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f55a80cd650 con 0x7f55b4102950 2023-07-21T22:35:13.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.422+0000 7f55b3fff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f559c0a13c0 0x7f559c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:13.470 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.423+0000 7f55b3fff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f559c0a13c0 0x7f559c0a3870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009d20 tx=0x7f55a4009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:13.474 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.426+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f55a80966f0 con 0x7f55b4102950 2023-07-21T22:35:13.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.798+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55a00059f0 con 0x7f55b4102950 2023-07-21T22:35:13.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.800+0000 7f55b17fa700 1 -- 172.21.15.66:0/825424941 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f55a8099d10 con 0x7f55b4102950 2023-07-21T22:35:13.848 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:35:13.848 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:35:01.110945+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:35:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f559c0a13c0 msgr2=0x7f559c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f559c0a13c0 0x7f559c0a3870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009d20 tx=0x7f55a4009260 comp rx=0 tx=0).stop 2023-07-21T22:35:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 msgr2=0x7f55b41983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:13.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b41983e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009fd0 tx=0x7f55a800f690 comp rx=0 tx=0).stop 2023-07-21T22:35:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 shutdown_connections 2023-07-21T22:35:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55b4106ff0 0x7f55b419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f559c0a13c0 0x7f559c0a3870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55b4101fb0 0x7f55b4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 --2- 172.21.15.66:0/825424941 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55b4102950 0x7f55b41983e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:13.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.803+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 >> 172.21.15.66:0/825424941 conn(0x7f55b40fd800 msgr2=0x7f55b40fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.804+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 shutdown_connections 2023-07-21T22:35:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:13.804+0000 7f55ba9ff700 1 -- 172.21.15.66:0/825424941 wait complete. 2023-07-21T22:35:13.853 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 49 2023-07-21T22:35:13.881 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:35:13.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 19888 31.07 MB/sec warmup 40 sec latency 12749.071 ms 2023-07-21T22:35:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 -- 172.21.15.66:0/577395894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55441059e0 msgr2=0x7f5544105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.167 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 --2- 172.21.15.66:0/577395894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55441059e0 0x7f5544105db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5540009990 tx=0x7f5540009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 -- 172.21.15.66:0/577395894 shutdown_connections 2023-07-21T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 --2- 172.21.15.66:0/577395894 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5544100380 0x7f554410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 --2- 172.21.15.66:0/577395894 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55441059e0 0x7f5544105db0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 --2- 172.21.15.66:0/577395894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55440ff9d0 0x7f55440ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.119+0000 7f554a99d700 1 -- 172.21.15.66:0/577395894 >> 172.21.15.66:0/577395894 conn(0x7f55440fb4b0 msgr2=0x7f55440fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.168 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.120+0000 7f554a99d700 1 -- 172.21.15.66:0/577395894 shutdown_connections 2023-07-21T22:35:14.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.120+0000 7f554a99d700 1 -- 172.21.15.66:0/577395894 wait complete. 2023-07-21T22:35:14.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.121+0000 7f554a99d700 1 Processor -- start 2023-07-21T22:35:14.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.122+0000 7f554a99d700 1 -- start start 2023-07-21T22:35:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.122+0000 7f554a99d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440ff9d0 0x7f5544106820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.122+0000 7f554a99d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 0x7f5544106d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 0x7f5544106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 0x7f5544106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57624/0 (socket says 172.21.15.66:57624) 2023-07-21T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554a99d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55441059e0 0x7f55441072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554a99d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55441a8230 con 0x7f55440ff9d0 2023-07-21T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554a99d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55441a83a0 con 0x7f55441059e0 2023-07-21T22:35:14.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554a99d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55441a8560 con 0x7f5544100380 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 -- 172.21.15.66:0/18787498 learned_addr learned my addr 172.21.15.66:0/18787498 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554a19c700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55441059e0 0x7f55441072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554999b700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440ff9d0 0x7f5544106820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 -- 172.21.15.66:0/18787498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55441059e0 msgr2=0x7f55441072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55441059e0 0x7f55441072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 -- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440ff9d0 msgr2=0x7f5544106820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440ff9d0 0x7f5544106820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.123+0000 7f554919a700 1 -- 172.21.15.66:0/18787498 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400095f0 con 0x7f5544100380 2023-07-21T22:35:14.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.124+0000 7f554919a700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 0x7f5544106d60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f553400e9a0 tx=0x7f553400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.124+0000 7f553affd700 1 -- 172.21.15.66:0/18787498 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534009790 con 0x7f5544100380 2023-07-21T22:35:14.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.124+0000 7f553affd700 1 -- 172.21.15.66:0/18787498 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553400cbd0 con 0x7f5544100380 2023-07-21T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.124+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55441a8840 con 0x7f5544100380 2023-07-21T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.124+0000 7f553affd700 1 -- 172.21.15.66:0/18787498 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534018680 con 0x7f5544100380 2023-07-21T22:35:14.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.124+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55441a8da0 con 0x7f5544100380 2023-07-21T22:35:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.128+0000 7f553affd700 1 -- 172.21.15.66:0/18787498 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f553400cd40 con 0x7f5544100380 2023-07-21T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.129+0000 7f553affd700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f55300a13e0 0x7f55300a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f553affd700 1 -- 172.21.15.66:0/18787498 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5534014070 con 0x7f5544100380 2023-07-21T22:35:14.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 Processor -- start 2023-07-21T22:35:14.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f554999b700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f55300a13e0 0x7f55300a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 -- start start 2023-07-21T22:35:14.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df418860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df418da0 0x5607df423700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 0x5607df426020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df31ecb0 con 0x5607df418490 2023-07-21T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.130+0000 7f555f46eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df371a40 con 0x5607df423c40 2023-07-21T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f555f46eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df36aed0 con 0x5607df418da0 2023-07-21T22:35:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 0x5607df426020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 0x5607df426020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38474/0 (socket says 172.21.15.66:38474) 2023-07-21T22:35:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 -- 172.21.15.66:0/3747841808 learned_addr learned my addr 172.21.15.66:0/3747841808 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554999b700 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df418860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554919a700 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df418da0 0x5607df423700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 -- 172.21.15.66:0/3747841808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df418da0 msgr2=0x5607df423700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df418da0 0x5607df423700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 -- 172.21.15.66:0/3747841808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 msgr2=0x5607df418860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df418860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554a19c700 1 -- 172.21.15.66:0/3747841808 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607df198de0 con 0x5607df423c40 2023-07-21T22:35:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554919a700 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df418da0 0x5607df423700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.131+0000 7f554999b700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f55300a13e0 0x7f55300a3890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f554000b2e0 tx=0x7f5540005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.132+0000 7f554a19c700 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 0x5607df426020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f553c011680 tx=0x7f553c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65e31555a7909bf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.132+0000 7f552ffff700 1 -- 172.21.15.66:0/3747841808 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00e040 con 0x5607df423c40 2023-07-21T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.132+0000 7f552ffff700 1 -- 172.21.15.66:0/3747841808 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553c00f040 con 0x5607df423c40 2023-07-21T22:35:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.132+0000 7f552ffff700 1 -- 172.21.15.66:0/3747841808 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c0048c0 con 0x5607df423c40 2023-07-21T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.133+0000 7f555f46eb80 1 -- 172.21.15.66:0/3747841808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 msgr2=0x5607df426020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.133+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 0x5607df426020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f553c011680 tx=0x7f553c009920 comp rx=0 tx=0).stop 2023-07-21T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.134+0000 7f555f46eb80 1 -- 172.21.15.66:0/3747841808 shutdown_connections 2023-07-21T22:35:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.134+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df423c40 0x5607df426020 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.134+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df418860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.134+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3747841808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df418da0 0x5607df423700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.134+0000 7f555f46eb80 1 -- 172.21.15.66:0/3747841808 >> 172.21.15.66:0/3747841808 conn(0x5607df40eef0 msgr2=0x5607df414860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.135+0000 7f555f46eb80 1 -- 172.21.15.66:0/3747841808 shutdown_connections 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.135+0000 7f555f46eb80 1 -- 172.21.15.66:0/3747841808 wait complete. 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.136+0000 7f555f46eb80 1 -- start start 2023-07-21T22:35:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.136+0000 7f555f46eb80 1 Processor -- start 2023-07-21T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df420930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df421020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df30aac0 con 0x5607df418da0 2023-07-21T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df30a7c0 con 0x5607df418490 2023-07-21T22:35:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df1ee920 con 0x5607df423c40 2023-07-21T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.138+0000 7f555f46eb80 10 client.0 _ll_register_callbacks cb 0x5607df308760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df421020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df421020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57648/0 (socket says 172.21.15.66:57648) 2023-07-21T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 -- 172.21.15.66:0/3871036841 learned_addr learned my addr 172.21.15.66:0/3871036841 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554999b700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 -- 172.21.15.66:0/3871036841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 msgr2=0x5607df4236b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4236b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.188 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 -- 172.21.15.66:0/3871036841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 msgr2=0x5607df420930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554919a700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df420930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df420930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554a19c700 1 -- 172.21.15.66:0/3871036841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553c011330 con 0x5607df423c40 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.139+0000 7f554919a700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df420930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.140+0000 7f554a19c700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df421020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f553c009b20 tx=0x7f553c005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff227d8666726c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.140+0000 7f552d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:14.189 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.140+0000 7f552d7fa700 1 -- 172.21.15.66:0/3871036841 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00e070 con 0x5607df423c40 2023-07-21T22:35:14.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.140+0000 7f552d7fa700 1 -- 172.21.15.66:0/3871036841 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f553c003c10 con 0x5607df423c40 2023-07-21T22:35:14.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.140+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607df2c44f0 con 0x5607df423c40 2023-07-21T22:35:14.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.141+0000 7f552d7fa700 1 -- 172.21.15.66:0/3871036841 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553c003dd0 con 0x5607df423c40 2023-07-21T22:35:14.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.141+0000 7f555f46eb80 10 client.27457 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:14.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.141+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5607df181c60 con 0x5607df423c40 2023-07-21T22:35:14.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.141+0000 7f552d7fa700 1 -- 172.21.15.66:0/3871036841 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607df2c44f0 con 0x5607df423c40 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f552d7fa700 1 -- 172.21.15.66:0/3871036841 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f553c027460 con 0x5607df423c40 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f555f46eb80 10 client.27457 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f555f46eb80 10 client.27457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f555f46eb80 20 client.27457 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df444560 0x5607df446970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f555f46eb80 4 client.27457 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:14.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.142+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607df363eb0 con 0x5607df444560 2023-07-21T22:35:14.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.143+0000 7f554999b700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df444560 0x5607df446970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.143+0000 7f552d7fa700 10 client.27457 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:14.193 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.143+0000 7f554999b700 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df444560 0x5607df446970 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6872ed6980b75061 server_cookie=cd341eac7cc68657 in_seq=0 out_seq=0 2023-07-21T22:35:14.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.144+0000 7f552d7fa700 1 -- 172.21.15.66:0/3871036841 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5607df363eb0 con 0x5607df444560 2023-07-21T22:35:14.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.144+0000 7f552d7fa700 10 client.27457 handle_command_reply: tid=0 2023-07-21T22:35:14.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.144+0000 7f555f46eb80 1 client.27457 shutdown 2023-07-21T22:35:14.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 msgr2=0x5607df421020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df421020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f553c009b20 tx=0x7f553c005ae0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 shutdown_connections 2023-07-21T22:35:14.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df444560 0x5607df446970 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df421020 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3871036841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df420930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 >> 172.21.15.66:0/3871036841 conn(0x5607df40eef0 msgr2=0x5607df4143f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f552d7fa700 0 client.27457 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 shutdown_connections 2023-07-21T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 1 -- 172.21.15.66:0/3871036841 wait complete. 2023-07-21T22:35:14.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.145+0000 7f555f46eb80 20 client.27457 trim_cache size 0 max 16384 2023-07-21T22:35:14.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:14 smithi066 ceph-mon[126379]: pgmap v430: 129 pgs: 129 active+clean; 994 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T22:35:14.197 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:14 smithi066 ceph-mon[126379]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T22:35:14.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/825424941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:35:14.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.163+0000 7f555f46eb80 1 Processor -- start 2023-07-21T22:35:14.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.163+0000 7f555f46eb80 1 -- start start 2023-07-21T22:35:14.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df41bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f555f46eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df41b350 con 0x5607df418da0 2023-07-21T22:35:14.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f555f46eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df441c60 con 0x5607df418490 2023-07-21T22:35:14.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f555f46eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df419680 con 0x5607df423c40 2023-07-21T22:35:14.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554999b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554999b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38500/0 (socket says 172.21.15.66:38500) 2023-07-21T22:35:14.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554999b700 1 -- 172.21.15.66:0/297574191 learned_addr learned my addr 172.21.15.66:0/297574191 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554a19c700 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554919a700 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df41bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554999b700 1 -- 172.21.15.66:0/297574191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 msgr2=0x5607df4231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.164+0000 7f554999b700 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f554999b700 1 -- 172.21.15.66:0/297574191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 msgr2=0x5607df41bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f554999b700 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df41bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f554999b700 1 -- 172.21.15.66:0/297574191 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607df1bf250 con 0x5607df418490 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f554a19c700 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f554999b700 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4408f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f554000c010 tx=0x7f554001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5e303863e78db8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f5538bff700 1 -- 172.21.15.66:0/297574191 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554002b070 con 0x5607df418490 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f5538bff700 1 -- 172.21.15.66:0/297574191 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554002c040 con 0x5607df418490 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.165+0000 7f5538bff700 1 -- 172.21.15.66:0/297574191 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554002f780 con 0x5607df418490 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 -- 172.21.15.66:0/297574191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 msgr2=0x5607df4408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4408f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f554000c010 tx=0x7f554001f000 comp rx=0 tx=0).stop 2023-07-21T22:35:14.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 -- 172.21.15.66:0/297574191 shutdown_connections 2023-07-21T22:35:14.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418490 0x5607df4408f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 --2- 172.21.15.66:0/297574191 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418da0 0x5607df41bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.216 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.166+0000 7f555f46eb80 1 -- 172.21.15.66:0/297574191 >> 172.21.15.66:0/297574191 conn(0x5607df40eef0 msgr2=0x5607df412f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.167+0000 7f555f46eb80 1 -- 172.21.15.66:0/297574191 shutdown_connections 2023-07-21T22:35:14.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.167+0000 7f555f46eb80 1 -- 172.21.15.66:0/297574191 wait complete. 2023-07-21T22:35:14.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.168+0000 7f555f46eb80 1 -- start start 2023-07-21T22:35:14.217 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.168+0000 7f555f46eb80 1 Processor -- start 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df43b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418da0 0x5607df43b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df439bc0 con 0x5607df418490 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df439d30 con 0x5607df418da0 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607df439ef0 con 0x5607df423c40 2023-07-21T22:35:14.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.169+0000 7f555f46eb80 10 client.0 _ll_register_callbacks cb 0x7f553c011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:14.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4395f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57654/0 (socket says 172.21.15.66:57654) 2023-07-21T22:35:14.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 -- 172.21.15.66:0/3579927100 learned_addr learned my addr 172.21.15.66:0/3579927100 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554999b700 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df43b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 -- 172.21.15.66:0/3579927100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418da0 msgr2=0x5607df43b670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:35:14.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418da0 0x5607df43b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 -- 172.21.15.66:0/3579927100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 msgr2=0x5607df43b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df43b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.170+0000 7f554a19c700 1 -- 172.21.15.66:0/3579927100 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607df181c60 con 0x5607df423c40 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f554a19c700 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4395f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f553c0060b0 tx=0x7f553c009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccaf5dcb67b3a1c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f552d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f552d7fa700 1 -- 172.21.15.66:0/3579927100 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c00e050 con 0x5607df423c40 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f553c00f040 con 0x5607df423c40 2023-07-21T22:35:14.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f552d7fa700 1 -- 172.21.15.66:0/3579927100 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f553c027ae0 con 0x5607df423c40 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f552d7fa700 1 -- 172.21.15.66:0/3579927100 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553c003c10 con 0x5607df423c40 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f555f46eb80 10 client.27463 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.171+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f554410d110 con 0x5607df423c40 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f552d7fa700 1 -- 172.21.15.66:0/3579927100 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553c00f040 con 0x5607df423c40 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f552d7fa700 1 -- 172.21.15.66:0/3579927100 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f553c03c070 con 0x5607df423c40 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f555f46eb80 10 client.27463 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f555f46eb80 10 client.27463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:35:14.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f555f46eb80 20 client.27463 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:14.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df4993a0 0x5607df49b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f555f46eb80 4 client.27463 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:35:14.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607df326050 con 0x5607df4993a0 2023-07-21T22:35:14.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.172+0000 7f554999b700 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df4993a0 0x5607df49b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f554999b700 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df4993a0 0x5607df49b7b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23bd6927c19043e0 server_cookie=51bd531702402417 in_seq=0 out_seq=0 2023-07-21T22:35:14.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f552d7fa700 10 client.27463 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:14.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f552d7fa700 1 -- 172.21.15.66:0/3579927100 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607df326050 con 0x5607df4993a0 2023-07-21T22:35:14.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f552d7fa700 10 client.27463 handle_command_reply: tid=0 2023-07-21T22:35:14.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 client.27463 shutdown 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 msgr2=0x5607df4395f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4395f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f553c0060b0 tx=0x7f553c009920 comp rx=0 tx=0).stop 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 shutdown_connections 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x5607df4993a0 0x5607df49b7b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5607df423c40 0x5607df4395f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5607df418490 0x5607df43b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 --2- 172.21.15.66:0/3579927100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607df418da0 0x5607df43b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f552d7fa700 0 client.27463 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:14.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.173+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 >> 172.21.15.66:0/3579927100 conn(0x5607df40eef0 msgr2=0x5607df41bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 shutdown_connections 2023-07-21T22:35:14.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f555f46eb80 1 -- 172.21.15.66:0/3579927100 wait complete. 2023-07-21T22:35:14.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f555f46eb80 20 client.27463 trim_cache size 0 max 16384 2023-07-21T22:35:14.226 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:35:14.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f55300a13e0 msgr2=0x7f55300a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f554a99d700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f55300a13e0 0x7f55300a3890 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f554000b2e0 tx=0x7f5540005fb0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 msgr2=0x7f5544106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.174+0000 7f554a99d700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 0x7f5544106d60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f553400e9a0 tx=0x7f553400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 shutdown_connections 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55441059e0 0x7f55441072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f55300a13e0 0x7f55300a3890 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f55440ff9d0 0x7f5544106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 --2- 172.21.15.66:0/18787498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5544100380 0x7f5544106d60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 >> 172.21.15.66:0/18787498 conn(0x7f55440fb4b0 msgr2=0x7f554410ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 shutdown_connections 2023-07-21T22:35:14.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.175+0000 7f554a99d700 1 -- 172.21.15.66:0/18787498 wait complete. 2023-07-21T22:35:14.243 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:35:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:14 smithi104 ceph-mon[129021]: pgmap v430: 129 pgs: 129 active+clean; 994 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T22:35:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:14 smithi104 ceph-mon[129021]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T22:35:14.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/825424941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:35:14.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:14 smithi149 ceph-mon[129268]: pgmap v430: 129 pgs: 129 active+clean; 994 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.7 MiB/s wr, 687 op/s 2023-07-21T22:35:14.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:14 smithi149 ceph-mon[129268]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T22:35:14.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/825424941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:35:14.614 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 -- 172.21.15.66:0/2144596809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0101f70 msgr2=0x7f10c0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.615 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 --2- 172.21.15.66:0/2144596809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0101f70 0x7f10c0102340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f10b00099c0 tx=0x7f10b0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 -- 172.21.15.66:0/2144596809 shutdown_connections 2023-07-21T22:35:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 --2- 172.21.15.66:0/2144596809 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c0106fb0 0x7f10c010eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 --2- 172.21.15.66:0/2144596809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0101f70 0x7f10c0102340 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.616 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 --2- 172.21.15.66:0/2144596809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0102910 0x7f10c01068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.567+0000 7f10c57ff700 1 -- 172.21.15.66:0/2144596809 >> 172.21.15.66:0/2144596809 conn(0x7f10c00fd7e0 msgr2=0x7f10c00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.568+0000 7f10c57ff700 1 -- 172.21.15.66:0/2144596809 shutdown_connections 2023-07-21T22:35:14.617 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.568+0000 7f10c57ff700 1 -- 172.21.15.66:0/2144596809 wait complete. 2023-07-21T22:35:14.618 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.570+0000 7f10c57ff700 1 Processor -- start 2023-07-21T22:35:14.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 -- start start 2023-07-21T22:35:14.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c0101f70 0x7f10c010e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.619 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 0x7f10c010ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0106fb0 0x7f10c010b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c010bf30 con 0x7f10c0101f70 2023-07-21T22:35:14.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c010c0a0 con 0x7f10c0106fb0 2023-07-21T22:35:14.620 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.571+0000 7f10c57ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c010c260 con 0x7f10c0102910 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 0x7f10c010ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 0x7f10c010ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57678/0 (socket says 172.21.15.66:57678) 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 -- 172.21.15.66:0/1041324048 learned_addr learned my addr 172.21.15.66:0/1041324048 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10beffd700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c0101f70 0x7f10c010e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0106fb0 0x7f10c010b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 -- 172.21.15.66:0/1041324048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0106fb0 msgr2=0x7f10c010b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.621 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0106fb0 0x7f10c010b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 -- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c0101f70 msgr2=0x7f10c010e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c0101f70 0x7f10c010e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 -- 172.21.15.66:0/1041324048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00095f0 con 0x7f10c0102910 2023-07-21T22:35:14.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0106fb0 0x7f10c010b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:14.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.572+0000 7f10be7fc700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 0x7f10c010ead0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10b400e940 tx=0x7f10b400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.622 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.573+0000 7f10a7fff700 1 -- 172.21.15.66:0/1041324048 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b400cbd0 con 0x7f10c0102910 2023-07-21T22:35:14.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.573+0000 7f10a7fff700 1 -- 172.21.15.66:0/1041324048 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b400cd30 con 0x7f10c0102910 2023-07-21T22:35:14.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.573+0000 7f10a7fff700 1 -- 172.21.15.66:0/1041324048 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b4018680 con 0x7f10c0102910 2023-07-21T22:35:14.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.573+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c01a8c10 con 0x7f10c0102910 2023-07-21T22:35:14.623 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.573+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c01a8fe0 con 0x7f10c0102910 2023-07-21T22:35:14.626 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.579+0000 7f10a7fff700 1 -- 172.21.15.66:0/1041324048 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f10b40187e0 con 0x7f10c0102910 2023-07-21T22:35:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.580+0000 7f10d905fb80 1 Processor -- start 2023-07-21T22:35:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 -- start start 2023-07-21T22:35:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a708580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a703350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a703da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a608cb0 con 0x559e4a7081b0 2023-07-21T22:35:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a65ba40 con 0x559e4a708ac0 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10d905fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a654ed0 con 0x559e4a703930 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10beffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a703350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10beffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a703350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38522/0 (socket says 172.21.15.66:38522) 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.581+0000 7f10beffd700 1 -- 172.21.15.66:0/2374999280 learned_addr learned my addr 172.21.15.66:0/2374999280 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10beffd700 1 -- 172.21.15.66:0/2374999280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 msgr2=0x559e4a703da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a708580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10beffd700 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a703da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10beffd700 1 -- 172.21.15.66:0/2374999280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 msgr2=0x559e4a708580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10beffd700 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a708580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10beffd700 1 -- 172.21.15.66:0/2374999280 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e4a482de0 con 0x559e4a708ac0 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a708580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.582+0000 7f10beffd700 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a703350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f10b00036c0 tx=0x7f10b0005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282779f84ed3d62c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.583+0000 7f10a57fa700 1 -- 172.21.15.66:0/2374999280 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b001d040 con 0x559e4a708ac0 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.583+0000 7f10a57fa700 1 -- 172.21.15.66:0/2374999280 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b0005070 con 0x559e4a708ac0 2023-07-21T22:35:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.583+0000 7f10a57fa700 1 -- 172.21.15.66:0/2374999280 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0003b10 con 0x559e4a708ac0 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.583+0000 7f10a7fff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f10a80a1450 0x7f10a80a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.583+0000 7f10a7fff700 1 -- 172.21.15.66:0/1041324048 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f10b4014070 con 0x7f10c0102910 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 -- 172.21.15.66:0/2374999280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 msgr2=0x559e4a703350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a703350 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f10b00036c0 tx=0x7f10b0005a90 comp rx=0 tx=0).stop 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 -- 172.21.15.66:0/2374999280 shutdown_connections 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a703da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a708580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 --2- 172.21.15.66:0/2374999280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a703350 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 -- 172.21.15.66:0/2374999280 >> 172.21.15.66:0/2374999280 conn(0x559e4a6f8ef0 msgr2=0x559e4a6fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 -- 172.21.15.66:0/2374999280 shutdown_connections 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.584+0000 7f10d905fb80 1 -- 172.21.15.66:0/2374999280 wait complete. 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 -- start start 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f10a80a1450 0x7f10a80a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 Processor -- start 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a70fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a70ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a70d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a5f4ac0 con 0x559e4a7081b0 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a5f47c0 con 0x559e4a708ac0 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.585+0000 7f10d905fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a4d8920 con 0x559e4a703930 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10d905fb80 10 client.0 _ll_register_callbacks cb 0x559e4a5f2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a70fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a70fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57702/0 (socket says 172.21.15.66:57702) 2023-07-21T22:35:14.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 -- 172.21.15.66:0/3578054135 learned_addr learned my addr 172.21.15.66:0/3578054135 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a70d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 -- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 msgr2=0x559e4a70d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a70d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 -- 172.21.15.66:0/3578054135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 msgr2=0x559e4a70ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a70ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10beffd700 1 -- 172.21.15.66:0/3578054135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c010f3c0 con 0x559e4a703930 2023-07-21T22:35:14.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.586+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a70d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.587+0000 7f10beffd700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a70fb30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f10b000a010 tx=0x7f10b0009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd561dab0c16c99a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.587+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f10a80a1450 0x7f10a80a3900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009710 tx=0x7f10b800b040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.587+0000 7f10a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.587+0000 7f10a2ff5700 1 -- 172.21.15.66:0/3578054135 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b001d040 con 0x559e4a703930 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.587+0000 7f10a2ff5700 1 -- 172.21.15.66:0/3578054135 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f10b000bd20 con 0x559e4a703930 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.587+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e4a5ae4f0 con 0x559e4a703930 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.588+0000 7f10a2ff5700 1 -- 172.21.15.66:0/3578054135 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b000bee0 con 0x559e4a703930 2023-07-21T22:35:14.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.588+0000 7f10d905fb80 10 client.27472 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.588+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559e4a46bc60 con 0x559e4a703930 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.588+0000 7f10a2ff5700 1 -- 172.21.15.66:0/3578054135 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e4a5ae4f0 con 0x559e4a703930 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10a2ff5700 1 -- 172.21.15.66:0/3578054135 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f10b0022a80 con 0x559e4a703930 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10d905fb80 10 client.27472 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10d905fb80 10 client.27472 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10d905fb80 20 client.27472 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:14.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a72dfb0 0x559e4a7303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10d905fb80 4 client.27472 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e4a64deb0 con 0x559e4a72dfb0 2023-07-21T22:35:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.589+0000 7f10be7fc700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a72dfb0 0x559e4a7303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.590+0000 7f10be7fc700 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a72dfb0 0x559e4a7303c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63e557073bd56bbc server_cookie=fe7374fba2409bfc in_seq=0 out_seq=0 2023-07-21T22:35:14.644 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.590+0000 7f10a2ff5700 10 client.27472 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10a2ff5700 1 -- 172.21.15.66:0/3578054135 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559e4a64deb0 con 0x559e4a72dfb0 2023-07-21T22:35:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10a2ff5700 10 client.27472 handle_command_reply: tid=0 2023-07-21T22:35:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 client.27472 shutdown 2023-07-21T22:35:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 msgr2=0x559e4a70fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.645 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a70fb30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f10b000a010 tx=0x7f10b0009cd0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 shutdown_connections 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a72dfb0 0x559e4a7303c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a70d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a70fb30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10a2ff5700 0 client.27472 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3578054135 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a70ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.646 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.595+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 >> 172.21.15.66:0/3578054135 conn(0x559e4a6f8ef0 msgr2=0x559e4a6fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.596+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 shutdown_connections 2023-07-21T22:35:14.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.596+0000 7f10d905fb80 1 -- 172.21.15.66:0/3578054135 wait complete. 2023-07-21T22:35:14.647 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.596+0000 7f10d905fb80 20 client.27472 trim_cache size 0 max 16384 2023-07-21T22:35:14.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.616+0000 7f10d905fb80 1 Processor -- start 2023-07-21T22:35:14.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.616+0000 7f10d905fb80 1 -- start start 2023-07-21T22:35:14.664 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a7255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a70f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a701a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10d905fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a70e3f0 con 0x559e4a7081b0 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10d905fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a70aea0 con 0x559e4a708ac0 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10d905fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a725b00 con 0x559e4a703930 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10beffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a7255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.665 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10beffd700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a7255c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57716/0 (socket says 172.21.15.66:57716) 2023-07-21T22:35:14.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10beffd700 1 -- 172.21.15.66:0/3093841155 learned_addr learned my addr 172.21.15.66:0/3093841155 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a701a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.666 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.617+0000 7f10beffd700 1 -- 172.21.15.66:0/3093841155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 msgr2=0x559e4a701a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10beffd700 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a701a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10beffd700 1 -- 172.21.15.66:0/3093841155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 msgr2=0x559e4a70f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10beffd700 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a70f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.667 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10beffd700 1 -- 172.21.15.66:0/3093841155 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e4a4a9250 con 0x559e4a703930 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10beffd700 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a7255c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f10b00098e0 tx=0x7f10b0005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a112ea4f3dcc0c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10a5ffb700 1 -- 172.21.15.66:0/3093841155 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b002a590 con 0x559e4a703930 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10a5ffb700 1 -- 172.21.15.66:0/3093841155 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b0005570 con 0x559e4a703930 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.618+0000 7f10a5ffb700 1 -- 172.21.15.66:0/3093841155 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b002aa80 con 0x559e4a703930 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 -- 172.21.15.66:0/3093841155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 msgr2=0x559e4a7255c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a7255c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f10b00098e0 tx=0x7f10b0005a90 comp rx=0 tx=0).stop 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 -- 172.21.15.66:0/3093841155 shutdown_connections 2023-07-21T22:35:14.668 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a708ac0 0x559e4a701a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a703930 0x559e4a7255c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 --2- 172.21.15.66:0/3093841155 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a7081b0 0x559e4a70f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 -- 172.21.15.66:0/3093841155 >> 172.21.15.66:0/3093841155 conn(0x559e4a6f8ef0 msgr2=0x559e4a72a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.619+0000 7f10d905fb80 1 -- 172.21.15.66:0/3093841155 shutdown_connections 2023-07-21T22:35:14.669 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.620+0000 7f10d905fb80 1 -- 172.21.15.66:0/3093841155 wait complete. 2023-07-21T22:35:14.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.620+0000 7f10d905fb80 1 -- start start 2023-07-21T22:35:14.670 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.622+0000 7f10d905fb80 1 Processor -- start 2023-07-21T22:35:14.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 0x559e4a775ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a7081b0 0x559e4a776530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.671 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a708ac0 0x559e4a780e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a776cc0 con 0x559e4a708ac0 2023-07-21T22:35:14.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a7813f0 con 0x559e4a703930 2023-07-21T22:35:14.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e4a7815b0 con 0x559e4a7081b0 2023-07-21T22:35:14.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.623+0000 7f10d905fb80 10 client.0 _ll_register_callbacks cb 0x7f10b4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:14.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 0x559e4a775ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.672 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 0x559e4a775ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38542/0 (socket says 172.21.15.66:38542) 2023-07-21T22:35:14.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 -- 172.21.15.66:0/880358578 learned_addr learned my addr 172.21.15.66:0/880358578 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:14.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10be7fc700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a7081b0 0x559e4a776530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.673 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a708ac0 0x559e4a780e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 -- 172.21.15.66:0/880358578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a7081b0 msgr2=0x559e4a776530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a7081b0 0x559e4a776530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 -- 172.21.15.66:0/880358578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a708ac0 msgr2=0x559e4a780e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:14.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a708ac0 0x559e4a780e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.674 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10beffd700 1 -- 172.21.15.66:0/880358578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e4a46bc60 con 0x559e4a703930 2023-07-21T22:35:14.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.624+0000 7f10bf7fe700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a708ac0 0x559e4a780e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:14.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.626+0000 7f10beffd700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 0x559e4a775ff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10b001a040 tx=0x7f10b000b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1892118dd96d40f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:14.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.627+0000 7f10a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:35:14.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.627+0000 7f10a2ff5700 1 -- 172.21.15.66:0/880358578 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b001d070 con 0x559e4a703930 2023-07-21T22:35:14.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.627+0000 7f10a2ff5700 1 -- 172.21.15.66:0/880358578 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f10b0000e40 con 0x559e4a703930 2023-07-21T22:35:14.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.627+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f10b0005070 con 0x559e4a703930 2023-07-21T22:35:14.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.628+0000 7f10a2ff5700 1 -- 172.21.15.66:0/880358578 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10b000bb20 con 0x559e4a703930 2023-07-21T22:35:14.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10d905fb80 10 client.27506 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:14.676 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x559e4a4812b0 con 0x559e4a703930 2023-07-21T22:35:14.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10a2ff5700 1 -- 172.21.15.66:0/880358578 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b0005070 con 0x559e4a703930 2023-07-21T22:35:14.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10a2ff5700 1 -- 172.21.15.66:0/880358578 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f10b0035070 con 0x559e4a703930 2023-07-21T22:35:14.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10d905fb80 10 client.27506 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:14.677 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10d905fb80 10 client.27506 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:35:14.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10d905fb80 20 client.27506 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:14.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.629+0000 7f10d905fb80 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a782920 0x559e4a784d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:14.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.630+0000 7f10d905fb80 4 client.27506 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:35:14.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.630+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559e4a610050 con 0x559e4a782920 2023-07-21T22:35:14.678 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.630+0000 7f10be7fc700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a782920 0x559e4a784d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:14.679 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.632+0000 7f10be7fc700 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a782920 0x559e4a784d30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=41629af2dbcc2b43 server_cookie=d21f2864aeb08292 in_seq=0 out_seq=0 2023-07-21T22:35:14.680 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.632+0000 7f10a2ff5700 10 client.27506 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.638+0000 7f10a2ff5700 1 -- 172.21.15.66:0/880358578 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559e4a610050 con 0x559e4a782920 2023-07-21T22:35:14.686 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.639+0000 7f10a2ff5700 10 client.27506 handle_command_reply: tid=0 2023-07-21T22:35:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.639+0000 7f10d905fb80 1 client.27506 shutdown 2023-07-21T22:35:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 msgr2=0x559e4a775ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.687 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 0x559e4a775ff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10b001a040 tx=0x7f10b000b540 comp rx=0 tx=0).stop 2023-07-21T22:35:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 shutdown_connections 2023-07-21T22:35:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559e4a782920 0x559e4a784d30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559e4a703930 0x559e4a775ff0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559e4a708ac0 0x559e4a780e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.688 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 --2- 172.21.15.66:0/880358578 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559e4a7081b0 0x559e4a776530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.640+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 >> 172.21.15.66:0/880358578 conn(0x559e4a6f8ef0 msgr2=0x559e4a701a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.641+0000 7f10a2ff5700 0 client.27506 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.641+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 shutdown_connections 2023-07-21T22:35:14.689 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.641+0000 7f10d905fb80 1 -- 172.21.15.66:0/880358578 wait complete. 2023-07-21T22:35:14.690 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.641+0000 7f10d905fb80 20 client.27506 trim_cache size 0 max 16384 2023-07-21T22:35:14.690 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:35:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.644+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f10a80a1450 msgr2=0x7f10a80a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.644+0000 7f10c57ff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f10a80a1450 0x7f10a80a3900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009710 tx=0x7f10b800b040 comp rx=0 tx=0).stop 2023-07-21T22:35:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.644+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 msgr2=0x7f10c010ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:14.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.644+0000 7f10c57ff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 0x7f10c010ead0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10b400e940 tx=0x7f10b400ed00 comp rx=0 tx=0).stop 2023-07-21T22:35:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.645+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 shutdown_connections 2023-07-21T22:35:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.645+0000 7f10c57ff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10c0106fb0 0x7f10c010b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.645+0000 7f10c57ff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f10a80a1450 0x7f10a80a3900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.645+0000 7f10c57ff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10c0101f70 0x7f10c010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.645+0000 7f10c57ff700 1 --2- 172.21.15.66:0/1041324048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10c0102910 0x7f10c010ead0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:14.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.645+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 >> 172.21.15.66:0/1041324048 conn(0x7f10c00fd7e0 msgr2=0x7f10c00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:14.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.646+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 shutdown_connections 2023-07-21T22:35:14.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.646+0000 7f10c57ff700 1 -- 172.21.15.66:0/1041324048 wait complete. 2023-07-21T22:35:14.719 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:35:14.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 20671 31.13 MB/sec warmup 41 sec latency 13749.127 ms 2023-07-21T22:35:15.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.963+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/3075093638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480fe8b0 msgr2=0x7f4e480fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.963+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/3075093638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480fe8b0 0x7f4e480fec80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4e38009990 tx=0x7f4e38009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.965+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/3075093638 shutdown_connections 2023-07-21T22:35:15.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.965+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/3075093638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480fe8b0 0x7f4e480fec80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.965+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/3075093638 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e4810c460 0x7f4e4810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.965+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/3075093638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.013 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.965+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/3075093638 >> 172.21.15.66:0/3075093638 conn(0x7f4e480fa4e0 msgr2=0x7f4e480fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:15.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.966+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/3075093638 shutdown_connections 2023-07-21T22:35:15.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.966+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/3075093638 wait complete. 2023-07-21T22:35:15.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 Processor -- start 2023-07-21T22:35:15.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 -- start start 2023-07-21T22:35:15.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e480fe8b0 0x7f4e4810b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4810c460 0x7f4e481082f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48108ac0 con 0x7f4e480fe8b0 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48108c30 con 0x7f4e480ff1c0 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.967+0000 7f4e4d7ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e48108df0 con 0x7f4e4810c460 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38560/0 (socket says 172.21.15.66:38560) 2023-07-21T22:35:15.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e467fc700 1 -- 172.21.15.66:0/1737804611 learned_addr learned my addr 172.21.15.66:0/1737804611 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:15.017 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e480fe8b0 0x7f4e4810b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1737804611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4810c460 msgr2=0x7f4e481082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4810c460 0x7f4e481082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1737804611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 msgr2=0x7f4e4810bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.968+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1737804611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e380095f0 con 0x7f4e480fe8b0 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:15.018 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e480fe8b0 0x7f4e4810b780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098e0 tx=0x7f4e38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:15.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e2ffff700 1 -- 172.21.15.66:0/1737804611 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3801d070 con 0x7f4e480fe8b0 2023-07-21T22:35:15.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e48109070 con 0x7f4e480fe8b0 2023-07-21T22:35:15.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e2ffff700 1 -- 172.21.15.66:0/1737804611 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e380054f0 con 0x7f4e480fe8b0 2023-07-21T22:35:15.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e2ffff700 1 -- 172.21.15.66:0/1737804611 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38004b90 con 0x7f4e480fe8b0 2023-07-21T22:35:15.019 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.969+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e48107400 con 0x7f4e480fe8b0 2023-07-21T22:35:15.022 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.975+0000 7f4e2ffff700 1 -- 172.21.15.66:0/1737804611 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4e3800fb90 con 0x7f4e480fe8b0 2023-07-21T22:35:15.023 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.976+0000 7f4e610aab80 1 Processor -- start 2023-07-21T22:35:15.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.976+0000 7f4e610aab80 1 -- start start 2023-07-21T22:35:15.024 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.977+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.977+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.977+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c56da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.977+0000 7f4e610aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4b5bcb0 con 0x5587b4c5b1b0 2023-07-21T22:35:15.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.977+0000 7f4e610aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4baea40 con 0x5587b4c56930 2023-07-21T22:35:15.025 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.977+0000 7f4e610aab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4ba7ed0 con 0x5587b4c5bac0 2023-07-21T22:35:15.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.978+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.978+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c56da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38570/0 (socket says 172.21.15.66:38570) 2023-07-21T22:35:15.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.978+0000 7f4e467fc700 1 -- 172.21.15.66:0/1885281900 learned_addr learned my addr 172.21.15.66:0/1885281900 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:15.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.978+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.026 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.978+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1885281900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 msgr2=0x5587b4c56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c56da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1885281900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 msgr2=0x5587b4c5b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.027 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1885281900 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b49d5de0 con 0x5587b4c5bac0 2023-07-21T22:35:15.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e2ffff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e300a1340 0x7f4e300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:15.028 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.979+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.980+0000 7f4e2ffff700 1 -- 172.21.15.66:0/1737804611 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4e380ce1e0 con 0x7f4e480fe8b0 2023-07-21T22:35:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.980+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c56350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e380cb590 tx=0x7f4e380d6550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c984d0ff37b177 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.980+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e300a1340 0x7f4e300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.980+0000 7f4e2d7fa700 1 -- 172.21.15.66:0/1885281900 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380972e0 con 0x5587b4c5bac0 2023-07-21T22:35:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.980+0000 7f4e2d7fa700 1 -- 172.21.15.66:0/1885281900 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3809ad70 con 0x5587b4c5bac0 2023-07-21T22:35:15.029 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.980+0000 7f4e2d7fa700 1 -- 172.21.15.66:0/1885281900 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e38005660 con 0x5587b4c5bac0 2023-07-21T22:35:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.981+0000 7f4e610aab80 1 -- 172.21.15.66:0/1885281900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 msgr2=0x5587b4c56350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.981+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c56350 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4e380cb590 tx=0x7f4e380d6550 comp rx=0 tx=0).stop 2023-07-21T22:35:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e300a1340 0x7f4e300a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e400060b0 tx=0x7f4e40009040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 -- 172.21.15.66:0/1885281900 shutdown_connections 2023-07-21T22:35:15.030 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1885281900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c56350 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 -- 172.21.15.66:0/1885281900 >> 172.21.15.66:0/1885281900 conn(0x5587b4c4bef0 msgr2=0x5587b4c51830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 -- 172.21.15.66:0/1885281900 shutdown_connections 2023-07-21T22:35:15.031 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.982+0000 7f4e610aab80 1 -- 172.21.15.66:0/1885281900 wait complete. 2023-07-21T22:35:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 -- start start 2023-07-21T22:35:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 Processor -- start 2023-07-21T22:35:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c62af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.032 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4b47ac0 con 0x5587b4c5b1b0 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4b477c0 con 0x5587b4c56930 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.983+0000 7f4e610aab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4a2b920 con 0x5587b4c5bac0 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.984+0000 7f4e610aab80 10 client.0 _ll_register_callbacks cb 0x5587b4b45760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.984+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.984+0000 7f4e467fc700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48344/0 (socket says 172.21.15.66:48344) 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.984+0000 7f4e467fc700 1 -- 172.21.15.66:0/1882902529 learned_addr learned my addr 172.21.15.66:0/1882902529 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:15.033 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.984+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e467fc700 1 -- 172.21.15.66:0/1882902529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 msgr2=0x5587b4c603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.034 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e467fc700 1 -- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 msgr2=0x5587b4c62af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c62af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e467fc700 1 -- 172.21.15.66:0/1882902529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e48107400 con 0x5587b4c5b1b0 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5fde0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c011d10 tx=0x7f4e3c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0ca5da5487818e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.985+0000 7f4e2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.66:3300/0 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.986+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1882902529 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c016070 con 0x5587b4c5b1b0 2023-07-21T22:35:15.035 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.986+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1882902529 <== mon.0 v2:172.21.15.66:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4e3c009ca0 con 0x5587b4c5b1b0 2023-07-21T22:35:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.986+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587b4b014f0 con 0x5587b4c5b1b0 2023-07-21T22:35:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.986+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1882902529 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e3c009e60 con 0x5587b4c5b1b0 2023-07-21T22:35:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.986+0000 7f4e610aab80 10 client.17454 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.986+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5587b49bec60 con 0x5587b4c5b1b0 2023-07-21T22:35:15.036 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.987+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1882902529 <== mon.0 v2:172.21.15.66:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5587b4b014f0 con 0x5587b4c5b1b0 2023-07-21T22:35:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1882902529 <== mon.0 v2:172.21.15.66:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4e3c019520 con 0x5587b4c5b1b0 2023-07-21T22:35:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e610aab80 10 client.17454 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e610aab80 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e610aab80 20 client.17454 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:15.037 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4c70290 0x5587b4c726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e610aab80 4 client.17454 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587b4ba0eb0 con 0x5587b4c70290 2023-07-21T22:35:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.989+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4c70290 0x5587b4c726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.990+0000 7f4e2aff5700 10 client.17454 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:15.038 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.990+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4c70290 0x5587b4c726a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ca1d667b7bade5 server_cookie=49e96202ca9eeea2 in_seq=0 out_seq=0 2023-07-21T22:35:15.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1882902529 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5587b4ba0eb0 con 0x5587b4c70290 2023-07-21T22:35:15.040 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e2aff5700 10 client.17454 handle_command_reply: tid=0 2023-07-21T22:35:15.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 client.17454 shutdown 2023-07-21T22:35:15.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 msgr2=0x5587b4c5fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5fde0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c011d10 tx=0x7f4e3c00f6a0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.041 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 shutdown_connections 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c62af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4c70290 0x5587b4c726a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1882902529 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c5fde0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e2aff5700 0 client.17454 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 >> 172.21.15.66:0/1882902529 conn(0x5587b4c4bef0 msgr2=0x5587b4c513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.993+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 shutdown_connections 2023-07-21T22:35:15.042 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.994+0000 7f4e610aab80 1 -- 172.21.15.66:0/1882902529 wait complete. 2023-07-21T22:35:15.043 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:14.994+0000 7f4e610aab80 20 client.17454 trim_cache size 0 max 16384 2023-07-21T22:35:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.011+0000 7f4e610aab80 1 Processor -- start 2023-07-21T22:35:15.059 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 -- start start 2023-07-21T22:35:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c4ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c54920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4b01520 con 0x5587b4c5b1b0 2023-07-21T22:35:15.061 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4c57740 con 0x5587b4c56930 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e610aab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4c5acc0 con 0x5587b4c5bac0 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c4ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c4ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38588/0 (socket says 172.21.15.66:38588) 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1091809442 learned_addr learned my addr 172.21.15.66:0/1091809442 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.012+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c54920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.062 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1091809442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 msgr2=0x5587b4c54920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c54920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1091809442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 msgr2=0x5587b4c62720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c62720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1091809442 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b49fc250 con 0x5587b4c56930 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c4ff60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4e3809aee0 tx=0x7f4e38009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd254fe444aedb5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e2dffb700 1 -- 172.21.15.66:0/1091809442 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3809aae0 con 0x5587b4c56930 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e2dffb700 1 -- 172.21.15.66:0/1091809442 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3809a900 con 0x5587b4c56930 2023-07-21T22:35:15.063 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.013+0000 7f4e2dffb700 1 -- 172.21.15.66:0/1091809442 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3809afa0 con 0x5587b4c56930 2023-07-21T22:35:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 -- 172.21.15.66:0/1091809442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 msgr2=0x5587b4c4ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.065 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c4ff60 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4e3809aee0 tx=0x7f4e38009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 -- 172.21.15.66:0/1091809442 shutdown_connections 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c4ff60 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4c54920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1091809442 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c62720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 -- 172.21.15.66:0/1091809442 >> 172.21.15.66:0/1091809442 conn(0x5587b4c4bef0 msgr2=0x5587b4c6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 -- 172.21.15.66:0/1091809442 shutdown_connections 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.014+0000 7f4e610aab80 1 -- 172.21.15.66:0/1091809442 wait complete. 2023-07-21T22:35:15.066 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 -- start start 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 Processor -- start 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c65370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4cc3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4c66040 con 0x5587b4c5b1b0 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4c661b0 con 0x5587b4c56930 2023-07-21T22:35:15.067 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b4cc3690 con 0x5587b4c5bac0 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.015+0000 7f4e610aab80 10 client.0 _ll_register_callbacks cb 0x7f4e3c0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c65370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c65370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38604/0 (socket says 172.21.15.66:38604) 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1330398196 learned_addr learned my addr 172.21.15.66:0/1330398196 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e477fe700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4cc3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1330398196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 msgr2=0x5587b4cc3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.068 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4cc3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1330398196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 msgr2=0x5587b4c658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 -- 172.21.15.66:0/1330398196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587b49d5de0 con 0x5587b4c56930 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e46ffd700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c65370 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4e380d6240 tx=0x7f4e38005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf008e5cff058efe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.016+0000 7f4e2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1330398196 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e380d1ea0 con 0x5587b4c56930 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1330398196 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4e38005660 con 0x5587b4c56930 2023-07-21T22:35:15.069 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e3809ad70 con 0x5587b4c56930 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1330398196 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e3809aae0 con 0x5587b4c56930 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e610aab80 10 client.27521 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x5587b49d42b0 con 0x5587b4c56930 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1330398196 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e3809ad70 con 0x5587b4c56930 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.017+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1330398196 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4e380976b0 con 0x5587b4c56930 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.018+0000 7f4e610aab80 10 client.27521 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:15.070 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.018+0000 7f4e610aab80 10 client.27521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.018+0000 7f4e610aab80 20 client.27521 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.018+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4cc48e0 0x5587b4cc6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.018+0000 7f4e610aab80 4 client.27521 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:35:15.071 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.018+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587b4b63050 con 0x5587b4cc48e0 2023-07-21T22:35:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.021+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4cc48e0 0x5587b4cc6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.022+0000 7f4e467fc700 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4cc48e0 0x5587b4cc6cf0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5613f94d1a5a598 server_cookie=39d22f2844e419d1 in_seq=0 out_seq=0 2023-07-21T22:35:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.022+0000 7f4e2aff5700 10 client.27521 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e2aff5700 1 -- 172.21.15.66:0/1330398196 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587b4b63050 con 0x5587b4cc48e0 2023-07-21T22:35:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e2aff5700 10 client.27521 handle_command_reply: tid=0 2023-07-21T22:35:15.072 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 client.27521 shutdown 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 msgr2=0x5587b4c65370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c65370 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4e380d6240 tx=0x7f4e38005ef0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 shutdown_connections 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5587b4c56930 0x5587b4c65370 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x5587b4cc48e0 0x5587b4cc6cf0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5587b4c5bac0 0x5587b4cc3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 --2- 172.21.15.66:0/1330398196 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x5587b4c5b1b0 0x5587b4c658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.073 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e2aff5700 0 client.27521 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.024+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 >> 172.21.15.66:0/1330398196 conn(0x5587b4c4bef0 msgr2=0x5587b4c6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.025+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 shutdown_connections 2023-07-21T22:35:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.025+0000 7f4e610aab80 1 -- 172.21.15.66:0/1330398196 wait complete. 2023-07-21T22:35:15.074 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.025+0000 7f4e610aab80 20 client.27521 trim_cache size 0 max 16384 2023-07-21T22:35:15.075 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:35:15.075 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e300a1340 msgr2=0x7f4e300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e300a1340 0x7f4e300a37f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e400060b0 tx=0x7f4e40009040 comp rx=0 tx=0).stop 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e480fe8b0 msgr2=0x7f4e4810b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e480fe8b0 0x7f4e4810b780 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4e380098e0 tx=0x7f4e38004730 comp rx=0 tx=0).stop 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 shutdown_connections 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f4e300a1340 0x7f4e300a37f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4e4810c460 0x7f4e481082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.076 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f4e480fe8b0 0x7f4e4810b780 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 --2- 172.21.15.66:0/1737804611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4e480ff1c0 0x7f4e4810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 >> 172.21.15.66:0/1737804611 conn(0x7f4e480fa4e0 msgr2=0x7f4e4810d820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 shutdown_connections 2023-07-21T22:35:15.077 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:15.026+0000 7f4e4d7ff700 1 -- 172.21.15.66:0/1737804611 wait complete. 2023-07-21T22:35:15.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 21622 31.78 MB/sec warmup 42 sec latency 14749.179 ms 2023-07-21T22:35:16.098 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T22:35:16.098 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T22:35:16.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:16 smithi104 ceph-mon[129021]: pgmap v432: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 515 op/s 2023-07-21T22:35:16.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:16 smithi066 ceph-mon[126379]: pgmap v432: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 515 op/s 2023-07-21T22:35:16.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:16 smithi149 ceph-mon[129268]: pgmap v432: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.9 MiB/s wr, 515 op/s 2023-07-21T22:35:16.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.441+0000 7f1098bff700 1 -- 172.21.15.66:0/2108435822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10940fea90 msgr2=0x7f10940fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.441+0000 7f1098bff700 1 --2- 172.21.15.66:0/2108435822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10940fea90 0x7f10940fee60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f108401a9a0 tx=0x7f108401acb0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 -- 172.21.15.66:0/2108435822 shutdown_connections 2023-07-21T22:35:16.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 --2- 172.21.15.66:0/2108435822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1094103e40 0x7f10941042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 --2- 172.21.15.66:0/2108435822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f10940fea90 0x7f10940fee60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 --2- 172.21.15.66:0/2108435822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10940ff3a0 0x7f1094103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 -- 172.21.15.66:0/2108435822 >> 172.21.15.66:0/2108435822 conn(0x7f10940fa4b0 msgr2=0x7f10940fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 -- 172.21.15.66:0/2108435822 shutdown_connections 2023-07-21T22:35:16.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.444+0000 7f1098bff700 1 -- 172.21.15.66:0/2108435822 wait complete. 2023-07-21T22:35:16.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.446+0000 7f1098bff700 1 Processor -- start 2023-07-21T22:35:16.500 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 -- start start 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10940fea90 0x7f109410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10940ff3a0 0x7f109410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 0x7f109419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109419fd30 con 0x7f10940ff3a0 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109419fea0 con 0x7f10940fea90 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.448+0000 7f1098bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10941a0060 con 0x7f1094103e40 2023-07-21T22:35:16.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.449+0000 7f109259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10940fea90 0x7f109410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.449+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 0x7f109419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.449+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 0x7f109419f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57788/0 (socket says 172.21.15.66:57788) 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.449+0000 7f1092d9d700 1 -- 172.21.15.66:0/592232822 learned_addr learned my addr 172.21.15.66:0/592232822 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.451+0000 7f1092d9d700 1 -- 172.21.15.66:0/592232822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10940fea90 msgr2=0x7f109410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.451+0000 7f1092d9d700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10940fea90 0x7f109410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.451+0000 7f1092d9d700 1 -- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10940ff3a0 msgr2=0x7f109410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.451+0000 7f1092d9d700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10940ff3a0 0x7f109410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.451+0000 7f1092d9d700 1 -- 172.21.15.66:0/592232822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108401a600 con 0x7f1094103e40 2023-07-21T22:35:16.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.454+0000 7f1092d9d700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 0x7f109419f760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f108c00b970 tx=0x7f108c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.455+0000 7f10837fe700 1 -- 172.21.15.66:0/592232822 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c00c930 con 0x7f1094103e40 2023-07-21T22:35:16.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.455+0000 7f10837fe700 1 -- 172.21.15.66:0/592232822 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108c014440 con 0x7f1094103e40 2023-07-21T22:35:16.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.455+0000 7f10837fe700 1 -- 172.21.15.66:0/592232822 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0075b0 con 0x7f1094103e40 2023-07-21T22:35:16.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.455+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10941a0340 con 0x7f1094103e40 2023-07-21T22:35:16.504 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.455+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10941a0800 con 0x7f1094103e40 2023-07-21T22:35:16.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.466+0000 7f10ac603b80 1 Processor -- start 2023-07-21T22:35:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.470+0000 7f10ac603b80 1 -- start start 2023-07-21T22:35:16.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.470+0000 7f10837fe700 1 -- 172.21.15.66:0/592232822 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f108c0145b0 con 0x7f1094103e40 2023-07-21T22:35:16.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7bb40 0x55c01ad763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad76e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f10ac603b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ac7bd30 con 0x55c01ad7bb40 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f10ac603b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01acceac0 con 0x55c01ad7b230 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f10ac603b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01acc7f50 con 0x55c01ad769b0 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.471+0000 7f1091d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.472+0000 7f1091d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38616/0 (socket says 172.21.15.66:38616) 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.472+0000 7f1091d9b700 1 -- 172.21.15.66:0/2287131003 learned_addr learned my addr 172.21.15.66:0/2287131003 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.472+0000 7f1091d9b700 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad76e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.473+0000 7f1091d9b700 1 -- 172.21.15.66:0/2287131003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 msgr2=0x55c01ad76e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.473+0000 7f1091d9b700 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad76e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.473+0000 7f1091d9b700 1 -- 172.21.15.66:0/2287131003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7bb40 msgr2=0x55c01ad763d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.473+0000 7f1091d9b700 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7bb40 0x55c01ad763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.473+0000 7f1091d9b700 1 -- 172.21.15.66:0/2287131003 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c01aaf5e60 con 0x55c01ad7b230 2023-07-21T22:35:16.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.473+0000 7f1091d9b700 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7b600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10880138d0 tx=0x7f1088009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb3622f571a7863 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.475+0000 7f10837fe700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f107c0a13c0 0x7f107c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.475+0000 7f10837fe700 1 -- 172.21.15.66:0/592232822 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f108c0cc670 con 0x7f1094103e40 2023-07-21T22:35:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.476+0000 7f1080ff9700 1 -- 172.21.15.66:0/2287131003 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1088004a30 con 0x55c01ad7b230 2023-07-21T22:35:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.476+0000 7f1080ff9700 1 -- 172.21.15.66:0/2287131003 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1088019070 con 0x55c01ad7b230 2023-07-21T22:35:16.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.476+0000 7f1080ff9700 1 -- 172.21.15.66:0/2287131003 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800f6e0 con 0x55c01ad7b230 2023-07-21T22:35:16.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.477+0000 7f109259c700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f107c0a13c0 0x7f107c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.478+0000 7f10ac603b80 1 -- 172.21.15.66:0/2287131003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 msgr2=0x55c01ad7b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.478+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7b600 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f10880138d0 tx=0x7f1088009710 comp rx=0 tx=0).stop 2023-07-21T22:35:16.527 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.479+0000 7f109259c700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f107c0a13c0 0x7f107c0a3870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c01ad77630 tx=0x7f108401c270 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.483+0000 7f10ac603b80 1 -- 172.21.15.66:0/2287131003 shutdown_connections 2023-07-21T22:35:16.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.483+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7b600 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.483+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad76e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.483+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2287131003 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7bb40 0x55c01ad763d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.483+0000 7f10ac603b80 1 -- 172.21.15.66:0/2287131003 >> 172.21.15.66:0/2287131003 conn(0x55c01ad6bf70 msgr2=0x55c01ad718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.486+0000 7f10ac603b80 1 -- 172.21.15.66:0/2287131003 shutdown_connections 2023-07-21T22:35:16.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.486+0000 7f10ac603b80 1 -- 172.21.15.66:0/2287131003 wait complete. 2023-07-21T22:35:16.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.486+0000 7f10ac603b80 1 -- start start 2023-07-21T22:35:16.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.487+0000 7f10ac603b80 1 Processor -- start 2023-07-21T22:35:16.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad82c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad7fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 0x55c01ad80510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ac67b40 con 0x55c01ad7b230 2023-07-21T22:35:16.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ac67840 con 0x55c01ad7bb40 2023-07-21T22:35:16.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ab4b9a0 con 0x55c01ad769b0 2023-07-21T22:35:16.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f10ac603b80 10 client.0 _ll_register_callbacks cb 0x55c01ac657e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:16.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 0x55c01ad80510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 0x55c01ad80510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38628/0 (socket says 172.21.15.66:38628) 2023-07-21T22:35:16.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f1092d9d700 1 -- 172.21.15.66:0/4107822399 learned_addr learned my addr 172.21.15.66:0/4107822399 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.488+0000 7f1091d9b700 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad7fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1092d9d700 1 -- 172.21.15.66:0/4107822399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 msgr2=0x55c01ad82c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1092d9d700 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad82c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1092d9d700 1 -- 172.21.15.66:0/4107822399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 msgr2=0x55c01ad7fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1092d9d700 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad7fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1092d9d700 1 -- 172.21.15.66:0/4107822399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10880135b0 con 0x55c01ad7bb40 2023-07-21T22:35:16.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1092d9d700 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 0x55c01ad80510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f108c0a1370 tx=0x7f108c0c9f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74244e8826b2dce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.489+0000 7f1079ffb700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:35:16.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.490+0000 7f1079ffb700 1 -- 172.21.15.66:0/4107822399 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c09d480 con 0x55c01ad7bb40 2023-07-21T22:35:16.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.490+0000 7f1079ffb700 1 -- 172.21.15.66:0/4107822399 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f108c0a18a0 con 0x55c01ad7bb40 2023-07-21T22:35:16.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.490+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c01ac21570 con 0x55c01ad7bb40 2023-07-21T22:35:16.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.490+0000 7f1079ffb700 1 -- 172.21.15.66:0/4107822399 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c09d120 con 0x55c01ad7bb40 2023-07-21T22:35:16.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.491+0000 7f10ac603b80 10 client.27530 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:16.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.491+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c01aadece0 con 0x55c01ad7bb40 2023-07-21T22:35:16.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f1079ffb700 1 -- 172.21.15.66:0/4107822399 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c01ac21570 con 0x55c01ad7bb40 2023-07-21T22:35:16.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f1079ffb700 1 -- 172.21.15.66:0/4107822399 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f108c0a1410 con 0x55c01ad7bb40 2023-07-21T22:35:16.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f10ac603b80 10 client.27530 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f10ac603b80 10 client.27530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f10ac603b80 20 client.27530 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f10ac603b80 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ad90420 0x55c01ad92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f10ac603b80 4 client.27530 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c01acc0f30 con 0x55c01ad90420 2023-07-21T22:35:16.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.492+0000 7f109259c700 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ad90420 0x55c01ad92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.493+0000 7f109259c700 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ad90420 0x55c01ad92830 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93683d28f371d73 server_cookie=ad34f81bac48894e in_seq=0 out_seq=0 2023-07-21T22:35:16.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.493+0000 7f1079ffb700 10 client.27530 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.495+0000 7f1079ffb700 1 -- 172.21.15.66:0/4107822399 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c01acc0f30 con 0x55c01ad90420 2023-07-21T22:35:16.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.495+0000 7f1079ffb700 10 client.27530 handle_command_reply: tid=0 2023-07-21T22:35:16.543 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.495+0000 7f10ac603b80 1 client.27530 shutdown 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 msgr2=0x55c01ad80510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 0x55c01ad80510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f108c0a1370 tx=0x7f108c0c9f20 comp rx=0 tx=0).stop 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 shutdown_connections 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7bb40 0x55c01ad80510 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ad90420 0x55c01ad92830 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad769b0 0x55c01ad82c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.544 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 --2- 172.21.15.66:0/4107822399 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad7fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 >> 172.21.15.66:0/4107822399 conn(0x55c01ad6bf70 msgr2=0x55c01ad714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f1079ffb700 0 client.27530 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 shutdown_connections 2023-07-21T22:35:16.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.496+0000 7f10ac603b80 1 -- 172.21.15.66:0/4107822399 wait complete. 2023-07-21T22:35:16.545 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.497+0000 7f10ac603b80 20 client.27530 trim_cache size 0 max 16384 2023-07-21T22:35:16.567 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.519+0000 7f10ac603b80 1 Processor -- start 2023-07-21T22:35:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 -- start start 2023-07-21T22:35:16.568 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad769b0 0x55c01ad762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad87590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ad77750 con 0x55c01ad7b230 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ad6e090 con 0x55c01ad769b0 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f10ac603b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ad690d0 con 0x55c01ad7bb40 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.569 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.520+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57818/0 (socket says 172.21.15.66:57818) 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 -- 172.21.15.66:0/933339154 learned_addr learned my addr 172.21.15.66:0/933339154 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1091d9b700 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad87590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f109259c700 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad769b0 0x55c01ad762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 -- 172.21.15.66:0/933339154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad769b0 msgr2=0x55c01ad762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad769b0 0x55c01ad762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 -- 172.21.15.66:0/933339154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 msgr2=0x55c01ad87590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad87590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.570 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 -- 172.21.15.66:0/933339154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c01ad97480 con 0x55c01ad7bb40 2023-07-21T22:35:16.571 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1091d9b700 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad87590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.521+0000 7f1092d9d700 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7d770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f108c00bda0 tx=0x7f108c095040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f58122970c0b129c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10817fa700 1 -- 172.21.15.66:0/933339154 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0a18a0 con 0x55c01ad7bb40 2023-07-21T22:35:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10817fa700 1 -- 172.21.15.66:0/933339154 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f108c09d480 con 0x55c01ad7bb40 2023-07-21T22:35:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10817fa700 1 -- 172.21.15.66:0/933339154 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c0a1570 con 0x55c01ad7bb40 2023-07-21T22:35:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10ac603b80 1 -- 172.21.15.66:0/933339154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 msgr2=0x55c01ad7d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10ac603b80 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7d770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f108c00bda0 tx=0x7f108c095040 comp rx=0 tx=0).stop 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10ac603b80 1 -- 172.21.15.66:0/933339154 shutdown_connections 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10ac603b80 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad769b0 0x55c01ad762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.522+0000 7f10ac603b80 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7d770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.523+0000 7f10ac603b80 1 --2- 172.21.15.66:0/933339154 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad7b230 0x55c01ad87590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.523+0000 7f10ac603b80 1 -- 172.21.15.66:0/933339154 >> 172.21.15.66:0/933339154 conn(0x55c01ad6bf70 msgr2=0x55c01ad7d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.523+0000 7f10ac603b80 1 -- 172.21.15.66:0/933339154 shutdown_connections 2023-07-21T22:35:16.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.523+0000 7f10ac603b80 1 -- 172.21.15.66:0/933339154 wait complete. 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.523+0000 7f10ac603b80 1 -- start start 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.523+0000 7f10ac603b80 1 Processor -- start 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad769b0 0x55c01ad7ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ade4500 con 0x55c01ad769b0 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ade4670 con 0x55c01ad7b230 2023-07-21T22:35:16.575 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c01ade4830 con 0x55c01ad7bb40 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f10ac603b80 10 client.0 _ll_register_callbacks cb 0x7f108c0992f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f1092d9d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57830/0 (socket says 172.21.15.66:57830) 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f1092d9d700 1 -- 172.21.15.66:0/2952708073 learned_addr learned my addr 172.21.15.66:0/2952708073 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f1091d9b700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.524+0000 7f109259c700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad769b0 0x55c01ad7ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1092d9d700 1 -- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 msgr2=0x55c01ad7f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.576 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1092d9d700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1092d9d700 1 -- 172.21.15.66:0/2952708073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad769b0 msgr2=0x55c01ad7ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1092d9d700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad769b0 0x55c01ad7ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1092d9d700 1 -- 172.21.15.66:0/2952708073 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c01aadece0 con 0x55c01ad7bb40 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1091d9b700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1092d9d700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7f8c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f108c099610 tx=0x7f108c09d350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0006734d4f15a2b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1079ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1079ffb700 1 -- 172.21.15.66:0/2952708073 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c098940 con 0x55c01ad7bb40 2023-07-21T22:35:16.577 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1088019070 con 0x55c01ad7bb40 2023-07-21T22:35:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1079ffb700 1 -- 172.21.15.66:0/2952708073 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f108c095360 con 0x55c01ad7bb40 2023-07-21T22:35:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f1079ffb700 1 -- 172.21.15.66:0/2952708073 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f108c098aa0 con 0x55c01ad7bb40 2023-07-21T22:35:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.525+0000 7f10ac603b80 10 client.27508 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:16.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c01ab1c2d0 con 0x55c01ad7bb40 2023-07-21T22:35:16.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f1079ffb700 1 -- 172.21.15.66:0/2952708073 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1088019070 con 0x55c01ad7bb40 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f1079ffb700 1 -- 172.21.15.66:0/2952708073 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f108c0b7a80 con 0x55c01ad7bb40 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 10 client.27508 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 10 client.27508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 20 client.27508 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ade5aa0 0x55c01ade7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 4 client.27508 mds_command: new command op to 24572 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c01ac830d0 con 0x55c01ade5aa0 2023-07-21T22:35:16.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.526+0000 7f109259c700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ade5aa0 0x55c01ade7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.528+0000 7f109259c700 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ade5aa0 0x55c01ade7e80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2929d13e4c60c44 server_cookie=1a00426aa6105041 in_seq=0 out_seq=0 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.528+0000 7f1079ffb700 10 client.27508 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.529+0000 7f1079ffb700 1 -- 172.21.15.66:0/2952708073 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c01ac830d0 con 0x55c01ade5aa0 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.529+0000 7f1079ffb700 10 client.27508 handle_command_reply: tid=0 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.529+0000 7f10ac603b80 1 client.27508 shutdown 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 msgr2=0x55c01ad7f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7f8c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f108c099610 tx=0x7f108c09d350 comp rx=0 tx=0).stop 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 shutdown_connections 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55c01ade5aa0 0x55c01ade7e80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c01ad7bb40 0x55c01ad7f8c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55c01ad769b0 0x55c01ad7ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 --2- 172.21.15.66:0/2952708073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c01ad7b230 0x55c01ad7f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 >> 172.21.15.66:0/2952708073 conn(0x55c01ad6bf70 msgr2=0x55c01ad7d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f1079ffb700 0 client.27508 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.582 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 shutdown_connections 2023-07-21T22:35:16.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.530+0000 7f10ac603b80 1 -- 172.21.15.66:0/2952708073 wait complete. 2023-07-21T22:35:16.583 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.531+0000 7f10ac603b80 20 client.27508 trim_cache size 0 max 16384 2023-07-21T22:35:16.583 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:35:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "return_code": 0, 2023-07-21T22:35:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "scrub_tag": "f11d1781-49ce-43b3-bbf7-7d7c4afcde1b", 2023-07-21T22:35:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": "asynchronous" 2023-07-21T22:35:16.584 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:35:16.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f107c0a13c0 msgr2=0x7f107c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f107c0a13c0 0x7f107c0a3870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c01ad77630 tx=0x7f108401c270 comp rx=0 tx=0).stop 2023-07-21T22:35:16.584 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 msgr2=0x7f109419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 0x7f109419f760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f108c00b970 tx=0x7f108c00bc80 comp rx=0 tx=0).stop 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 shutdown_connections 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10940fea90 0x7f109410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f107c0a13c0 0x7f107c0a3870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1094103e40 0x7f109419f760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 --2- 172.21.15.66:0/592232822 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f10940ff3a0 0x7f109410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.585 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.532+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 >> 172.21.15.66:0/592232822 conn(0x7f10940fa4b0 msgr2=0x7f1094101990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.533+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 shutdown_connections 2023-07-21T22:35:16.586 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.533+0000 7f1098bff700 1 -- 172.21.15.66:0/592232822 wait complete. 2023-07-21T22:35:16.602 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:35:16.871 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.824+0000 7f74563ff700 1 -- 172.21.15.66:0/4075256184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 msgr2=0x7f74501010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.872 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.824+0000 7f74563ff700 1 --2- 172.21.15.66:0/4075256184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 0x7f74501010b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f74400098f0 tx=0x7f7440009c00 comp rx=0 tx=0).stop 2023-07-21T22:35:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 -- 172.21.15.66:0/4075256184 shutdown_connections 2023-07-21T22:35:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 --2- 172.21.15.66:0/4075256184 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f745010a020 0x7f745010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 --2- 172.21.15.66:0/4075256184 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 0x7f74501010b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 --2- 172.21.15.66:0/4075256184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74501015f0 0x7f7450109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 -- 172.21.15.66:0/4075256184 >> 172.21.15.66:0/4075256184 conn(0x7f74500fa4b0 msgr2=0x7f74500fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 -- 172.21.15.66:0/4075256184 shutdown_connections 2023-07-21T22:35:16.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.825+0000 7f74563ff700 1 -- 172.21.15.66:0/4075256184 wait complete. 2023-07-21T22:35:16.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.826+0000 7f74563ff700 1 Processor -- start 2023-07-21T22:35:16.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.826+0000 7f74563ff700 1 -- start start 2023-07-21T22:35:16.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f74563ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 0x7f745019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f74563ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74501015f0 0x7f745019bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f74563ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 0x7f7450195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f74563ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74501961b0 con 0x7f7450100ce0 2023-07-21T22:35:16.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f74563ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7450196320 con 0x7f745010a020 2023-07-21T22:35:16.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f74563ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74501964e0 con 0x7f74501015f0 2023-07-21T22:35:16.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f745499c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 0x7f7450195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f745499c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 0x7f7450195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38660/0 (socket says 172.21.15.66:38660) 2023-07-21T22:35:16.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f745499c700 1 -- 172.21.15.66:0/917858071 learned_addr learned my addr 172.21.15.66:0/917858071 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.827+0000 7f744f7fe700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74501015f0 0x7f745019bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.828+0000 7f745499c700 1 -- 172.21.15.66:0/917858071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74501015f0 msgr2=0x7f745019bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.828+0000 7f745499c700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74501015f0 0x7f745019bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.828+0000 7f745499c700 1 -- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 msgr2=0x7f745019ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.828+0000 7f745499c700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 0x7f745019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.828+0000 7f745499c700 1 -- 172.21.15.66:0/917858071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7440009520 con 0x7f745010a020 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.828+0000 7f744f7fe700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74501015f0 0x7f745019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:16.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.829+0000 7f745499c700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 0x7f7450195b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f744800ef30 tx=0x7f744800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.829+0000 7f744d7fa700 1 -- 172.21.15.66:0/917858071 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448018070 con 0x7f745010a020 2023-07-21T22:35:16.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.829+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7450196760 con 0x7f745010a020 2023-07-21T22:35:16.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.829+0000 7f744d7fa700 1 -- 172.21.15.66:0/917858071 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f744800f040 con 0x7f745010a020 2023-07-21T22:35:16.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.829+0000 7f744d7fa700 1 -- 172.21.15.66:0/917858071 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448003dc0 con 0x7f745010a020 2023-07-21T22:35:16.879 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.829+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74500fea40 con 0x7f745010a020 2023-07-21T22:35:16.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.834+0000 7f744d7fa700 1 -- 172.21.15.66:0/917858071 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f744800caa0 con 0x7f745010a020 2023-07-21T22:35:16.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.835+0000 7f7469cedb80 1 Processor -- start 2023-07-21T22:35:16.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 -- start start 2023-07-21T22:35:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5cfcbcb0 con 0x555f5d0d42d0 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d01ea40 con 0x555f5d0d39c0 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.836+0000 7f7469cedb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d017ed0 con 0x555f5d0c6930 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:48424/0 (socket says 172.21.15.66:48424) 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744ffff700 1 -- 172.21.15.66:0/2736943937 learned_addr learned my addr 172.21.15.66:0/2736943937 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744d7fa700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74380a1340 0x7f74380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744d7fa700 1 -- 172.21.15.66:0/917858071 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f74480ccc90 con 0x7f745010a020 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744f7fe700 1 -- 172.21.15.66:0/2736943937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 msgr2=0x555f5d0d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744f7fe700 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744f7fe700 1 -- 172.21.15.66:0/2736943937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 msgr2=0x555f5d0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744f7fe700 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744f7fe700 1 -- 172.21.15.66:0/2736943937 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f5ce45de0 con 0x555f5d0c6930 2023-07-21T22:35:16.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f745499c700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74380a1340 0x7f74380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.837+0000 7f744ffff700 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.838+0000 7f744f7fe700 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0c6da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7444005fd0 tx=0x7f7444012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa63631e1efb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.838+0000 7f74367fc700 1 -- 172.21.15.66:0/2736943937 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444012e90 con 0x555f5d0c6930 2023-07-21T22:35:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.838+0000 7f74367fc700 1 -- 172.21.15.66:0/2736943937 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7444009e70 con 0x555f5d0c6930 2023-07-21T22:35:16.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.838+0000 7f74367fc700 1 -- 172.21.15.66:0/2736943937 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744401b6e0 con 0x555f5d0c6930 2023-07-21T22:35:16.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 -- 172.21.15.66:0/2736943937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 msgr2=0x555f5d0c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f745499c700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74380a1340 0x7f74380a37f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74480ca040 tx=0x7f74480b7f90 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0c6da0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7444005fd0 tx=0x7f7444012730 comp rx=0 tx=0).stop 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 -- 172.21.15.66:0/2736943937 shutdown_connections 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0c6da0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2736943937 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.839+0000 7f7469cedb80 1 -- 172.21.15.66:0/2736943937 >> 172.21.15.66:0/2736943937 conn(0x555f5d0bbef0 msgr2=0x555f5d0c1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.840+0000 7f7469cedb80 1 -- 172.21.15.66:0/2736943937 shutdown_connections 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.840+0000 7f7469cedb80 1 -- 172.21.15.66:0/2736943937 wait complete. 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.840+0000 7f7469cedb80 1 -- start start 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.840+0000 7f7469cedb80 1 Processor -- start 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0db410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5cfb7ac0 con 0x555f5d0d42d0 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5cfb77c0 con 0x555f5d0d39c0 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5ce9b920 con 0x555f5d0c6930 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f7469cedb80 10 client.0 _ll_register_callbacks cb 0x555f5cfb5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f744ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f744ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:38686/0 (socket says 172.21.15.66:38686) 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.841+0000 7f744ffff700 1 -- 172.21.15.66:0/2795562327 learned_addr learned my addr 172.21.15.66:0/2795562327 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.842+0000 7f744ffff700 1 -- 172.21.15.66:0/2795562327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 msgr2=0x555f5d0db410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.842+0000 7f744ffff700 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0db410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.842+0000 7f744ffff700 1 -- 172.21.15.66:0/2795562327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 msgr2=0x555f5d0d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.842+0000 7f744ffff700 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.842+0000 7f744ffff700 1 -- 172.21.15.66:0/2795562327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745010d170 con 0x555f5d0d39c0 2023-07-21T22:35:16.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.842+0000 7f744f7fe700 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f744ffff700 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d8690 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7440005e90 tx=0x7f7440009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699ae8a04bd2d2d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f742f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:35:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f742f7fe700 1 -- 172.21.15.66:0/2795562327 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744001e070 con 0x555f5d0d39c0 2023-07-21T22:35:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f5cf714f0 con 0x555f5d0d39c0 2023-07-21T22:35:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f742f7fe700 1 -- 172.21.15.66:0/2795562327 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f744000b560 con 0x555f5d0d39c0 2023-07-21T22:35:16.895 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f742f7fe700 1 -- 172.21.15.66:0/2795562327 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f744000b720 con 0x555f5d0d39c0 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f7469cedb80 10 client.27545 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.843+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f7444009a30 con 0x555f5d0d39c0 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f742f7fe700 1 -- 172.21.15.66:0/2795562327 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f5cf714f0 con 0x555f5d0d39c0 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f742f7fe700 1 -- 172.21.15.66:0/2795562327 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7440015020 con 0x555f5d0d39c0 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f7469cedb80 10 client.27545 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f7469cedb80 10 client.27545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:16.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f7469cedb80 20 client.27545 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:16.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d0e8d80 0x555f5d0eb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f7469cedb80 4 client.27545 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:16.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f5d010eb0 con 0x555f5d0e8d80 2023-07-21T22:35:16.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.844+0000 7f744f7fe700 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d0e8d80 0x555f5d0eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.847+0000 7f744f7fe700 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d0e8d80 0x555f5d0eb190 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895af2cc3e8aeb67 server_cookie=1556886af2d28778 in_seq=0 out_seq=0 2023-07-21T22:35:16.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.847+0000 7f742f7fe700 10 client.27545 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f742f7fe700 1 -- 172.21.15.66:0/2795562327 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f5d010eb0 con 0x555f5d0e8d80 2023-07-21T22:35:16.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f742f7fe700 10 client.27545 handle_command_reply: tid=0 2023-07-21T22:35:16.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 client.27545 shutdown 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 msgr2=0x555f5d0d8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d8690 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7440005e90 tx=0x7f7440009400 comp rx=0 tx=0).stop 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 shutdown_connections 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d0e8d80 0x555f5d0eb190 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0db410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 --2- 172.21.15.66:0/2795562327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0d8690 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.899 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f742f7fe700 0 client.27545 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.848+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 >> 172.21.15.66:0/2795562327 conn(0x555f5d0bbef0 msgr2=0x555f5d0c13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.849+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 shutdown_connections 2023-07-21T22:35:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.849+0000 7f7469cedb80 1 -- 172.21.15.66:0/2795562327 wait complete. 2023-07-21T22:35:16.900 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.849+0000 7f7469cedb80 20 client.27545 trim_cache size 0 max 16384 2023-07-21T22:35:16.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.869+0000 7f7469cedb80 1 Processor -- start 2023-07-21T22:35:16.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.869+0000 7f7469cedb80 1 -- start start 2023-07-21T22:35:16.917 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f7469cedb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d0d9c70 con 0x555f5d0d42d0 2023-07-21T22:35:16.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f7469cedb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d0d6bf0 con 0x555f5d0d39c0 2023-07-21T22:35:16.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f7469cedb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d0d34d0 con 0x555f5d0c6930 2023-07-21T22:35:16.918 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f744f7fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0dfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57864/0 (socket says 172.21.15.66:57864) 2023-07-21T22:35:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f744f7fe700 1 -- 172.21.15.66:0/1544999989 learned_addr learned my addr 172.21.15.66:0/1544999989 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f744f7fe700 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.870+0000 7f744ffff700 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f744f7fe700 1 -- 172.21.15.66:0/1544999989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 msgr2=0x555f5d0db040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f744f7fe700 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.920 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f744f7fe700 1 -- 172.21.15.66:0/1544999989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 msgr2=0x555f5d0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f744f7fe700 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f744f7fe700 1 -- 172.21.15.66:0/1544999989 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f5ce2ec60 con 0x555f5d0c6930 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f744f7fe700 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0dfd80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7444005fd0 tx=0x7f744401d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32a63551b154d54f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f7436ffd700 1 -- 172.21.15.66:0/1544999989 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444030be0 con 0x555f5d0c6930 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f7436ffd700 1 -- 172.21.15.66:0/1544999989 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7444020040 con 0x555f5d0c6930 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.871+0000 7f7436ffd700 1 -- 172.21.15.66:0/1544999989 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444004290 con 0x555f5d0c6930 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 -- 172.21.15.66:0/1544999989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 msgr2=0x555f5d0dfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.921 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0dfd80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7444005fd0 tx=0x7f744401d040 comp rx=0 tx=0).stop 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 -- 172.21.15.66:0/1544999989 shutdown_connections 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0d42d0 0x555f5d0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0c6930 0x555f5d0dfd80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 --2- 172.21.15.66:0/1544999989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d39c0 0x555f5d0db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 -- 172.21.15.66:0/1544999989 >> 172.21.15.66:0/1544999989 conn(0x555f5d0bbef0 msgr2=0x555f5d0e50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 -- 172.21.15.66:0/1544999989 shutdown_connections 2023-07-21T22:35:16.922 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.872+0000 7f7469cedb80 1 -- 172.21.15.66:0/1544999989 wait complete. 2023-07-21T22:35:16.923 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.873+0000 7f7469cedb80 1 -- start start 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.873+0000 7f7469cedb80 1 Processor -- start 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0c6930 0x555f5d0db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 0x555f5d0d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d42d0 0x555f5d0da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d0daa20 con 0x555f5d0c6930 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d0dab60 con 0x555f5d0d42d0 2023-07-21T22:35:16.925 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f5d0dacf0 con 0x555f5d0d39c0 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f7469cedb80 10 client.0 _ll_register_callbacks cb 0x7f7440009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 0x555f5d0d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 0x555f5d0d9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:57870/0 (socket says 172.21.15.66:57870) 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 -- 172.21.15.66:0/4179080199 learned_addr learned my addr 172.21.15.66:0/4179080199 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744f7fe700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0c6930 0x555f5d0db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744f7fe700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d42d0 0x555f5d0da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 -- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d42d0 msgr2=0x555f5d0da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.926 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d42d0 0x555f5d0da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 -- 172.21.15.66:0/4179080199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0c6930 msgr2=0x555f5d0db1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0c6930 0x555f5d0db1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744ffff700 1 -- 172.21.15.66:0/4179080199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7444022040 con 0x555f5d0d39c0 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.874+0000 7f744f7fe700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0c6930 0x555f5d0db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f744ffff700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 0x555f5d0d9fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74400057b0 tx=0x7f744000eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c53cef62b21ff4c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f742f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:16.927 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f742f7fe700 1 -- 172.21.15.66:0/4179080199 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744001e040 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7444009e70 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f742f7fe700 1 -- 172.21.15.66:0/4179080199 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f74400058e0 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f742f7fe700 1 -- 172.21.15.66:0/4179080199 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7440003e70 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f7469cedb80 10 client.27526 fetch_fsmap learned FSMap version 49 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.875+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x555f5ce6c250 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f742f7fe700 1 -- 172.21.15.66:0/4179080199 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7444009e70 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f742f7fe700 1 -- 172.21.15.66:0/4179080199 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 49) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7440015070 con 0x555f5d0d39c0 2023-07-21T22:35:16.928 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f7469cedb80 10 client.27526 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f7469cedb80 10 client.27526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f7469cedb80 20 client.27526 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f7469cedb80 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d13dd80 0x555f5d140190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f7469cedb80 4 client.27526 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f5cfd3050 con 0x555f5d13dd80 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.876+0000 7f744f7fe700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d13dd80 0x555f5d140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:16.929 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.877+0000 7f744f7fe700 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d13dd80 0x555f5d140190 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef576ef6ed24752 server_cookie=e767e1408c4edda5 in_seq=0 out_seq=0 2023-07-21T22:35:16.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.877+0000 7f742f7fe700 10 client.27526 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.877+0000 7f742f7fe700 1 -- 172.21.15.66:0/4179080199 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555f5cfd3050 con 0x555f5d13dd80 2023-07-21T22:35:16.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.877+0000 7f742f7fe700 10 client.27526 handle_command_reply: tid=0 2023-07-21T22:35:16.930 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.877+0000 7f7469cedb80 1 client.27526 shutdown 2023-07-21T22:35:16.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 msgr2=0x555f5d0d9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 0x555f5d0d9fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f74400057b0 tx=0x7f744000eec0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 shutdown_connections 2023-07-21T22:35:16.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555f5d0d42d0 0x555f5d0da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x555f5d13dd80 0x555f5d140190 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.931 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x555f5d0c6930 0x555f5d0db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f742f7fe700 0 client.27526 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:16.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 --2- 172.21.15.66:0/4179080199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555f5d0d39c0 0x555f5d0d9fa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 >> 172.21.15.66:0/4179080199 conn(0x555f5d0bbef0 msgr2=0x555f5d0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.878+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 shutdown_connections 2023-07-21T22:35:16.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.879+0000 7f7469cedb80 1 -- 172.21.15.66:0/4179080199 wait complete. 2023-07-21T22:35:16.932 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.879+0000 7f7469cedb80 20 client.27526 trim_cache size 0 max 16384 2023-07-21T22:35:16.933 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:35:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T22:35:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": { 2023-07-21T22:35:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "f11d1781-49ce-43b3-bbf7-7d7c4afcde1b": { 2023-07-21T22:35:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/", 2023-07-21T22:35:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "tag": "f11d1781-49ce-43b3-bbf7-7d7c4afcde1b", 2023-07-21T22:35:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: "options": "recursive,force" 2023-07-21T22:35:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:35:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:35:16.934 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:35:16.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74380a1340 msgr2=0x7f74380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74380a1340 0x7f74380a37f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f74480ca040 tx=0x7f74480b7f90 comp rx=0 tx=0).stop 2023-07-21T22:35:16.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 msgr2=0x7f7450195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:16.935 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 0x7f7450195b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f744800ef30 tx=0x7f744800c450 comp rx=0 tx=0).stop 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 shutdown_connections 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f745010a020 0x7f7450195b50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f74380a1340 0x7f74380a37f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f7450100ce0 0x7f745019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 --2- 172.21.15.66:0/917858071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f74501015f0 0x7f745019bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 >> 172.21.15.66:0/917858071 conn(0x7f74500fa4b0 msgr2=0x7f745010c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:16.936 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 shutdown_connections 2023-07-21T22:35:16.937 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:16.880+0000 7f74563ff700 1 -- 172.21.15.66:0/917858071 wait complete. 2023-07-21T22:35:16.937 INFO:tasks.workunit.client.0.smithi066.stdout: 10 22312 31.99 MB/sec warmup 43 sec latency 14846.480 ms 2023-07-21T22:35:16.950 INFO:tasks.cephfs.filesystem:scrub status for tag:f11d1781-49ce-43b3-bbf7-7d7c4afcde1b - {'path': '/', 'tag': 'f11d1781-49ce-43b3-bbf7-7d7c4afcde1b', 'options': 'recursive,force'} 2023-07-21T22:35:17.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:17 smithi066 ceph-mon[126379]: scrub queued for path: / 2023-07-21T22:35:17.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:17 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:17.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:17 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:35:17.198 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:17 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:35:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:17 smithi104 ceph-mon[129021]: scrub queued for path: / 2023-07-21T22:35:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:17 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:17 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:35:17.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:17 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:35:17.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:17 smithi149 ceph-mon[129268]: scrub queued for path: / 2023-07-21T22:35:17.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:17 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:17.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:17 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:35:17.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:17 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:35:17.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 22910 31.88 MB/sec warmup 44 sec latency 2033.480 ms 2023-07-21T22:35:18.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:18 smithi104 ceph-mon[129021]: pgmap v433: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2023-07-21T22:35:18.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:35:18.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:35:18.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:35:18.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:18 smithi066 ceph-mon[126379]: pgmap v433: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2023-07-21T22:35:18.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:18.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:18 smithi149 ceph-mon[129268]: pgmap v433: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2023-07-21T22:35:18.929 INFO:tasks.workunit.client.0.smithi066.stdout: 10 23794 32.27 MB/sec warmup 45 sec latency 3033.557 ms 2023-07-21T22:35:19.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 24613 32.55 MB/sec warmup 46 sec latency 4033.642 ms 2023-07-21T22:35:20.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:20 smithi104 ceph-mon[129021]: pgmap v434: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2023-07-21T22:35:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:20 smithi066 ceph-mon[126379]: pgmap v434: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2023-07-21T22:35:20.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:20 smithi149 ceph-mon[129268]: pgmap v434: 129 pgs: 129 active+clean; 1011 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.2 MiB/s wr, 461 op/s 2023-07-21T22:35:20.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 25490 32.91 MB/sec warmup 47 sec latency 5033.736 ms 2023-07-21T22:35:21.494 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.978+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000009 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ auth{0=1,1=1} v43540 ap=1 f(v0 m2023-07-21T22:22:33.840296+0000 9=0+9) n(v72 rc2023-07-21T22:35:14.554656+0000 b15979126 75=65+10) old_inodes=8 (inest mix dirty) (ifile excl) (iversion lock) caps={25342=pAsLsXsFsx/pAsLsXsFsx@1},l=25342 | dirtyscattered=1 lock=0 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 authpin=1 discoverbase=0 scrubqueue=0 0x55842456e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000009:[<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000009:[<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:22:33.840296+0000 9=0+9)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:14.554656+0000 b15979126 75=65+10)","memory_value.dirstat":"f(v0 m2023-07-21T22:22:33.840296+0000 9=0+9)","memory_value.rstat":"n(v72 rc2023-07-21T22:35:14.554656+0000 b15979126 75=65+10)","error_str":""},"return_code":0} 2023-07-21T22:35:21.494 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.984+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000215 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/COREL/ auth{0=1} v87243 ap=1 f(v2893 m2023-07-21T22:35:10.975683+0000 11=11+0) n(v69 rc2023-07-21T22:35:10.975683+0000 b2861056 12=11+1) old_inodes=7 (inest mix) (ifile excl) (iversion lock) caps={25342=pAsLsXsFsx/pAsLsXsFsx@13173},l=25342 | dirtyscattered=0 request=0 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55842456f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000215:[<0x10000000009/COREL v86032>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000215:[<0x10000000009/COREL v87243>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:10.975683+0000 11=11+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:10.975683+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v2893 m2023-07-21T22:35:10.975683+0000 11=11+0)","memory_value.rstat":"n(v69 rc2023-07-21T22:35:10.975683+0000 b2861056 12=11+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.494 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.986+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/EXCEL/ auth{0=1,1=1} v87451 pv87471 ap=2 f(v1191 m2023-07-21T22:35:15.638648+0000 7=7+0)->f(v1192 m2023-07-21T22:35:15.675648+0000 8=8+0) n(v23 rc2023-07-21T22:35:07.983705+0000 b608576 9=8+1)->n(v23 rc2023-07-21T22:35:07.983705+0000 b608576 9=8+1) old_inodes=7 (inest mix dirty) (ifile mix->sync w=1 flushing) (iversion lock) caps={25342=pAsLsXs/pAsLsXsFsx@1} mcw={1=AsLsXsFs} | dirtyscattered=2 lock=1 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x5584253ef700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000020c:[<0x10000000009/EXCEL v86277>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x1000000020c:[<0x10000000009/EXCEL v87451>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:15.638648+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:07.983705+0000 b608576 9=8+1)","memory_value.dirstat":"f(v1191 m2023-07-21T22:35:15.638648+0000 7=7+0)","memory_value.rstat":"n(v23 rc2023-07-21T22:35:07.983705+0000 b608576 9=8+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.495 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.988+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{0=1,1=1} v87333 pv87469 ap=2 f(v1846 m2023-07-21T22:35:12.727670+0000 7=7+0)->f(v1847 m2023-07-21T22:35:12.800669+0000 5=5+0) n(v39 rc2023-07-21T22:35:05.022727+0000 b780288 6=5+1)->n(v39 rc2023-07-21T22:35:05.022727+0000 b780288 6=5+1) old_inodes=7 (inest mix dirty) (ifile mix->sync w=1 flushing) (iversion lock) caps={25342=pAsLsXs/pAsLsXsFsx@2808} mcw={1=AsLsXsFs} | dirtyscattered=2 request=0 lock=1 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x558425066c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v86325>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v87333>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:12.727670+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:05.022727+0000 b780288 6=5+1)","memory_value.dirstat":"f(v1846 m2023-07-21T22:35:12.727670+0000 7=7+0)","memory_value.rstat":"n(v39 rc2023-07-21T22:35:05.022727+0000 b780288 6=5+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.495 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.989+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000240 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/SEED/ auth v86760 ap=1 f(v0 m2023-07-21T22:34:47.464856+0000 3=3+0) n(v0 rc2023-07-21T22:34:59.015771+0000 b3244032 4=3+1) old_inodes=7 (inest lock) (ifile excl) (iversion lock) caps={25342=pAsLsXsFsx/pAsLsXsFsx@1},l=25342 | importingcaps=0 dirfrag=1 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x5584256c4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000240:[<0x10000000009/SEED v86100>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000240:[<0x10000000009/SEED v86760>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:34:47.464856+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:34:59.015771+0000 b3244032 4=3+1)","memory_value.dirstat":"f(v0 m2023-07-21T22:34:47.464856+0000 3=3+0)","memory_value.rstat":"n(v0 rc2023-07-21T22:34:59.015771+0000 b3244032 4=3+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.495 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.990+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{0=1,1=1} v87465 ap=3 f(v1093 m2023-07-21T22:35:15.807647+0000 5=5+0) n(v26 rc2023-07-21T22:35:15.807647+0000 b790863 6=5+1) old_inodes=7 (isnap sync r=1) (inest mix dirty) (ifile sync->lock g=0) (iversion lock) caps={25342=pAsLsXs/pAsLsXsFsx@2} mcw={1=AsLsXsFs} | dirtyscattered=1 lock=1 importingcaps=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5584256c4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v86361>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v87465>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:15.807647+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:15.807647+0000 b790863 6=5+1)","memory_value.dirstat":"f(v1093 m2023-07-21T22:35:15.807647+0000 5=5+0)","memory_value.rstat":"n(v26 rc2023-07-21T22:35:15.807647+0000 b790863 6=5+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.496 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.991+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{0=1,1=1} fragtree_t(*^3) v87431 ap=1 f(v3257 m2023-07-21T22:35:14.554656+0000 24=24+0) n(v79 rc2023-07-21T22:35:14.554656+0000 b5201920 25=24+1) old_inodes=7 (inest mix) (iversion lock) caps={25342=pAsLsXsFs/pAsLsXsFsx@10316} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 importingcaps=0 importing=0 dirfrag=8 caps=1 stickydirs=0 dirwaiter=0 replicated=1 dirty=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x55842456eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v86321>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v87431>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:14.554656+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:14.554656+0000 b5201920 25=24+1)","memory_value.dirstat":"f(v3257 m2023-07-21T22:35:14.554656+0000 24=24+0)","memory_value.rstat":"n(v79 rc2023-07-21T22:35:14.554656+0000 b5201920 25=24+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.497 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.991+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PM/ auth{0=1} v87257 ap=1 f(v6845 m2023-07-21T22:35:11.044682+0000 3=3+0) n(v69 rc2023-07-21T22:35:11.044682+0000 b780288 4=3+1) old_inodes=8 (inest mix) (ifile excl) (iversion lock) caps={25342=pAsLsXsFsx/pAsLsXsFsx@44836},l=25342 | dirtyscattered=0 request=0 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55842456f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v86044>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v87257>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:11.044682+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:11.044682+0000 b780288 4=3+1)","memory_value.dirstat":"f(v6845 m2023-07-21T22:35:11.044682+0000 3=3+0)","memory_value.rstat":"n(v69 rc2023-07-21T22:35:11.044682+0000 b780288 4=3+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.498 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.992+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{0=1,1=1} v87303 pv87467 ap=2 f(v3127 m2023-07-21T22:35:12.027675+0000 7=7+0)->f(v3128 m2023-07-21T22:35:12.085674+0000 7=7+0) n(v76 rc2023-07-21T22:35:02.927742+0000 b1462582 8=7+1)->n(v76 rc2023-07-21T22:35:02.927742+0000 b1462582 8=7+1) old_inodes=7 (inest mix dirty) (ifile mix->sync w=1 flushing) (iversion lock) caps={25342=pAsLsXs/pAsLsXsFsx@14925} mcw={1=AsLsXsFs} | dirtyscattered=2 request=0 lock=1 importingcaps=0 importing=0 dirfrag=1 caps=1 stickydirs=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55842494bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v86217>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v87303>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:12.027675+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:02.927742+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v3127 m2023-07-21T22:35:12.027675+0000 7=7+0)","memory_value.rstat":"n(v76 rc2023-07-21T22:35:02.927742+0000 b1462582 8=7+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.498 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:35:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:35:20.993+0000 7fb547cbe700 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000023e [...18,head] /volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ACCESS/ auth{0=1,1=1} v87271 ap=1 f(v892 m2023-07-21T22:35:11.203681+0000 1=1+0) n(v39 rc2023-07-21T22:35:11.203681+0000 b260096 2=1+1) old_inodes=7 (inest mix) (iversion lock) caps={25342=pAsLsXsFs/pAsLsXsFsx@2059} mcw={1=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 importingcaps=0 importing=0 dirfrag=1 caps=1 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558424bc4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000023e:[<0x10000000009/ACCESS v86094>,<0x10000000008/~dmtmp v43266>,<0x10000000007/client0 v1996>,<0x10000000006/clients v1282>,<0x10000000005/tmp v1247>,<0x100000001fb/client.0 v1190>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1177>,<0x10000000001/sv_1 v1139>,<0x10000000000/_nogroup v1146>,<0x1/volumes v1118>]//[]","memoryvalue":"(2)0x1000000023e:[<0x10000000009/ACCESS v87271>,<0x10000000008/~dmtmp v43540>,<0x10000000007/client0 v2328>,<0x10000000006/clients v1358>,<0x10000000005/tmp v1302>,<0x100000001fb/client.0 v1240>,<0x100000001fa/d9a6ac6a-b684-4b83-bca0-bd809aa020ca v1171>,<0x10000000001/sv_1 v1169>,<0x10000000000/_nogroup v1120>,<0x1/volumes v1148>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T22:35:11.203681+0000 1=1+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T22:35:11.203681+0000 b260096 2=1+1)","memory_value.dirstat":"f(v892 m2023-07-21T22:35:11.203681+0000 1=1+0)","memory_value.rstat":"n(v39 rc2023-07-21T22:35:11.203681+0000 b260096 2=1+1)","error_str":""},"return_code":0} 2023-07-21T22:35:21.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 26408 33.23 MB/sec warmup 48 sec latency 5118.500 ms 2023-07-21T22:35:22.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 552 op/s 2023-07-21T22:35:22.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: scrub summary: active paths [/] 2023-07-21T22:35:22.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000009 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000215 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x1000000020c (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000208 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000240 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000221 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000229 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x10000000225 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PM) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x1000000020f (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: Scrub error on inode 0x1000000023e (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:22.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:22 smithi104 ceph-mon[129021]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:22.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 552 op/s 2023-07-21T22:35:22.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: scrub summary: active paths [/] 2023-07-21T22:35:22.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000009 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000215 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x1000000020c (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000208 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000240 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000221 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000229 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x10000000225 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PM) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x1000000020f (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: Scrub error on inode 0x1000000023e (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:22.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:22 smithi066 ceph-mon[126379]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:22.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: pgmap v435: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 552 op/s 2023-07-21T22:35:22.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: scrub summary: active paths [/] 2023-07-21T22:35:22.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000009 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000215 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/COREL) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x1000000020c (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/EXCEL) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000208 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000240 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000221 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000229 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x10000000225 (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PM) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x1000000020f (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: Scrub error on inode 0x1000000023e (/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ACCESS) see mds.i log and `damage ls` output for details 2023-07-21T22:35:22.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:22.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:22 smithi149 ceph-mon[129268]: scrub summary: idle+waiting paths [/] 2023-07-21T22:35:22.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 27293 33.74 MB/sec warmup 49 sec latency 58.386 ms 2023-07-21T22:35:23.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 27987 33.68 MB/sec warmup 50 sec latency 142.277 ms 2023-07-21T22:35:24.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:24 smithi104 ceph-mon[129021]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 553 op/s 2023-07-21T22:35:24.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:24 smithi066 ceph-mon[126379]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 553 op/s 2023-07-21T22:35:24.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:24 smithi149 ceph-mon[129268]: pgmap v436: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.8 MiB/s wr, 553 op/s 2023-07-21T22:35:24.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 28652 33.61 MB/sec warmup 51 sec latency 111.245 ms 2023-07-21T22:35:25.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 29569 33.98 MB/sec warmup 52 sec latency 262.624 ms 2023-07-21T22:35:26.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:26 smithi104 ceph-mon[129021]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 645 op/s 2023-07-21T22:35:26.458 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:26 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a[126355]: 2023-07-21T22:35:26.043+0000 7f9d86663700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:26.458 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:26 smithi066 ceph-mon[126379]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 645 op/s 2023-07-21T22:35:26.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:26 smithi149 ceph-mon[129268]: pgmap v437: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 645 op/s 2023-07-21T22:35:26.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 30464 34.28 MB/sec warmup 53 sec latency 69.308 ms 2023-07-21T22:35:27.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:27 smithi104 ceph-mon[129021]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:27.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:27 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:active 2023-07-21T22:35:27.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:27 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:35:27.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:27 smithi066 ceph-mon[126379]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:27.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:27 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:active 2023-07-21T22:35:27.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:27 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:35:27.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:27 smithi149 ceph-mon[129268]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-07-21T22:35:27.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:27 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] up:active 2023-07-21T22:35:27.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:27 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:35:27.930 INFO:tasks.workunit.client.0.smithi066.stdout: 10 31159 34.39 MB/sec warmup 54 sec latency 79.049 ms 2023-07-21T22:35:28.372 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:28 smithi104 ceph-mon[129021]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-07-21T22:35:28.452 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:28 smithi066 ceph-mon[126379]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-07-21T22:35:28.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:28 smithi149 ceph-mon[129268]: pgmap v438: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-07-21T22:35:28.931 INFO:tasks.workunit.client.0.smithi066.stdout: 10 31529 33.98 MB/sec warmup 55 sec latency 329.357 ms 2023-07-21T22:35:29.931 INFO:tasks.workunit.client.0.smithi066.stdout: 10 31792 33.52 MB/sec warmup 56 sec latency 542.095 ms 2023-07-21T22:35:30.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:30 smithi066 ceph-mon[126379]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-07-21T22:35:30.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:30 smithi149 ceph-mon[129268]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-07-21T22:35:30.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:30 smithi104 ceph-mon[129021]: pgmap v439: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.7 MiB/s wr, 502 op/s 2023-07-21T22:35:30.931 INFO:tasks.workunit.client.0.smithi066.stdout: 10 32354 33.63 MB/sec warmup 57 sec latency 185.503 ms 2023-07-21T22:35:31.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:31 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:35:31.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:31 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:35:31.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:31 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:35:31.931 INFO:tasks.workunit.client.0.smithi066.stdout: 10 33486 34.02 MB/sec warmup 58 sec latency 47.137 ms 2023-07-21T22:35:32.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:32 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:35:32.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:32 smithi066 ceph-mon[126379]: scrub summary: idle 2023-07-21T22:35:32.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:32 smithi066 ceph-mon[126379]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 16 MiB/s wr, 812 op/s 2023-07-21T22:35:32.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:32 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:35:32.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:32 smithi149 ceph-mon[129268]: scrub summary: idle 2023-07-21T22:35:32.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:32 smithi149 ceph-mon[129268]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 16 MiB/s wr, 812 op/s 2023-07-21T22:35:32.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:32 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:35:32.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:32 smithi104 ceph-mon[129021]: scrub summary: idle 2023-07-21T22:35:32.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:32 smithi104 ceph-mon[129021]: pgmap v440: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 16 MiB/s wr, 812 op/s 2023-07-21T22:35:32.932 INFO:tasks.workunit.client.0.smithi066.stdout: 10 34401 34.50 MB/sec warmup 59 sec latency 41.586 ms 2023-07-21T22:35:33.931 INFO:tasks.workunit.client.0.smithi066.stdout: 10 34896 34.26 MB/sec warmup 60 sec latency 196.012 ms 2023-07-21T22:35:34.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:34 smithi066 ceph-mon[126379]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 556 op/s 2023-07-21T22:35:34.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:34 smithi149 ceph-mon[129268]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 556 op/s 2023-07-21T22:35:34.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:34 smithi104 ceph-mon[129021]: pgmap v441: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 556 op/s 2023-07-21T22:35:34.934 INFO:tasks.workunit.client.0.smithi066.stdout: 10 35126 33.93 MB/sec warmup 61 sec latency 741.234 ms 2023-07-21T22:35:35.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:35 smithi149 ceph-mon[129268]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 21 MiB/s wr, 862 op/s 2023-07-21T22:35:35.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:35 smithi104 ceph-mon[129021]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 21 MiB/s wr, 862 op/s 2023-07-21T22:35:35.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:35 smithi066 ceph-mon[126379]: pgmap v442: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 21 MiB/s wr, 862 op/s 2023-07-21T22:35:35.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 35256 33.49 MB/sec warmup 62 sec latency 1233.935 ms 2023-07-21T22:35:36.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 35642 33.16 MB/sec warmup 63 sec latency 154.581 ms 2023-07-21T22:35:37.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 36220 33.17 MB/sec warmup 64 sec latency 77.942 ms 2023-07-21T22:35:38.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:38 smithi104 ceph-mon[129021]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 757 op/s 2023-07-21T22:35:38.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:38 smithi066 ceph-mon[126379]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 757 op/s 2023-07-21T22:35:38.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:38 smithi149 ceph-mon[129268]: pgmap v443: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 20 MiB/s wr, 757 op/s 2023-07-21T22:35:38.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 37140 33.49 MB/sec warmup 65 sec latency 59.279 ms 2023-07-21T22:35:39.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 38020 33.63 MB/sec warmup 66 sec latency 42.644 ms 2023-07-21T22:35:40.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:40 smithi104 ceph-mon[129021]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 693 op/s 2023-07-21T22:35:40.450 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:40 smithi066 ceph-mon[126379]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 693 op/s 2023-07-21T22:35:40.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:40 smithi149 ceph-mon[129268]: pgmap v444: 129 pgs: 129 active+clean; 1.2 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 19 MiB/s wr, 693 op/s 2023-07-21T22:35:40.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 38873 33.87 MB/sec warmup 67 sec latency 62.998 ms 2023-07-21T22:35:41.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 39623 34.04 MB/sec warmup 68 sec latency 1054.443 ms 2023-07-21T22:35:42.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:42 smithi104 ceph-mon[129021]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.07k op/s 2023-07-21T22:35:42.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:42 smithi066 ceph-mon[126379]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.07k op/s 2023-07-21T22:35:42.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:42 smithi149 ceph-mon[129268]: pgmap v445: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 24 MiB/s wr, 1.07k op/s 2023-07-21T22:35:42.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 40342 33.99 MB/sec warmup 69 sec latency 1876.394 ms 2023-07-21T22:35:43.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 41367 34.35 MB/sec warmup 70 sec latency 67.382 ms 2023-07-21T22:35:44.371 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:44 smithi104 ceph-mon[129021]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 757 op/s 2023-07-21T22:35:44.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:44 smithi066 ceph-mon[126379]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 757 op/s 2023-07-21T22:35:44.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:44 smithi149 ceph-mon[129268]: pgmap v446: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 15 MiB/s wr, 757 op/s 2023-07-21T22:35:44.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 42201 34.45 MB/sec warmup 71 sec latency 53.357 ms 2023-07-21T22:35:45.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 43027 34.55 MB/sec warmup 72 sec latency 37.300 ms 2023-07-21T22:35:46.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:46 smithi104 ceph-mon[129021]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 916 op/s 2023-07-21T22:35:46.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:46 smithi066 ceph-mon[126379]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 916 op/s 2023-07-21T22:35:46.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:46 smithi149 ceph-mon[129268]: pgmap v447: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 18 MiB/s wr, 916 op/s 2023-07-21T22:35:46.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 43897 34.76 MB/sec warmup 73 sec latency 57.327 ms 2023-07-21T22:35:46.952 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T22:35:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.228+0000 7f7082fff700 1 -- 172.21.15.66:0/1774980847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c1004b0 msgr2=0x7f707c10c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.228+0000 7f7082fff700 1 --2- 172.21.15.66:0/1774980847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c1004b0 0x7f707c10c940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7074009990 tx=0x7f7074009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.229+0000 7f7082fff700 1 -- 172.21.15.66:0/1774980847 shutdown_connections 2023-07-21T22:35:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.229+0000 7f7082fff700 1 --2- 172.21.15.66:0/1774980847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c1004b0 0x7f707c10c940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.229+0000 7f7082fff700 1 --2- 172.21.15.66:0/1774980847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c105b10 0x7f707c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.229+0000 7f7082fff700 1 --2- 172.21.15.66:0/1774980847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c0ffb00 0x7f707c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.229+0000 7f7082fff700 1 -- 172.21.15.66:0/1774980847 >> 172.21.15.66:0/1774980847 conn(0x7f707c0fb600 msgr2=0x7f707c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.231+0000 7f7082fff700 1 -- 172.21.15.66:0/1774980847 shutdown_connections 2023-07-21T22:35:47.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.231+0000 7f7082fff700 1 -- 172.21.15.66:0/1774980847 wait complete. 2023-07-21T22:35:47.278 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.232+0000 7f7082fff700 1 Processor -- start 2023-07-21T22:35:47.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.233+0000 7f7082fff700 1 -- start start 2023-07-21T22:35:47.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.233+0000 7f7082fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 0x7f707c10e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.233+0000 7f7082fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c1004b0 0x7f707c10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.280 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7082fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c105b10 0x7f707c10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7082fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1b0cd0 con 0x7f707c105b10 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7082fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1a83f0 con 0x7f707c0ffb00 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7082fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c1a85b0 con 0x7f707c1004b0 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 0x7f707c10e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 0x7f707c10e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41376/0 (socket says 172.21.15.66:41376) 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 -- 172.21.15.66:0/4269261572 learned_addr learned my addr 172.21.15.66:0/4269261572 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:47.281 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f708159c700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c105b10 0x7f707c10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.282 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7073fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c1004b0 0x7f707c10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 -- 172.21.15.66:0/4269261572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c1004b0 msgr2=0x7f707c10e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c1004b0 0x7f707c10e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 -- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c105b10 msgr2=0x7f707c10ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c105b10 0x7f707c10ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.234+0000 7f7080d9b700 1 -- 172.21.15.66:0/4269261572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70740095f0 con 0x7f707c0ffb00 2023-07-21T22:35:47.283 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7073fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c1004b0 0x7f707c10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T22:35:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7080d9b700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 0x7f707c10e180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f707800b840 tx=0x7f707800bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7071ffb700 1 -- 172.21.15.66:0/4269261572 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078010860 con 0x7f707c0ffb00 2023-07-21T22:35:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7071ffb700 1 -- 172.21.15.66:0/4269261572 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7078010e80 con 0x7f707c0ffb00 2023-07-21T22:35:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7071ffb700 1 -- 172.21.15.66:0/4269261572 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078007370 con 0x7f707c0ffb00 2023-07-21T22:35:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707c1a8890 con 0x7f707c0ffb00 2023-07-21T22:35:47.284 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.235+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c1a8d50 con 0x7f707c0ffb00 2023-07-21T22:35:47.287 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.241+0000 7f7071ffb700 1 -- 172.21.15.66:0/4269261572 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f707800f5d0 con 0x7f707c0ffb00 2023-07-21T22:35:47.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.242+0000 7f7096944b80 1 Processor -- start 2023-07-21T22:35:47.288 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.242+0000 7f7096944b80 1 -- start start 2023-07-21T22:35:47.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7096944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e726580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7096944b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 0x55927e721350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.289 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7096944b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e721da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7096944b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e626cb0 con 0x55927e721930 2023-07-21T22:35:47.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7096944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e679a40 con 0x55927e7261b0 2023-07-21T22:35:47.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7096944b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e672ed0 con 0x55927e726ac0 2023-07-21T22:35:47.290 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7080d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 0x55927e721350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7080d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 0x55927e721350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47862/0 (socket says 172.21.15.66:47862) 2023-07-21T22:35:47.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f7080d9b700 1 -- 172.21.15.66:0/567743532 learned_addr learned my addr 172.21.15.66:0/567743532 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:47.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.243+0000 7f708159c700 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e726580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7073fff700 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e721da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7080d9b700 1 -- 172.21.15.66:0/567743532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 msgr2=0x55927e726580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.291 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7080d9b700 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e726580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7080d9b700 1 -- 172.21.15.66:0/567743532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 msgr2=0x55927e721da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7080d9b700 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e721da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7080d9b700 1 -- 172.21.15.66:0/567743532 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55927e4a0de0 con 0x55927e726ac0 2023-07-21T22:35:47.292 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.244+0000 7f7071ffb700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70640a1300 0x7f70640a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.245+0000 7f7071ffb700 1 -- 172.21.15.66:0/4269261572 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f70780cce90 con 0x7f707c0ffb00 2023-07-21T22:35:47.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.245+0000 7f7080d9b700 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 0x55927e721350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70780ca1b0 tx=0x7f70780d53c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d7ca7422fa17bc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.245+0000 7f708159c700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70640a1300 0x7f70640a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.293 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.245+0000 7f7062ffd700 1 -- 172.21.15.66:0/567743532 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078095f00 con 0x55927e726ac0 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.246+0000 7f7062ffd700 1 -- 172.21.15.66:0/567743532 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7078099990 con 0x55927e726ac0 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.246+0000 7f7062ffd700 1 -- 172.21.15.66:0/567743532 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078099520 con 0x55927e726ac0 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.246+0000 7f7096944b80 1 -- 172.21.15.66:0/567743532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 msgr2=0x55927e721350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.246+0000 7f7096944b80 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 0x55927e721350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f70780ca1b0 tx=0x7f70780d53c0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.246+0000 7f708159c700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70640a1300 0x7f70640a37b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f70740098e0 tx=0x7f707400b280 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.247+0000 7f7096944b80 1 -- 172.21.15.66:0/567743532 shutdown_connections 2023-07-21T22:35:47.294 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.247+0000 7f7096944b80 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e726580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.247+0000 7f7096944b80 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e721da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.247+0000 7f7096944b80 1 --2- 172.21.15.66:0/567743532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e726ac0 0x55927e721350 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.295 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.247+0000 7f7096944b80 1 -- 172.21.15.66:0/567743532 >> 172.21.15.66:0/567743532 conn(0x55927e716ef0 msgr2=0x55927e71c830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.247+0000 7f7096944b80 1 -- 172.21.15.66:0/567743532 shutdown_connections 2023-07-21T22:35:47.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.248+0000 7f7096944b80 1 -- 172.21.15.66:0/567743532 wait complete. 2023-07-21T22:35:47.297 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.248+0000 7f7096944b80 1 -- start start 2023-07-21T22:35:47.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 Processor -- start 2023-07-21T22:35:47.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.298 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 0x55927e72ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e72b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e612ac0 con 0x55927e7261b0 2023-07-21T22:35:47.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e6127c0 con 0x55927e726ac0 2023-07-21T22:35:47.299 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e4f6920 con 0x55927e721930 2023-07-21T22:35:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.249+0000 7f7096944b80 10 client.0 _ll_register_callbacks cb 0x55927e610760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.250+0000 7f7073fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 0x55927e72ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.300 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.250+0000 7f7073fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 0x55927e72ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46038/0 (socket says 172.21.15.66:46038) 2023-07-21T22:35:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.250+0000 7f7073fff700 1 -- 172.21.15.66:0/1380788075 learned_addr learned my addr 172.21.15.66:0/1380788075 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.250+0000 7f7080d9b700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.250+0000 7f708159c700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e72b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.301 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7080d9b700 1 -- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 msgr2=0x55927e72b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.303 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7080d9b700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e72b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7080d9b700 1 -- 172.21.15.66:0/1380788075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 msgr2=0x55927e72ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7080d9b700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 0x55927e72ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7080d9b700 1 -- 172.21.15.66:0/1380788075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707c1a8d50 con 0x55927e721930 2023-07-21T22:35:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7073fff700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 0x55927e72ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:47.304 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.251+0000 7f7080d9b700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72db90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7078099870 tx=0x7f7078004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc7d4e05db5de60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f705bfff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:47.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f705bfff700 1 -- 172.21.15.66:0/1380788075 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70780b1070 con 0x55927e721930 2023-07-21T22:35:47.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f705bfff700 1 -- 172.21.15.66:0/1380788075 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7078099bc0 con 0x55927e721930 2023-07-21T22:35:47.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55927e5cc4f0 con 0x55927e721930 2023-07-21T22:35:47.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f705bfff700 1 -- 172.21.15.66:0/1380788075 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7078099700 con 0x55927e721930 2023-07-21T22:35:47.305 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f7096944b80 10 client.27532 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55927e489c60 con 0x55927e721930 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f705bfff700 1 -- 172.21.15.66:0/1380788075 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55927e5cc4f0 con 0x55927e721930 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.252+0000 7f705bfff700 1 -- 172.21.15.66:0/1380788075 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f70780b2220 con 0x55927e721930 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7096944b80 10 client.27532 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7096944b80 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7096944b80 20 client.27532 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:47.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7096944b80 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e73af90 0x55927e73d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7096944b80 4 client.27532 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55927e66beb0 con 0x55927e73af90 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.253+0000 7f7073fff700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e73af90 0x55927e73d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.254+0000 7f7073fff700 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e73af90 0x55927e73d370 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c6c575ee83dfd4 server_cookie=ea4cf4449b416bc5 in_seq=0 out_seq=0 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.254+0000 7f705bfff700 10 client.27532 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.256+0000 7f705bfff700 1 -- 172.21.15.66:0/1380788075 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55927e66beb0 con 0x55927e73af90 2023-07-21T22:35:47.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.256+0000 7f705bfff700 10 client.27532 handle_command_reply: tid=0 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.256+0000 7f7096944b80 1 client.27532 shutdown 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.256+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 msgr2=0x55927e72db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.256+0000 7f7096944b80 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72db90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7078099870 tx=0x7f7078004b60 comp rx=0 tx=0).stop 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 shutdown_connections 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e72b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e73af90 0x55927e73d370 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72db90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 --2- 172.21.15.66:0/1380788075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e7261b0 0x55927e72ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 >> 172.21.15.66:0/1380788075 conn(0x55927e716ef0 msgr2=0x55927e71c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f705bfff700 0 client.27532 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 shutdown_connections 2023-07-21T22:35:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 1 -- 172.21.15.66:0/1380788075 wait complete. 2023-07-21T22:35:47.309 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.257+0000 7f7096944b80 20 client.27532 trim_cache size 0 max 16384 2023-07-21T22:35:47.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.289+0000 7f7096944b80 1 Processor -- start 2023-07-21T22:35:47.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.289+0000 7f7096944b80 1 -- start start 2023-07-21T22:35:47.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7096944b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7096944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e73cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7096944b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e726ac0 0x55927e73d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7096944b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e719010 con 0x55927e726ac0 2023-07-21T22:35:47.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7096944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e714050 con 0x55927e7261b0 2023-07-21T22:35:47.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7096944b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e72bda0 con 0x55927e721930 2023-07-21T22:35:47.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7080d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7073fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e73cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7080d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47882/0 (socket says 172.21.15.66:47882) 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7080d9b700 1 -- 172.21.15.66:0/1535365339 learned_addr learned my addr 172.21.15.66:0/1535365339 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7073fff700 1 -- 172.21.15.66:0/1535365339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 msgr2=0x55927e72d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7073fff700 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7073fff700 1 -- 172.21.15.66:0/1535365339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e726ac0 msgr2=0x55927e73d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.338 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7073fff700 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e726ac0 0x55927e73d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.290+0000 7f7073fff700 1 -- 172.21.15.66:0/1535365339 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55927e741ec0 con 0x55927e7261b0 2023-07-21T22:35:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.291+0000 7f7073fff700 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e73cfa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55927e73dba0 tx=0x7f706c01ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ab5d894866557f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.291+0000 7f70637fe700 1 -- 172.21.15.66:0/1535365339 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c017040 con 0x55927e7261b0 2023-07-21T22:35:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.291+0000 7f70637fe700 1 -- 172.21.15.66:0/1535365339 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f706c019040 con 0x55927e7261b0 2023-07-21T22:35:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.291+0000 7f70637fe700 1 -- 172.21.15.66:0/1535365339 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c026900 con 0x55927e7261b0 2023-07-21T22:35:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 -- 172.21.15.66:0/1535365339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 msgr2=0x55927e73cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e73cfa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x55927e73dba0 tx=0x7f706c01ec40 comp rx=0 tx=0).stop 2023-07-21T22:35:47.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 -- 172.21.15.66:0/1535365339 shutdown_connections 2023-07-21T22:35:47.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e726ac0 0x55927e73d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e721930 0x55927e72d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 --2- 172.21.15.66:0/1535365339 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e7261b0 0x55927e73cfa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.292+0000 7f7096944b80 1 -- 172.21.15.66:0/1535365339 >> 172.21.15.66:0/1535365339 conn(0x55927e716ef0 msgr2=0x55927e71fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.293+0000 7f7096944b80 1 -- 172.21.15.66:0/1535365339 shutdown_connections 2023-07-21T22:35:47.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.293+0000 7f7096944b80 1 -- 172.21.15.66:0/1535365339 wait complete. 2023-07-21T22:35:47.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.293+0000 7f7096944b80 1 -- start start 2023-07-21T22:35:47.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.293+0000 7f7096944b80 1 Processor -- start 2023-07-21T22:35:47.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e730750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 0x55927e730c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e78dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e78e280 con 0x55927e721930 2023-07-21T22:35:47.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e78e3c0 con 0x55927e726ac0 2023-07-21T22:35:47.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55927e78e500 con 0x55927e7261b0 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7096944b80 10 client.0 _ll_register_callbacks cb 0x7f706c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7073fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 0x55927e730c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7073fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 0x55927e730c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47890/0 (socket says 172.21.15.66:47890) 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.295+0000 7f7073fff700 1 -- 172.21.15.66:0/184512051 learned_addr learned my addr 172.21.15.66:0/184512051 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f708159c700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e78dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f7080d9b700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e730750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f7073fff700 1 -- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 msgr2=0x55927e78dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f7073fff700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e78dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f7073fff700 1 -- 172.21.15.66:0/184512051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 msgr2=0x55927e730750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:47.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f7073fff700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e730750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.296+0000 7f7073fff700 1 -- 172.21.15.66:0/184512051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c009d60 con 0x55927e7261b0 2023-07-21T22:35:47.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.297+0000 7f7073fff700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 0x55927e730c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f706c005920 tx=0x7f706c01e570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72dc7009010ebbaa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.297+0000 7f705bfff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f705bfff700 1 -- 172.21.15.66:0/184512051 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c017050 con 0x55927e7261b0 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f705bfff700 1 -- 172.21.15.66:0/184512051 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f706c004300 con 0x55927e7261b0 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7078099990 con 0x55927e7261b0 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f705bfff700 1 -- 172.21.15.66:0/184512051 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f706c0044c0 con 0x55927e7261b0 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f7096944b80 10 client.27538 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55927e4c7250 con 0x55927e7261b0 2023-07-21T22:35:47.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.298+0000 7f705bfff700 1 -- 172.21.15.66:0/184512051 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7078099990 con 0x55927e7261b0 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f705bfff700 1 -- 172.21.15.66:0/184512051 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f706c0139c0 con 0x55927e7261b0 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7096944b80 10 client.27538 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7096944b80 10 client.27538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7096944b80 20 client.27538 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7096944b80 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e78fbe0 0x55927e791ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7096944b80 4 client.27538 mds_command: new command op to 24572 tid=0[{"prefix": "scrub status"}] 2023-07-21T22:35:47.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55927e62e050 con 0x55927e78fbe0 2023-07-21T22:35:47.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.299+0000 7f7080d9b700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e78fbe0 0x55927e791ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.300+0000 7f7080d9b700 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e78fbe0 0x55927e791ff0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a1be9dc34f1050 server_cookie=6ce6f25aa8c8159c in_seq=0 out_seq=0 2023-07-21T22:35:47.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.300+0000 7f705bfff700 10 client.27538 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:47.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.301+0000 7f705bfff700 1 -- 172.21.15.66:0/184512051 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55927e62e050 con 0x55927e78fbe0 2023-07-21T22:35:47.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.301+0000 7f705bfff700 10 client.27538 handle_command_reply: tid=0 2023-07-21T22:35:47.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.302+0000 7f7096944b80 1 client.27538 shutdown 2023-07-21T22:35:47.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 msgr2=0x55927e730c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 0x55927e730c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f706c005920 tx=0x7f706c01e570 comp rx=0 tx=0).stop 2023-07-21T22:35:47.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 shutdown_connections 2023-07-21T22:35:47.355 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55927e726ac0 0x55927e78dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55927e78fbe0 0x55927e791ff0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55927e721930 0x55927e730750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 --2- 172.21.15.66:0/184512051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55927e7261b0 0x55927e730c90 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f706c005920 tx=0x7f706c01e570 comp rx=0 tx=0).stop 2023-07-21T22:35:47.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.309+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 >> 172.21.15.66:0/184512051 conn(0x55927e716ef0 msgr2=0x55927e71fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.356 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.310+0000 7f705bfff700 0 client.27538 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:47.357 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.311+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 shutdown_connections 2023-07-21T22:35:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.318+0000 7f7096944b80 1 -- 172.21.15.66:0/184512051 wait complete. 2023-07-21T22:35:47.364 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.318+0000 7f7096944b80 20 client.27538 trim_cache size 0 max 16384 2023-07-21T22:35:47.365 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:35:47.365 INFO:teuthology.orchestra.run.smithi066.stdout: "status": "no active scrubs running", 2023-07-21T22:35:47.365 INFO:teuthology.orchestra.run.smithi066.stdout: "scrubs": {} 2023-07-21T22:35:47.365 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:35:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.320+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70640a1300 msgr2=0x7f70640a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.320+0000 7f7082fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70640a1300 0x7f70640a37b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f70740098e0 tx=0x7f707400b280 comp rx=0 tx=0).stop 2023-07-21T22:35:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.320+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 msgr2=0x7f707c10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.367 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.320+0000 7f7082fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 0x7f707c10e180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f707800b840 tx=0x7f707800bb50 comp rx=0 tx=0).stop 2023-07-21T22:35:47.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 shutdown_connections 2023-07-21T22:35:47.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f707c0ffb00 0x7f707c10e180 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.368 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f70640a1300 0x7f70640a37b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f707c105b10 0x7f707c10ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 --2- 172.21.15.66:0/4269261572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f707c1004b0 0x7f707c10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 >> 172.21.15.66:0/4269261572 conn(0x7f707c0fb600 msgr2=0x7f707c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 shutdown_connections 2023-07-21T22:35:47.369 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.322+0000 7f7082fff700 1 -- 172.21.15.66:0/4269261572 wait complete. 2023-07-21T22:35:47.419 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T22:35:47.420 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:35:47.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 -- 172.21.15.66:0/2970363990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 msgr2=0x7fc7c810bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/2970363990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c810bb00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc009990 tx=0x7fc7bc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 -- 172.21.15.66:0/2970363990 shutdown_connections 2023-07-21T22:35:47.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/2970363990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c80678e0 0x7fc7c8067cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/2970363990 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c810c170 0x7fc7c810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/2970363990 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c810bb00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.693+0000 7fc7cd39d700 1 -- 172.21.15.66:0/2970363990 >> 172.21.15.66:0/2970363990 conn(0x7fc7c8074a30 msgr2=0x7fc7c8074e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:47.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.694+0000 7fc7cd39d700 1 -- 172.21.15.66:0/2970363990 shutdown_connections 2023-07-21T22:35:47.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.694+0000 7fc7cd39d700 1 -- 172.21.15.66:0/2970363990 wait complete. 2023-07-21T22:35:47.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.695+0000 7fc7cd39d700 1 Processor -- start 2023-07-21T22:35:47.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.695+0000 7fc7cd39d700 1 -- start start 2023-07-21T22:35:47.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7cd39d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80678e0 0x7fc7c806a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7cd39d700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c806ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7cd39d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c810c170 0x7fc7c819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7cd39d700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c806b480 con 0x7fc7c80678e0 2023-07-21T22:35:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7cd39d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c819b6f0 con 0x7fc7c810c170 2023-07-21T22:35:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7cd39d700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c819b860 con 0x7fc7c8068280 2023-07-21T22:35:47.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c806ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7ccb9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c810c170 0x7fc7c819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7c77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c806ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47900/0 (socket says 172.21.15.66:47900) 2023-07-21T22:35:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7c77fe700 1 -- 172.21.15.66:0/786826465 learned_addr learned my addr 172.21.15.66:0/786826465 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:47.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.696+0000 7fc7c7fff700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80678e0 0x7fc7c806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.697+0000 7fc7c77fe700 1 -- 172.21.15.66:0/786826465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c810c170 msgr2=0x7fc7c819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.697+0000 7fc7c77fe700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c810c170 0x7fc7c819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.697+0000 7fc7c77fe700 1 -- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80678e0 msgr2=0x7fc7c806a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.697+0000 7fc7c77fe700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80678e0 0x7fc7c806a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:47.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.697+0000 7fc7c77fe700 1 -- 172.21.15.66:0/786826465 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0095f0 con 0x7fc7c8068280 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.697+0000 7fc7c7fff700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80678e0 0x7fc7c806a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.698+0000 7fc7c77fe700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c806ab70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00b5c0 tx=0x7fc7bc00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.698+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc01d070 con 0x7fc7c8068280 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.698+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7bc00fb60 con 0x7fc7c8068280 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.698+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc017650 con 0x7fc7c8068280 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.698+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c819bae0 con 0x7fc7c8068280 2023-07-21T22:35:47.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.698+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c819bf40 con 0x7fc7c8068280 2023-07-21T22:35:47.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.700+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8067220 con 0x7fc7c8068280 2023-07-21T22:35:47.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.702+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc7bc00fcd0 con 0x7fc7c8068280 2023-07-21T22:35:47.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.703+0000 7fc7c57fa700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc7b00a1430 0x7fc7b00a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:47.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.703+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc7bc0ce700 con 0x7fc7c8068280 2023-07-21T22:35:47.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.704+0000 7fc7c7fff700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc7b00a1430 0x7fc7b00a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:47.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.704+0000 7fc7c7fff700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc7b00a1430 0x7fc7b00a38e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005fd0 tx=0x7fc7b8005e30 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:47.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:47.705+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc7bc0977a0 con 0x7fc7c8068280 2023-07-21T22:35:47.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 44719 34.92 MB/sec warmup 74 sec latency 61.082 ms 2023-07-21T22:35:48.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.073+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7c806bb60 con 0x7fc7c8068280 2023-07-21T22:35:48.119 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.074+0000 7fc7c57fa700 1 -- 172.21.15.66:0/786826465 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+10830 (secure 0 0 0) 0x7fc7bc09adc0 con 0x7fc7c8068280 2023-07-21T22:35:48.120 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:35:48.120 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:35:26.045220+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":250,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:35:48.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc7b00a1430 msgr2=0x7fc7b00a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc7b00a1430 0x7fc7b00a38e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005fd0 tx=0x7fc7b8005e30 comp rx=0 tx=0).stop 2023-07-21T22:35:48.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 msgr2=0x7fc7c806ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.123 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c806ab70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00b5c0 tx=0x7fc7bc00f670 comp rx=0 tx=0).stop 2023-07-21T22:35:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 shutdown_connections 2023-07-21T22:35:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7c810c170 0x7fc7c819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc7b00a1430 0x7fc7b00a38e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc7c80678e0 0x7fc7c806a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 --2- 172.21.15.66:0/786826465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc7c8068280 0x7fc7c806ab70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.124 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 >> 172.21.15.66:0/786826465 conn(0x7fc7c8074a30 msgr2=0x7fc7c80fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 shutdown_connections 2023-07-21T22:35:48.125 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.077+0000 7fc7cd39d700 1 -- 172.21.15.66:0/786826465 wait complete. 2023-07-21T22:35:48.126 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 50 2023-07-21T22:35:48.152 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T22:35:48.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:48 smithi104 ceph-mon[129021]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 687 op/s 2023-07-21T22:35:48.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:48 smithi066 ceph-mon[126379]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 687 op/s 2023-07-21T22:35:48.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.411+0000 7f6bee1ff700 1 -- 172.21.15.66:0/4051585841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 msgr2=0x7f6be8112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.411+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/4051585841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 0x7f6be8112f30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6be000d210 tx=0x7f6be000d520 comp rx=0 tx=0).stop 2023-07-21T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.413+0000 7f6bee1ff700 1 -- 172.21.15.66:0/4051585841 shutdown_connections 2023-07-21T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.413+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/4051585841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 0x7f6be8112f30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.413+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/4051585841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be8107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.413+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/4051585841 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be8108090 0x7f6be8110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.413+0000 7f6bee1ff700 1 -- 172.21.15.66:0/4051585841 >> 172.21.15.66:0/4051585841 conn(0x7f6be806c200 msgr2=0x7f6be806c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.414+0000 7f6bee1ff700 1 -- 172.21.15.66:0/4051585841 shutdown_connections 2023-07-21T22:35:48.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.414+0000 7f6bee1ff700 1 -- 172.21.15.66:0/4051585841 wait complete. 2023-07-21T22:35:48.461 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.415+0000 7f6bee1ff700 1 Processor -- start 2023-07-21T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 -- start start 2023-07-21T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be81aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be8108090 0x7f6be81aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 0x7f6be810f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be810fc20 con 0x7f6be8108090 2023-07-21T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be810fd90 con 0x7f6be8110b50 2023-07-21T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.416+0000 7f6bee1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be810ff50 con 0x7f6be8107780 2023-07-21T22:35:48.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.417+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be81aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.417+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be81aa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47912/0 (socket says 172.21.15.66:47912) 2023-07-21T22:35:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.417+0000 7f6be77fe700 1 -- 172.21.15.66:0/3795072572 learned_addr learned my addr 172.21.15.66:0/3795072572 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.418+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be8108090 0x7f6be81aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.418+0000 7f6be77fe700 1 -- 172.21.15.66:0/3795072572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 msgr2=0x7f6be810f4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:35:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.418+0000 7f6be77fe700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 0x7f6be810f4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.418+0000 7f6be77fe700 1 -- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be8108090 msgr2=0x7f6be81aae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.418+0000 7f6be77fe700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be8108090 0x7f6be81aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.418+0000 7f6be77fe700 1 -- 172.21.15.66:0/3795072572 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be000be50 con 0x7f6be8107780 2023-07-21T22:35:48.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.419+0000 7f6be77fe700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be81aa940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80078d0 tx=0x7f6bd8007c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.419+0000 7f6be4ff9700 1 -- 172.21.15.66:0/3795072572 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800f040 con 0x7f6be8107780 2023-07-21T22:35:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.419+0000 7f6be4ff9700 1 -- 172.21.15.66:0/3795072572 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bd800de80 con 0x7f6be8107780 2023-07-21T22:35:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.419+0000 7f6be4ff9700 1 -- 172.21.15.66:0/3795072572 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd8013e30 con 0x7f6be8107780 2023-07-21T22:35:48.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.420+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be8110230 con 0x7f6be8107780 2023-07-21T22:35:48.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.420+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be81b18a0 con 0x7f6be8107780 2023-07-21T22:35:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.428+0000 7f6be4ff9700 1 -- 172.21.15.66:0/3795072572 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6bd800d9c0 con 0x7f6be8107780 2023-07-21T22:35:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.429+0000 7f6c01ba2b80 1 Processor -- start 2023-07-21T22:35:48.475 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.429+0000 7f6c01ba2b80 1 -- start start 2023-07-21T22:35:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.429+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bcd21b0 0x56167bcd2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.429+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bccd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.430+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bccd930 0x56167bccdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.430+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bbd2cb0 con 0x56167bccd930 2023-07-21T22:35:48.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.430+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bc25a40 con 0x56167bcd2ac0 2023-07-21T22:35:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.430+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bc1eed0 con 0x56167bcd21b0 2023-07-21T22:35:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bccd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bccd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41472/0 (socket says 172.21.15.66:41472) 2023-07-21T22:35:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 -- 172.21.15.66:0/2431076390 learned_addr learned my addr 172.21.15.66:0/2431076390 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be4ff9700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6bd00a14d0 0x7f6bd00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bccd930 0x56167bccdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.478 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 -- 172.21.15.66:0/2431076390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bcd21b0 msgr2=0x56167bcd2580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T22:35:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bcd21b0 0x56167bcd2580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 -- 172.21.15.66:0/2431076390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bccd930 msgr2=0x56167bccdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bccd930 0x56167bccdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.431+0000 7f6be77fe700 1 -- 172.21.15.66:0/2431076390 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56167ba4cde0 con 0x56167bcd2ac0 2023-07-21T22:35:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.432+0000 7f6be77fe700 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bccd350 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8095e00 tx=0x7f6bd801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3acd66f21b0e9937 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.432+0000 7f6be4ff9700 1 -- 172.21.15.66:0/3795072572 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6bd80cc630 con 0x7f6be8107780 2023-07-21T22:35:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.433+0000 7f6be7fff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6bd00a14d0 0x7f6bd00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.434+0000 7f6bcdffb700 1 -- 172.21.15.66:0/2431076390 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd801dec0 con 0x56167bcd2ac0 2023-07-21T22:35:48.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.434+0000 7f6bcdffb700 1 -- 172.21.15.66:0/2431076390 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bd8099390 con 0x56167bcd2ac0 2023-07-21T22:35:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.434+0000 7f6bcdffb700 1 -- 172.21.15.66:0/2431076390 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd809d170 con 0x56167bcd2ac0 2023-07-21T22:35:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.435+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2431076390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 msgr2=0x56167bccd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.435+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bccd350 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8095e00 tx=0x7f6bd801f000 comp rx=0 tx=0).stop 2023-07-21T22:35:48.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.435+0000 7f6be7fff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6bd00a14d0 0x7f6bd00a3980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6be000d9a0 tx=0x7f6be000aed0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.436+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2431076390 shutdown_connections 2023-07-21T22:35:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.436+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bccd930 0x56167bccdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.436+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bcd21b0 0x56167bcd2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.436+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2431076390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bccd350 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.436+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2431076390 >> 172.21.15.66:0/2431076390 conn(0x56167bcc2ef0 msgr2=0x56167bcc8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.483 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.436+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2431076390 shutdown_connections 2023-07-21T22:35:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.438+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2431076390 wait complete. 2023-07-21T22:35:48.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.438+0000 7f6c01ba2b80 1 -- start start 2023-07-21T22:35:48.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.439+0000 7f6c01ba2b80 1 Processor -- start 2023-07-21T22:35:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.490 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bbbeac0 con 0x56167bcd2ac0 2023-07-21T22:35:48.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bbbe7c0 con 0x56167bcd21b0 2023-07-21T22:35:48.491 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167baa2920 con 0x56167bccd930 2023-07-21T22:35:48.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6c01ba2b80 10 client.0 _ll_register_callbacks cb 0x56167bbbc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:48.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.492 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.441+0000 7f6be6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41480/0 (socket says 172.21.15.66:41480) 2023-07-21T22:35:48.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47920/0 (socket says 172.21.15.66:47920) 2023-07-21T22:35:48.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be6ffd700 1 -- 172.21.15.66:0/3607498282 learned_addr learned my addr 172.21.15.66:0/3607498282 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 -- 172.21.15.66:0/3607498282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 msgr2=0x56167bcd6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 -- 172.21.15.66:0/3607498282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 msgr2=0x56167bcd7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 -- 172.21.15.66:0/3607498282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be81b18a0 con 0x56167bccd930 2023-07-21T22:35:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6be77fe700 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd9b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x56167bcce5b0 tx=0x7f6bd8099000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d276e551adc04c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.442+0000 7f6bc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:48.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/3607498282 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd809d3c0 con 0x56167bccd930 2023-07-21T22:35:48.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/3607498282 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6bd809d170 con 0x56167bccd930 2023-07-21T22:35:48.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56167bb784f0 con 0x56167bccd930 2023-07-21T22:35:48.496 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/3607498282 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bd80a1750 con 0x56167bccd930 2023-07-21T22:35:48.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6c01ba2b80 10 client.27553 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:48.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56167ba35c60 con 0x56167bccd930 2023-07-21T22:35:48.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.443+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/3607498282 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56167bb784f0 con 0x56167bccd930 2023-07-21T22:35:48.497 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/3607498282 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6bd80c9c00 con 0x56167bccd930 2023-07-21T22:35:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6c01ba2b80 10 client.27553 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6c01ba2b80 10 client.27553 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:35:48.498 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6c01ba2b80 20 client.27553 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bce6e90 0x56167bce92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6c01ba2b80 4 client.27553 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:48.499 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56167bc17eb0 con 0x56167bce6e90 2023-07-21T22:35:48.499 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:48 smithi149 ceph-mon[129268]: pgmap v448: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 687 op/s 2023-07-21T22:35:48.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.444+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bce6e90 0x56167bce92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.445+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bce6e90 0x56167bce92a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9de5592401223d1 server_cookie=c1f2858baa745441 in_seq=0 out_seq=0 2023-07-21T22:35:48.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.445+0000 7f6bc6ffd700 10 client.27553 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:48.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.446+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/3607498282 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56167bc17eb0 con 0x56167bce6e90 2023-07-21T22:35:48.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.446+0000 7f6bc6ffd700 10 client.27553 handle_command_reply: tid=0 2023-07-21T22:35:48.501 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.446+0000 7f6c01ba2b80 1 client.27553 shutdown 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.446+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 msgr2=0x56167bcd9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.446+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd9b70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x56167bcce5b0 tx=0x7f6bd8099000 comp rx=0 tx=0).stop 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 shutdown_connections 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bce6e90 0x56167bce92a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd9b70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.502 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/3607498282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 >> 172.21.15.66:0/3607498282 conn(0x56167bcc2ef0 msgr2=0x56167bcc8420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6bc6ffd700 0 client.27553 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 shutdown_connections 2023-07-21T22:35:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.447+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/3607498282 wait complete. 2023-07-21T22:35:48.503 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.448+0000 7f6c01ba2b80 20 client.27553 trim_cache size 0 max 16384 2023-07-21T22:35:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 Processor -- start 2023-07-21T22:35:48.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 -- start start 2023-07-21T22:35:48.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcde410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd21b0 0x56167bce8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bcd9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bcce1a0 con 0x56167bcd21b0 2023-07-21T22:35:48.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bcc75e0 con 0x56167bcd2ac0 2023-07-21T22:35:48.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.469+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bcce3e0 con 0x56167bccd930 2023-07-21T22:35:48.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcde410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcde410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47924/0 (socket says 172.21.15.66:47924) 2023-07-21T22:35:48.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bcd9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 -- 172.21.15.66:0/313555798 learned_addr learned my addr 172.21.15.66:0/313555798 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd21b0 0x56167bce8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 -- 172.21.15.66:0/313555798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 msgr2=0x56167bcd9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bcd9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 -- 172.21.15.66:0/313555798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd21b0 msgr2=0x56167bce8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd21b0 0x56167bce8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.470+0000 7f6be77fe700 1 -- 172.21.15.66:0/313555798 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56167ba73250 con 0x56167bccd930 2023-07-21T22:35:48.522 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.471+0000 7f6be77fe700 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcde410 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8095890 tx=0x7f6bd8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f299effce65579e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.471+0000 7f6bce7fc700 1 -- 172.21.15.66:0/313555798 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd809d3c0 con 0x56167bccd930 2023-07-21T22:35:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.471+0000 7f6bce7fc700 1 -- 172.21.15.66:0/313555798 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bd80cc070 con 0x56167bccd930 2023-07-21T22:35:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.471+0000 7f6bce7fc700 1 -- 172.21.15.66:0/313555798 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd809f0d0 con 0x56167bccd930 2023-07-21T22:35:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/313555798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 msgr2=0x56167bcde410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcde410 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6bd8095890 tx=0x7f6bd8000bc0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/313555798 shutdown_connections 2023-07-21T22:35:48.524 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd2ac0 0x56167bcd9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcde410 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/313555798 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd21b0 0x56167bce8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.525 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/313555798 >> 172.21.15.66:0/313555798 conn(0x56167bcc2ef0 msgr2=0x56167bce3610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/313555798 shutdown_connections 2023-07-21T22:35:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.472+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/313555798 wait complete. 2023-07-21T22:35:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.473+0000 7f6c01ba2b80 1 -- start start 2023-07-21T22:35:48.526 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.473+0000 7f6c01ba2b80 1 Processor -- start 2023-07-21T22:35:48.531 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.532 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bcd6560 con 0x56167bcd2ac0 2023-07-21T22:35:48.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bcd66d0 con 0x56167bcd21b0 2023-07-21T22:35:48.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56167bcd6890 con 0x56167bccd930 2023-07-21T22:35:48.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6c01ba2b80 10 client.0 _ll_register_callbacks cb 0x7f6bd8099160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:48.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.533 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6be7fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46156/0 (socket says 172.21.15.66:46156) 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6be7fff700 1 -- 172.21.15.66:0/2042346218 learned_addr learned my addr 172.21.15.66:0/2042346218 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6be77fe700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47930/0 (socket says 172.21.15.66:47930) 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.474+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be6ffd700 1 -- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 msgr2=0x56167bcd55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.534 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be6ffd700 1 -- 172.21.15.66:0/2042346218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 msgr2=0x56167bcd6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be6ffd700 1 -- 172.21.15.66:0/2042346218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56167ba35c60 con 0x56167bcd21b0 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be7fff700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6be6ffd700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd5ae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x56167baa2940 tx=0x7f6bdc01d660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd72ee01d3289c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6bc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.475+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/2042346218 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc017040 con 0x56167bcd21b0 2023-07-21T22:35:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/2042346218 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6bdc01dbc0 con 0x56167bcd21b0 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6bd8099390 con 0x56167bcd21b0 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/2042346218 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bdc01dd80 con 0x56167bcd21b0 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6c01ba2b80 10 client.27587 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x56167ba4b2b0 con 0x56167bcd21b0 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/2042346218 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bd8099390 con 0x56167bcd21b0 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.476+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/2042346218 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6bdc0125f0 con 0x56167bcd21b0 2023-07-21T22:35:48.536 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6c01ba2b80 10 client.27587 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:48.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6c01ba2b80 10 client.27587 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T22:35:48.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6c01ba2b80 20 client.27587 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:48.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bd3b930 0x56167bd3dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6c01ba2b80 4 client.27587 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:35:48.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 --> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56167bbda050 con 0x56167bd3b930 2023-07-21T22:35:48.537 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6be77fe700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bd3b930 0x56167bd3dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6be77fe700 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bd3b930 0x56167bd3dd40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d7d00b7fd80bd16 server_cookie=5528004915b6ce0c in_seq=0 out_seq=0 2023-07-21T22:35:48.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.477+0000 7f6bc6ffd700 10 client.27587 ms_handle_connect on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:48.538 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6bc6ffd700 1 -- 172.21.15.66:0/2042346218 <== mds.2 v2:172.21.15.149:6838/2751755690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+2243 (crc 0 0 0) 0x56167bbda050 con 0x56167bd3b930 2023-07-21T22:35:48.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6bc6ffd700 10 client.27587 handle_command_reply: tid=0 2023-07-21T22:35:48.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 client.27587 shutdown 2023-07-21T22:35:48.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 msgr2=0x56167bcd5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.539 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd5ae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x56167baa2940 tx=0x7f6bdc01d660 comp rx=0 tx=0).stop 2023-07-21T22:35:48.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 shutdown_connections 2023-07-21T22:35:48.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:6838/2751755690,v1:172.21.15.149:6839/2751755690] conn(0x56167bd3b930 0x56167bd3dd40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.540 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x56167bcd2ac0 0x56167bcd6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56167bccd930 0x56167bcd55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 --2- 172.21.15.66:0/2042346218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56167bcd21b0 0x56167bcd5ae0 secure :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0x56167baa2940 tx=0x7f6bdc01d660 comp rx=0 tx=0).stop 2023-07-21T22:35:48.541 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6bc6ffd700 0 client.27587 ms_handle_reset on v2:172.21.15.149:6838/2751755690 2023-07-21T22:35:48.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.479+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 >> 172.21.15.66:0/2042346218 conn(0x56167bcc2ef0 msgr2=0x56167bce8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.481+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 shutdown_connections 2023-07-21T22:35:48.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.481+0000 7f6c01ba2b80 1 -- 172.21.15.66:0/2042346218 wait complete. 2023-07-21T22:35:48.542 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.481+0000 7f6c01ba2b80 20 client.27587 trim_cache size 0 max 16384 2023-07-21T22:35:48.543 INFO:teuthology.orchestra.run.smithi066.stdout:[ 2023-07-21T22:35:48.543 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.543 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.543 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 95207311, 2023-07-21T22:35:48.543 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628296, 2023-07-21T22:35:48.544 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORD" 2023-07-21T22:35:48.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.544 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.544 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.544 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 249701508, 2023-07-21T22:35:48.545 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628309, 2023-07-21T22:35:48.545 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/COREL" 2023-07-21T22:35:48.545 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.545 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.545 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 407744692, 2023-07-21T22:35:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628300, 2023-07-21T22:35:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/EXCEL" 2023-07-21T22:35:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.546 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.547 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 505810414, 2023-07-21T22:35:48.547 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628352, 2023-07-21T22:35:48.547 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/SEED" 2023-07-21T22:35:48.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.547 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.547 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 2000672907, 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511627785, 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp" 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.548 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 2596043579, 2023-07-21T22:35:48.549 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628321, 2023-07-21T22:35:48.549 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORDPRO" 2023-07-21T22:35:48.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.549 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.549 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.549 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 3367264304, 2023-07-21T22:35:48.550 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628329, 2023-07-21T22:35:48.550 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PARADOX" 2023-07-21T22:35:48.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.550 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 3378681424, 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628303, 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PWRPNT" 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.551 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.552 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 3784312346, 2023-07-21T22:35:48.552 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628350, 2023-07-21T22:35:48.552 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ACCESS" 2023-07-21T22:35:48.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:35:48.552 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:35:48.552 INFO:teuthology.orchestra.run.smithi066.stdout: "damage_type": "backtrace", 2023-07-21T22:35:48.553 INFO:teuthology.orchestra.run.smithi066.stdout: "id": 4090255633, 2023-07-21T22:35:48.553 INFO:teuthology.orchestra.run.smithi066.stdout: "ino": 1099511628325, 2023-07-21T22:35:48.553 INFO:teuthology.orchestra.run.smithi066.stdout: "path": "/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PM" 2023-07-21T22:35:48.553 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:35:48.553 INFO:teuthology.orchestra.run.smithi066.stdout:] 2023-07-21T22:35:48.554 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.482+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6bd00a14d0 msgr2=0x7f6bd00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.482+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6bd00a14d0 0x7f6bd00a3980 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6be000d9a0 tx=0x7f6be000aed0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 msgr2=0x7f6be81aa940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be81aa940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80078d0 tx=0x7f6bd8007c90 comp rx=0 tx=0).stop 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 shutdown_connections 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6be8110b50 0x7f6be810f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6bd00a14d0 0x7f6bd00a3980 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.555 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6be8107780 0x7f6be81aa940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 --2- 172.21.15.66:0/3795072572 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6be8108090 0x7f6be81aae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 >> 172.21.15.66:0/3795072572 conn(0x7f6be806c200 msgr2=0x7f6be8110f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 shutdown_connections 2023-07-21T22:35:48.556 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.483+0000 7f6bee1ff700 1 -- 172.21.15.66:0/3795072572 wait complete. 2023-07-21T22:35:48.557 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T22:35:48.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 -- 172.21.15.66:0/2866415582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 msgr2=0x7f6934101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2866415582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f6934101070 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6928009990 tx=0x7f6928009ca0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.826 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 -- 172.21.15.66:0/2866415582 shutdown_connections 2023-07-21T22:35:48.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2866415582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f693410a070 0x7f693410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2866415582 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f6934101070 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2866415582 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f69341015b0 0x7f6934109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.827 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 -- 172.21.15.66:0/2866415582 >> 172.21.15.66:0/2866415582 conn(0x7f69340fa4b0 msgr2=0x7f69340fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 -- 172.21.15.66:0/2866415582 shutdown_connections 2023-07-21T22:35:48.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.780+0000 7f693b1ff700 1 -- 172.21.15.66:0/2866415582 wait complete. 2023-07-21T22:35:48.828 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.782+0000 7f693b1ff700 1 Processor -- start 2023-07-21T22:35:48.829 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.782+0000 7f693b1ff700 1 -- start start 2023-07-21T22:35:48.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693b1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f69341a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693b1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69341015b0 0x7f69341a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693b1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f693410a070 0x7f6934108750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693b1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934108f00 con 0x7f693410a070 2023-07-21T22:35:48.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693b1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934109070 con 0x7f69341015b0 2023-07-21T22:35:48.830 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693b1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934109230 con 0x7f6934100ca0 2023-07-21T22:35:48.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f6938f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f69341a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f6938f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f69341a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47948/0 (socket says 172.21.15.66:47948) 2023-07-21T22:35:48.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f6938f9b700 1 -- 172.21.15.66:0/2394782967 learned_addr learned my addr 172.21.15.66:0/2394782967 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.831 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.783+0000 7f693979c700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f693410a070 0x7f6934108750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.784+0000 7f6938f9b700 1 -- 172.21.15.66:0/2394782967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69341015b0 msgr2=0x7f69341a1460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T22:35:48.832 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.784+0000 7f6938f9b700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69341015b0 0x7f69341a1460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.784+0000 7f6938f9b700 1 -- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f693410a070 msgr2=0x7f6934108750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.784+0000 7f6938f9b700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f693410a070 0x7f6934108750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.784+0000 7f6938f9b700 1 -- 172.21.15.66:0/2394782967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69280095f0 con 0x7f6934100ca0 2023-07-21T22:35:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.784+0000 7f6938f9b700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f69341a0ee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69280095c0 tx=0x7f692800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.833 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.785+0000 7f6931ffb700 1 -- 172.21.15.66:0/2394782967 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692801d070 con 0x7f6934100ca0 2023-07-21T22:35:48.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.785+0000 7f6931ffb700 1 -- 172.21.15.66:0/2394782967 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69280054f0 con 0x7f6934100ca0 2023-07-21T22:35:48.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.785+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69341094b0 con 0x7f6934100ca0 2023-07-21T22:35:48.834 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.785+0000 7f6931ffb700 1 -- 172.21.15.66:0/2394782967 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928004b30 con 0x7f6934100ca0 2023-07-21T22:35:48.835 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.785+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693406a1c0 con 0x7f6934100ca0 2023-07-21T22:35:48.836 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.790+0000 7f6931ffb700 1 -- 172.21.15.66:0/2394782967 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f692800fb80 con 0x7f6934100ca0 2023-07-21T22:35:48.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.792+0000 7f6931ffb700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f69240a1340 0x7f69240a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.838 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.792+0000 7f6931ffb700 1 -- 172.21.15.66:0/2394782967 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f69280cdf20 con 0x7f6934100ca0 2023-07-21T22:35:48.839 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.792+0000 7f6933fff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f69240a1340 0x7f69240a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 Processor -- start 2023-07-21T22:35:48.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 -- start start 2023-07-21T22:35:48.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f6933fff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f69240a1340 0x7f69240a37f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6934109f40 tx=0x7f692000b3f0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f5091b0 0x559a9f509580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.841 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f504350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f504930 0x559a9f504da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f409cb0 con 0x559a9f5091b0 2023-07-21T22:35:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f45ca40 con 0x559a9f504930 2023-07-21T22:35:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.793+0000 7f694ea36b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f455ed0 con 0x559a9f509ac0 2023-07-21T22:35:48.842 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f504350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f504350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47958/0 (socket says 172.21.15.66:47958) 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 -- 172.21.15.66:0/1750986142 learned_addr learned my addr 172.21.15.66:0/1750986142 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f693979c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f504930 0x559a9f504da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6938f9b700 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f5091b0 0x559a9f509580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 -- 172.21.15.66:0/1750986142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f504930 msgr2=0x559a9f504da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f504930 0x559a9f504da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.843 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 -- 172.21.15.66:0/1750986142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f5091b0 msgr2=0x559a9f509580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f5091b0 0x559a9f509580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 -- 172.21.15.66:0/1750986142 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9f283de0 con 0x559a9f509ac0 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6938f9b700 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f5091b0 0x559a9f509580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.794+0000 7f6933fff700 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f504350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f692000b330 tx=0x7f6920010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35992f166160da48 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.795+0000 7f691effd700 1 -- 172.21.15.66:0/1750986142 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920021070 con 0x559a9f509ac0 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.795+0000 7f691effd700 1 -- 172.21.15.66:0/1750986142 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692000ded0 con 0x559a9f509ac0 2023-07-21T22:35:48.844 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.795+0000 7f691effd700 1 -- 172.21.15.66:0/1750986142 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920005280 con 0x559a9f509ac0 2023-07-21T22:35:48.845 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 -- 172.21.15.66:0/1750986142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 msgr2=0x559a9f504350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f504350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f692000b330 tx=0x7f6920010000 comp rx=0 tx=0).stop 2023-07-21T22:35:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 -- 172.21.15.66:0/1750986142 shutdown_connections 2023-07-21T22:35:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f504930 0x559a9f504da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f5091b0 0x559a9f509580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1750986142 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f504350 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.846 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 -- 172.21.15.66:0/1750986142 >> 172.21.15.66:0/1750986142 conn(0x559a9f4f9ef0 msgr2=0x559a9f4ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.796+0000 7f694ea36b80 1 -- 172.21.15.66:0/1750986142 shutdown_connections 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.797+0000 7f694ea36b80 1 -- 172.21.15.66:0/1750986142 wait complete. 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.797+0000 7f694ea36b80 1 -- start start 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.797+0000 7f694ea36b80 1 Processor -- start 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f510c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.847 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f3f5ac0 con 0x559a9f504930 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f3f57c0 con 0x559a9f5091b0 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f2d9920 con 0x559a9f509ac0 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.798+0000 7f694ea36b80 10 client.0 _ll_register_callbacks cb 0x559a9f3f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.799+0000 7f6933fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.799+0000 7f6938f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f510c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.848 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.799+0000 7f6933fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41522/0 (socket says 172.21.15.66:41522) 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.799+0000 7f6933fff700 1 -- 172.21.15.66:0/3545212829 learned_addr learned my addr 172.21.15.66:0/3545212829 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.799+0000 7f693979c700 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f693979c700 1 -- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 msgr2=0x559a9f50de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f693979c700 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f693979c700 1 -- 172.21.15.66:0/3545212829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 msgr2=0x559a9f510c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f693979c700 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f510c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.849 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f693979c700 1 -- 172.21.15.66:0/3545212829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692000d8d0 con 0x559a9f509ac0 2023-07-21T22:35:48.850 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f693979c700 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50e580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f692c00ae80 tx=0x7f692c012510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c3329a27519e7b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f6913fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f6913fff700 1 -- 172.21.15.66:0/3545212829 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00f040 con 0x559a9f509ac0 2023-07-21T22:35:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a9f3af4f0 con 0x559a9f509ac0 2023-07-21T22:35:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.800+0000 7f6913fff700 1 -- 172.21.15.66:0/3545212829 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f692c012aa0 con 0x559a9f509ac0 2023-07-21T22:35:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f6913fff700 1 -- 172.21.15.66:0/3545212829 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692c012c60 con 0x559a9f509ac0 2023-07-21T22:35:48.851 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 10 client.27571 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x559a9f26cc60 con 0x559a9f509ac0 2023-07-21T22:35:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f6913fff700 1 -- 172.21.15.66:0/3545212829 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559a9f3af4f0 con 0x559a9f509ac0 2023-07-21T22:35:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f6913fff700 1 -- 172.21.15.66:0/3545212829 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f692c0083f0 con 0x559a9f509ac0 2023-07-21T22:35:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 10 client.27571 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:48.852 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 10 client.27571 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 20 client.27571 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f51e540 0x559a9f520950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 4 client.27571 mds_command: new command op to 24530 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.801+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a9f44eeb0 con 0x559a9f51e540 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.802+0000 7f6938f9b700 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f51e540 0x559a9f520950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.802+0000 7f6938f9b700 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f51e540 0x559a9f520950 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a35e804694f7dca5 server_cookie=9dbf86029af2b339 in_seq=0 out_seq=0 2023-07-21T22:35:48.853 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.802+0000 7f6913fff700 10 client.27571 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.806+0000 7f6913fff700 1 -- 172.21.15.66:0/3545212829 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a9f44eeb0 con 0x559a9f51e540 2023-07-21T22:35:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.806+0000 7f6913fff700 10 client.27571 handle_command_reply: tid=0 2023-07-21T22:35:48.854 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.807+0000 7f694ea36b80 1 client.27571 shutdown 2023-07-21T22:35:48.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 msgr2=0x559a9f50e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.855 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50e580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f692c00ae80 tx=0x7f692c012510 comp rx=0 tx=0).stop 2023-07-21T22:35:48.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 shutdown_connections 2023-07-21T22:35:48.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50e580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f510c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f51e540 0x559a9f520950 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.856 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 --2- 172.21.15.66:0/3545212829 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f6913fff700 0 client.27571 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:48.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 >> 172.21.15.66:0/3545212829 conn(0x559a9f4f9ef0 msgr2=0x559a9f4ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 shutdown_connections 2023-07-21T22:35:48.857 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.809+0000 7f694ea36b80 1 -- 172.21.15.66:0/3545212829 wait complete. 2023-07-21T22:35:48.858 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.810+0000 7f694ea36b80 20 client.27571 trim_cache size 0 max 16384 2023-07-21T22:35:48.873 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.827+0000 7f694ea36b80 1 Processor -- start 2023-07-21T22:35:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.827+0000 7f694ea36b80 1 -- start start 2023-07-21T22:35:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f515540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f694ea36b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f4fc040 con 0x559a9f504930 2023-07-21T22:35:48.874 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f694ea36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f5056d0 con 0x559a9f5091b0 2023-07-21T22:35:48.875 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f694ea36b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f500e80 con 0x559a9f509ac0 2023-07-21T22:35:48.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f6933fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f6933fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41534/0 (socket says 172.21.15.66:41534) 2023-07-21T22:35:48.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f6933fff700 1 -- 172.21.15.66:0/1373732995 learned_addr learned my addr 172.21.15.66:0/1373732995 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.828+0000 7f693979c700 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6938f9b700 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f515540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.876 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6933fff700 1 -- 172.21.15.66:0/1373732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 msgr2=0x559a9f50dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6933fff700 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6933fff700 1 -- 172.21.15.66:0/1373732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 msgr2=0x559a9f515540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6933fff700 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f515540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6933fff700 1 -- 172.21.15.66:0/1373732995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a9f2aa250 con 0x559a9f5091b0 2023-07-21T22:35:48.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f693979c700 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:48.877 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.829+0000 7f6933fff700 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50d6e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f692000b810 tx=0x7f692000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66fd0431be4a3cb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f691f7fe700 1 -- 172.21.15.66:0/1373732995 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920021070 con 0x559a9f5091b0 2023-07-21T22:35:48.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f691f7fe700 1 -- 172.21.15.66:0/1373732995 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6920022040 con 0x559a9f5091b0 2023-07-21T22:35:48.878 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f691f7fe700 1 -- 172.21.15.66:0/1373732995 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69200038e0 con 0x559a9f5091b0 2023-07-21T22:35:48.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 -- 172.21.15.66:0/1373732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 msgr2=0x559a9f50d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50d6e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f692000b810 tx=0x7f692000b840 comp rx=0 tx=0).stop 2023-07-21T22:35:48.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 -- 172.21.15.66:0/1373732995 shutdown_connections 2023-07-21T22:35:48.880 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f509ac0 0x559a9f50dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f504930 0x559a9f515540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 --2- 172.21.15.66:0/1373732995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50d6e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.830+0000 7f694ea36b80 1 -- 172.21.15.66:0/1373732995 >> 172.21.15.66:0/1373732995 conn(0x559a9f4f9ef0 msgr2=0x559a9f502a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.831+0000 7f694ea36b80 1 -- 172.21.15.66:0/1373732995 shutdown_connections 2023-07-21T22:35:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.831+0000 7f694ea36b80 1 -- 172.21.15.66:0/1373732995 wait complete. 2023-07-21T22:35:48.881 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.831+0000 7f694ea36b80 1 -- start start 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.831+0000 7f694ea36b80 1 Processor -- start 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 0x559a9f515740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f509ac0 0x559a9f50cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f50d5c0 con 0x559a9f509ac0 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f5723c0 con 0x559a9f5091b0 2023-07-21T22:35:48.882 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a9f572500 con 0x559a9f504930 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f694ea36b80 10 client.0 _ll_register_callbacks cb 0x7f692c00a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 0x559a9f515740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 0x559a9f515740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:47998/0 (socket says 172.21.15.66:47998) 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 -- 172.21.15.66:0/2277418075 learned_addr learned my addr 172.21.15.66:0/2277418075 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6933fff700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f693979c700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f509ac0 0x559a9f50cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 -- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 msgr2=0x559a9f50c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 -- 172.21.15.66:0/2277418075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f509ac0 msgr2=0x559a9f50cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f509ac0 0x559a9f50cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.832+0000 7f6938f9b700 1 -- 172.21.15.66:0/2277418075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a9f26cc60 con 0x559a9f504930 2023-07-21T22:35:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f6938f9b700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 0x559a9f515740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f692809a9e0 tx=0x7f6928009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69816e77bb2d416 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f6913fff700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:48.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f6913fff700 1 -- 172.21.15.66:0/2277418075 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280b33d0 con 0x559a9f504930 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f692000ded0 con 0x559a9f504930 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f6913fff700 1 -- 172.21.15.66:0/2277418075 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f692809af40 con 0x559a9f504930 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f6913fff700 1 -- 172.21.15.66:0/2277418075 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f692809b120 con 0x559a9f504930 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f694ea36b80 10 client.27577 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.833+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x559a9f283de0 con 0x559a9f504930 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f6913fff700 1 -- 172.21.15.66:0/2277418075 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f692807d080 con 0x559a9f504930 2023-07-21T22:35:48.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f6913fff700 1 -- 172.21.15.66:0/2277418075 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6928079bb0 con 0x559a9f504930 2023-07-21T22:35:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f694ea36b80 10 client.27577 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f694ea36b80 10 client.27577 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T22:35:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f694ea36b80 20 client.27577 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f694ea36b80 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f5737f0 0x559a9f575c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f694ea36b80 4 client.27577 mds_command: new command op to 24530 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:35:48.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 --> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a9f411050 con 0x559a9f5737f0 2023-07-21T22:35:48.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.834+0000 7f6933fff700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f5737f0 0x559a9f575c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:48.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.835+0000 7f6933fff700 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f5737f0 0x559a9f575c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=549e471c74258122 server_cookie=db3c7ad7d25b7bcf in_seq=0 out_seq=0 2023-07-21T22:35:48.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.835+0000 7f6913fff700 10 client.27577 ms_handle_connect on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:48.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f6913fff700 1 -- 172.21.15.66:0/2277418075 <== mds.1 v2:172.21.15.104:6836/1237242254 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a9f411050 con 0x559a9f5737f0 2023-07-21T22:35:48.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f6913fff700 10 client.27577 handle_command_reply: tid=0 2023-07-21T22:35:48.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 client.27577 shutdown 2023-07-21T22:35:48.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 msgr2=0x559a9f515740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 0x559a9f515740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f692809a9e0 tx=0x7f6928009310 comp rx=0 tx=0).stop 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 shutdown_connections 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x559a9f509ac0 0x559a9f50cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559a9f504930 0x559a9f515740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:6836/1237242254,v1:172.21.15.104:6837/1237242254] conn(0x559a9f5737f0 0x559a9f575c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f6913fff700 0 client.27577 ms_handle_reset on v2:172.21.15.104:6836/1237242254 2023-07-21T22:35:48.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 --2- 172.21.15.66:0/2277418075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9f5091b0 0x559a9f50c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.838+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 >> 172.21.15.66:0/2277418075 conn(0x559a9f4f9ef0 msgr2=0x559a9f503950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.839+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 shutdown_connections 2023-07-21T22:35:48.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.839+0000 7f694ea36b80 1 -- 172.21.15.66:0/2277418075 wait complete. 2023-07-21T22:35:48.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.839+0000 7f694ea36b80 20 client.27577 trim_cache size 0 max 16384 2023-07-21T22:35:48.890 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:35:48.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f69240a1340 msgr2=0x7f69240a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f69240a1340 0x7f69240a37f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6934109f40 tx=0x7f692000b3f0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 msgr2=0x7f69341a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:48.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f69341a0ee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f69280095c0 tx=0x7f692800f690 comp rx=0 tx=0).stop 2023-07-21T22:35:48.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 shutdown_connections 2023-07-21T22:35:48.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f69240a1340 0x7f69240a37f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f693410a070 0x7f6934108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6934100ca0 0x7f69341a0ee0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 --2- 172.21.15.66:0/2394782967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69341015b0 0x7f69341a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 >> 172.21.15.66:0/2394782967 conn(0x7f69340fa4b0 msgr2=0x7f693410c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 shutdown_connections 2023-07-21T22:35:48.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:48.840+0000 7f693b1ff700 1 -- 172.21.15.66:0/2394782967 wait complete. 2023-07-21T22:35:48.910 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T22:35:48.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 45613 35.18 MB/sec warmup 75 sec latency 38.345 ms 2023-07-21T22:35:49.199 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:49 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/786826465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:35:49.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.171+0000 7f1d383ff700 1 -- 172.21.15.66:0/3009781321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 msgr2=0x7f1d30112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.171+0000 7f1d383ff700 1 --2- 172.21.15.66:0/3009781321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d30112f30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c310 tx=0x7f1d2800c620 comp rx=0 tx=0).stop 2023-07-21T22:35:49.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 -- 172.21.15.66:0/3009781321 shutdown_connections 2023-07-21T22:35:49.218 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 --2- 172.21.15.66:0/3009781321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d30112f30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 --2- 172.21.15.66:0/3009781321 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d30107780 0x7f1d30107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 --2- 172.21.15.66:0/3009781321 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 0x7f1d30110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 -- 172.21.15.66:0/3009781321 >> 172.21.15.66:0/3009781321 conn(0x7f1d3006c200 msgr2=0x7f1d3006c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:49.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 -- 172.21.15.66:0/3009781321 shutdown_connections 2023-07-21T22:35:49.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.172+0000 7f1d383ff700 1 -- 172.21.15.66:0/3009781321 wait complete. 2023-07-21T22:35:49.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.173+0000 7f1d383ff700 1 Processor -- start 2023-07-21T22:35:49.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 -- start start 2023-07-21T22:35:49.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d30107780 0x7f1d3010ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 0x7f1d301104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d3010a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3010a8a0 con 0x7f1d30107780 2023-07-21T22:35:49.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3010aa10 con 0x7f1d30110b50 2023-07-21T22:35:49.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.175+0000 7f1d383ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3010abd0 con 0x7f1d30108090 2023-07-21T22:35:49.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d3010a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d3010a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41554/0 (socket says 172.21.15.66:41554) 2023-07-21T22:35:49.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3699c700 1 -- 172.21.15.66:0/1125242267 learned_addr learned my addr 172.21.15.66:0/1125242267 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:49.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3599a700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 0x7f1d301104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3599a700 1 -- 172.21.15.66:0/1125242267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 msgr2=0x7f1d3010a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3599a700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d3010a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3599a700 1 -- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d30107780 msgr2=0x7f1d3010ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3599a700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d30107780 0x7f1d3010ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.176+0000 7f1d3599a700 1 -- 172.21.15.66:0/1125242267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2800c050 con 0x7f1d30108090 2023-07-21T22:35:49.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.177+0000 7f1d3599a700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 0x7f1d301104c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00e9c0 tx=0x7f1d2c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:49.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.177+0000 7f1d277fe700 1 -- 172.21.15.66:0/1125242267 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00cbd0 con 0x7f1d30108090 2023-07-21T22:35:49.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.177+0000 7f1d277fe700 1 -- 172.21.15.66:0/1125242267 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d2c00cd30 con 0x7f1d30108090 2023-07-21T22:35:49.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.177+0000 7f1d277fe700 1 -- 172.21.15.66:0/1125242267 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c018680 con 0x7f1d30108090 2023-07-21T22:35:49.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.178+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d301b9dc0 con 0x7f1d30108090 2023-07-21T22:35:49.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.178+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d301ba190 con 0x7f1d30108090 2023-07-21T22:35:49.231 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.185+0000 7f1d4bca1b80 1 Processor -- start 2023-07-21T22:35:49.233 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.187+0000 7f1d277fe700 1 -- 172.21.15.66:0/1125242267 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1d2c0187e0 con 0x7f1d30108090 2023-07-21T22:35:49.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.189+0000 7f1d277fe700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1d1c0a1380 0x7f1d1c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.235 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.189+0000 7f1d277fe700 1 -- 172.21.15.66:0/1125242267 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1d2c014070 con 0x7f1d30108090 2023-07-21T22:35:49.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.190+0000 7f1d4bca1b80 1 -- start start 2023-07-21T22:35:49.236 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.190+0000 7f1d3699c700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1d1c0a1380 0x7f1d1c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb0ac0 0x55a037cab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d3699c700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1d1c0a1380 0x7f1d1c0a3830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007830 tx=0x7f1d2800c9b0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:49.237 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 0x55a037cabda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037bb0cb0 con 0x55a037cab930 2023-07-21T22:35:49.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037c03a40 con 0x55a037cb0ac0 2023-07-21T22:35:49.238 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.191+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037bfced0 con 0x55a037cb01b0 2023-07-21T22:35:49.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.194+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 0x55a037cabda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.194+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 0x55a037cabda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:46240/0 (socket says 172.21.15.66:46240) 2023-07-21T22:35:49.240 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.194+0000 7f1d3699c700 1 -- 172.21.15.66:0/93008190 learned_addr learned my addr 172.21.15.66:0/93008190 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:49.241 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.195+0000 7f1d3599a700 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb0ac0 0x55a037cab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3599a700 1 -- 172.21.15.66:0/93008190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 msgr2=0x55a037cb0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3599a700 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3599a700 1 -- 172.21.15.66:0/93008190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 msgr2=0x55a037cabda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3599a700 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 0x55a037cabda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3599a700 1 -- 172.21.15.66:0/93008190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a037a2ade0 con 0x55a037cb0ac0 2023-07-21T22:35:49.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3699c700 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 0x55a037cabda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:35:49.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.196+0000 7f1d3599a700 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb0ac0 0x55a037cab350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0ca5b0 tx=0x7f1d2c099af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5935ed2a6228be9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:49.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.197+0000 7f1d24bfe700 1 -- 172.21.15.66:0/93008190 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c099f30 con 0x55a037cb0ac0 2023-07-21T22:35:49.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.197+0000 7f1d24bfe700 1 -- 172.21.15.66:0/93008190 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d2c09df60 con 0x55a037cb0ac0 2023-07-21T22:35:49.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.197+0000 7f1d24bfe700 1 -- 172.21.15.66:0/93008190 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c099420 con 0x55a037cb0ac0 2023-07-21T22:35:49.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/93008190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb0ac0 msgr2=0x55a037cab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb0ac0 0x55a037cab350 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0ca5b0 tx=0x7f1d2c099af0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/93008190 shutdown_connections 2023-07-21T22:35:49.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cab930 0x55a037cabda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/93008190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb0ac0 0x55a037cab350 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.245 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.198+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/93008190 >> 172.21.15.66:0/93008190 conn(0x55a037ca0ef0 msgr2=0x55a037ca6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:49.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.200+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/93008190 shutdown_connections 2023-07-21T22:35:49.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.201+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/93008190 wait complete. 2023-07-21T22:35:49.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.201+0000 7f1d4bca1b80 1 -- start start 2023-07-21T22:35:49.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.202+0000 7f1d4bca1b80 1 Processor -- start 2023-07-21T22:35:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cab930 0x55a037cb4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cb0ac0 0x55a037cb5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037b9cac0 con 0x55a037cb0ac0 2023-07-21T22:35:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037b9c7c0 con 0x55a037cab930 2023-07-21T22:35:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037a80920 con 0x55a037cb01b0 2023-07-21T22:35:49.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.203+0000 7f1d4bca1b80 10 client.0 _ll_register_callbacks cb 0x55a037b9a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.204+0000 7f1d3599a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.204+0000 7f1d3619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cab930 0x55a037cb4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.204+0000 7f1d3619b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cab930 0x55a037cb4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41576/0 (socket says 172.21.15.66:41576) 2023-07-21T22:35:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.204+0000 7f1d3619b700 1 -- 172.21.15.66:0/3899129470 learned_addr learned my addr 172.21.15.66:0/3899129470 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:49.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d3599a700 1 -- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cab930 msgr2=0x55a037cb4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d3599a700 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cab930 0x55a037cb4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d3599a700 1 -- 172.21.15.66:0/3899129470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cb0ac0 msgr2=0x55a037cb5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d3599a700 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cb0ac0 0x55a037cb5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d3599a700 1 -- 172.21.15.66:0/3899129470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d30113bd0 con 0x55a037cb01b0 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d3599a700 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb54c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c095cb0 tx=0x7f1d2c099650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93b6480bf9915a87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.205+0000 7f1d19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:49.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.206+0000 7f1d19ffb700 1 -- 172.21.15.66:0/3899129470 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c099f30 con 0x55a037cb01b0 2023-07-21T22:35:49.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.206+0000 7f1d19ffb700 1 -- 172.21.15.66:0/3899129470 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1d2c0a2380 con 0x55a037cb01b0 2023-07-21T22:35:49.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.206+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a037b564f0 con 0x55a037cb01b0 2023-07-21T22:35:49.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.206+0000 7f1d19ffb700 1 -- 172.21.15.66:0/3899129470 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d2c09a3a0 con 0x55a037cb01b0 2023-07-21T22:35:49.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.206+0000 7f1d4bca1b80 10 client.27586 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:49.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.206+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f1d301ba190 con 0x55a037cb01b0 2023-07-21T22:35:49.254 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d19ffb700 1 -- 172.21.15.66:0/3899129470 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a037b564f0 con 0x55a037cb01b0 2023-07-21T22:35:49.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d19ffb700 1 -- 172.21.15.66:0/3899129470 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1d2c0a1ef0 con 0x55a037cb01b0 2023-07-21T22:35:49.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d4bca1b80 10 client.27586 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:49.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d4bca1b80 10 client.27586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:49.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d4bca1b80 20 client.27586 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:49.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cc51f0 0x55a037cc7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d4bca1b80 4 client.27586 mds_command: new command op to 24572 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T22:35:49.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.207+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a037bf5eb0 con 0x55a037cc51f0 2023-07-21T22:35:49.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.208+0000 7f1d3619b700 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cc51f0 0x55a037cc7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.266 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.220+0000 7f1d3619b700 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cc51f0 0x55a037cc7600 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c3891fd9b15b3a server_cookie=6d5e67d56c6dfca5 in_seq=0 out_seq=0 2023-07-21T22:35:49.267 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.220+0000 7f1d19ffb700 10 client.27586 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:49.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.224+0000 7f1d19ffb700 1 -- 172.21.15.66:0/3899129470 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a037bf5eb0 con 0x55a037cc51f0 2023-07-21T22:35:49.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.224+0000 7f1d19ffb700 10 client.27586 handle_command_reply: tid=0 2023-07-21T22:35:49.271 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.225+0000 7f1d4bca1b80 1 client.27586 shutdown 2023-07-21T22:35:49.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.226+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 msgr2=0x55a037cb54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.226+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb54c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c095cb0 tx=0x7f1d2c099650 comp rx=0 tx=0).stop 2023-07-21T22:35:49.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.226+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 shutdown_connections 2023-07-21T22:35:49.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.226+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cab930 0x55a037cb4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cc51f0 0x55a037cc7600 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037cb0ac0 0x55a037cb5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.273 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3899129470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cb01b0 0x55a037cb54c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 >> 172.21.15.66:0/3899129470 conn(0x55a037ca0ef0 msgr2=0x55a037ca6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:49.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d19ffb700 0 client.27586 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:49.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 shutdown_connections 2023-07-21T22:35:49.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.227+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3899129470 wait complete. 2023-07-21T22:35:49.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.228+0000 7f1d4bca1b80 20 client.27586 trim_cache size 0 max 16384 2023-07-21T22:35:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.258+0000 7f1d4bca1b80 1 Processor -- start 2023-07-21T22:35:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 -- start start 2023-07-21T22:35:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cbc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.306 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cb5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037b564f0 con 0x55a037ca0ef0 2023-07-21T22:35:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037cb8160 con 0x55a037cb01b0 2023-07-21T22:35:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.259+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037ca09b0 con 0x55a037cab930 2023-07-21T22:35:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.260+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cb5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.260+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cb5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41578/0 (socket says 172.21.15.66:41578) 2023-07-21T22:35:49.307 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.260+0000 7f1d3699c700 1 -- 172.21.15.66:0/3037951845 learned_addr learned my addr 172.21.15.66:0/3037951845 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.260+0000 7f1d3599a700 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cbc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.261+0000 7f1d3699c700 1 -- 172.21.15.66:0/3037951845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 msgr2=0x55a037cbc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.261+0000 7f1d3699c700 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cbc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.261+0000 7f1d3699c700 1 -- 172.21.15.66:0/3037951845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 msgr2=0x55a037cab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.261+0000 7f1d3699c700 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.308 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.261+0000 7f1d3699c700 1 -- 172.21.15.66:0/3037951845 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a037ca5230 con 0x55a037cb01b0 2023-07-21T22:35:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.264+0000 7f1d3699c700 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cb5210 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1d28012a90 tx=0x7f1d2800c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8a375290437aa07 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.266+0000 7f1d253ff700 1 -- 172.21.15.66:0/3037951845 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28023040 con 0x55a037cb01b0 2023-07-21T22:35:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.266+0000 7f1d253ff700 1 -- 172.21.15.66:0/3037951845 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d2801eb10 con 0x55a037cb01b0 2023-07-21T22:35:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.266+0000 7f1d253ff700 1 -- 172.21.15.66:0/3037951845 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2800f040 con 0x55a037cb01b0 2023-07-21T22:35:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.267+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3037951845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 msgr2=0x55a037cb5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.313 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.267+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cb5210 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1d28012a90 tx=0x7f1d2800c620 comp rx=0 tx=0).stop 2023-07-21T22:35:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.268+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3037951845 shutdown_connections 2023-07-21T22:35:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.268+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cb5210 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.314 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.268+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.268+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/3037951845 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cbc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.315 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.268+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3037951845 >> 172.21.15.66:0/3037951845 conn(0x55a037cc1c60 msgr2=0x55a037caaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:49.316 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.271+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3037951845 shutdown_connections 2023-07-21T22:35:49.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.271+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/3037951845 wait complete. 2023-07-21T22:35:49.318 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.272+0000 7f1d4bca1b80 1 -- start start 2023-07-21T22:35:49.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.272+0000 7f1d4bca1b80 1 Processor -- start 2023-07-21T22:35:49.319 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cbc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cbac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037cbb1a0 con 0x55a037ca0ef0 2023-07-21T22:35:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037cbb310 con 0x55a037cb01b0 2023-07-21T22:35:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a037cbb4d0 con 0x55a037cab930 2023-07-21T22:35:49.320 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.273+0000 7f1d4bca1b80 10 client.0 _ll_register_callbacks cb 0x7f1d28004a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T22:35:49.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.274+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cbac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.274+0000 7f1d3699c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cbac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:41592/0 (socket says 172.21.15.66:41592) 2023-07-21T22:35:49.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.274+0000 7f1d3699c700 1 -- 172.21.15.66:0/2691979847 learned_addr learned my addr 172.21.15.66:0/2691979847 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:35:49.321 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.275+0000 7f1d3599a700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.275+0000 7f1d3619b700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cbc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.276+0000 7f1d3599a700 1 -- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 msgr2=0x55a037cbac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.276+0000 7f1d3599a700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cbac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.276+0000 7f1d3599a700 1 -- 172.21.15.66:0/2691979847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 msgr2=0x55a037cbc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T22:35:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.276+0000 7f1d3599a700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cbc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.322 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.276+0000 7f1d3599a700 1 -- 172.21.15.66:0/2691979847 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d28012800 con 0x55a037cab930 2023-07-21T22:35:49.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.276+0000 7f1d3599a700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cba720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0cabb0 tx=0x7f1d2c0a2280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=227a47e30a229748 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:35:49.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.277+0000 7f1d19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.149:3300/0 2023-07-21T22:35:49.323 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.277+0000 7f1d3619b700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cbc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:35:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.278+0000 7f1d19ffb700 1 -- 172.21.15.66:0/2691979847 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c098fb0 con 0x55a037cab930 2023-07-21T22:35:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.278+0000 7f1d19ffb700 1 -- 172.21.15.66:0/2691979847 <== mon.1 v2:172.21.15.149:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1d2c0a2380 con 0x55a037cab930 2023-07-21T22:35:49.324 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.278+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a037cc2030 con 0x55a037cab930 2023-07-21T22:35:49.326 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.280+0000 7f1d19ffb700 1 -- 172.21.15.66:0/2691979847 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d2c099f30 con 0x55a037cab930 2023-07-21T22:35:49.327 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.282+0000 7f1d4bca1b80 10 client.27592 fetch_fsmap learned FSMap version 50 2023-07-21T22:35:49.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.282+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55a037a13c60 con 0x55a037cab930 2023-07-21T22:35:49.328 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.282+0000 7f1d19ffb700 1 -- 172.21.15.66:0/2691979847 <== mon.1 v2:172.21.15.149:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a037cc2030 con 0x55a037cab930 2023-07-21T22:35:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d19ffb700 1 -- 172.21.15.66:0/2691979847 <== mon.1 v2:172.21.15.149:3300/0 5 ==== fsmap(e 50) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1d2c095d20 con 0x55a037cab930 2023-07-21T22:35:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d4bca1b80 10 client.27592 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T22:35:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d4bca1b80 10 client.27592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T22:35:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d4bca1b80 20 client.27592 populate_metadata read hostname 'smithi066' 2023-07-21T22:35:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cb0ac0 0x55a037d1c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:35:49.329 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d4bca1b80 4 client.27592 mds_command: new command op to 24572 tid=0[{"prefix": "damage ls"}] 2023-07-21T22:35:49.330 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.283+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 --> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f1d2c09df60 con 0x55a037cb0ac0 2023-07-21T22:35:49.333 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.287+0000 7f1d3619b700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cb0ac0 0x55a037d1c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:35:49.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.294+0000 7f1d3619b700 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cb0ac0 0x55a037d1c140 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4122b2e61594b1e5 server_cookie=c0c8c5c4f7400e89 in_seq=0 out_seq=0 2023-07-21T22:35:49.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.295+0000 7f1d19ffb700 10 client.27592 ms_handle_connect on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:49.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.301+0000 7f1d19ffb700 1 -- 172.21.15.66:0/2691979847 <== mds.0 v2:172.21.15.104:6838/1299576617 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f1d2c09df60 con 0x55a037cb0ac0 2023-07-21T22:35:49.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.301+0000 7f1d19ffb700 10 client.27592 handle_command_reply: tid=0 2023-07-21T22:35:49.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.301+0000 7f1d4bca1b80 1 client.27592 shutdown 2023-07-21T22:35:49.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 msgr2=0x55a037cba720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cba720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0cabb0 tx=0x7f1d2c0a2280 comp rx=0 tx=0).stop 2023-07-21T22:35:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 shutdown_connections 2023-07-21T22:35:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a037cb01b0 0x55a037cbac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.104:6838/1299576617,v1:172.21.15.104:6839/1299576617] conn(0x55a037cb0ac0 0x55a037d1c140 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x55a037ca0ef0 0x55a037cbc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 --2- 172.21.15.66:0/2691979847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a037cab930 0x55a037cba720 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.349 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.302+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 >> 172.21.15.66:0/2691979847 conn(0x55a037cc1c60 msgr2=0x55a037cb5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.303+0000 7f1d19ffb700 0 client.27592 ms_handle_reset on v2:172.21.15.104:6838/1299576617 2023-07-21T22:35:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.304+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 shutdown_connections 2023-07-21T22:35:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.304+0000 7f1d4bca1b80 1 -- 172.21.15.66:0/2691979847 wait complete. 2023-07-21T22:35:49.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.304+0000 7f1d4bca1b80 20 client.27592 trim_cache size 0 max 16384 2023-07-21T22:35:49.351 INFO:teuthology.orchestra.run.smithi066.stdout:[] 2023-07-21T22:35:49.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1d1c0a1380 msgr2=0x7f1d1c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1d1c0a1380 0x7f1d1c0a3830 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007830 tx=0x7f1d2800c9b0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 msgr2=0x7f1d301104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:35:49.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 0x7f1d301104c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00e9c0 tx=0x7f1d2c00ed80 comp rx=0 tx=0).stop 2023-07-21T22:35:49.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 shutdown_connections 2023-07-21T22:35:49.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30110b50 0x7f1d3010a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1d1c0a1380 0x7f1d1c0a3830 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1d30107780 0x7f1d3010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 --2- 172.21.15.66:0/1125242267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1d30108090 0x7f1d301104c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:35:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 >> 172.21.15.66:0/1125242267 conn(0x7f1d3006c200 msgr2=0x7f1d3006eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:35:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 shutdown_connections 2023-07-21T22:35:49.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:35:49.306+0000 7f1d383ff700 1 -- 172.21.15.66:0/1125242267 wait complete. 2023-07-21T22:35:49.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:49 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/786826465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:35:49.376 ERROR:tasks.fwd_scrub:rank 2 damaged: [{'damage_type': 'backtrace', 'id': 95207311, 'ino': 1099511628296, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORD'}, {'damage_type': 'backtrace', 'id': 249701508, 'ino': 1099511628309, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/COREL'}, {'damage_type': 'backtrace', 'id': 407744692, 'ino': 1099511628300, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/EXCEL'}, {'damage_type': 'backtrace', 'id': 505810414, 'ino': 1099511628352, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/SEED'}, {'damage_type': 'backtrace', 'id': 2000672907, 'ino': 1099511627785, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp'}, {'damage_type': 'backtrace', 'id': 2596043579, 'ino': 1099511628321, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/WORDPRO'}, {'damage_type': 'backtrace', 'id': 3367264304, 'ino': 1099511628329, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PARADOX'}, {'damage_type': 'backtrace', 'id': 3378681424, 'ino': 1099511628303, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PWRPNT'}, {'damage_type': 'backtrace', 'id': 3784312346, 'ino': 1099511628350, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/ACCESS'}, {'damage_type': 'backtrace', 'id': 4090255633, 'ino': 1099511628325, 'path': '/volumes/_nogroup/sv_1/d9a6ac6a-b684-4b83-bca0-bd809aa020ca/client.0/tmp/clients/client0/~dmtmp/PM'}] 2023-07-21T22:35:49.376 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-07-21T22:35:49.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:49 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/786826465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:35:49.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 46267 35.07 MB/sec warmup 76 sec latency 59.915 ms 2023-07-21T22:35:50.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:50 smithi104 ceph-mon[129021]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.3 MiB/s wr, 610 op/s 2023-07-21T22:35:50.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:50 smithi066 ceph-mon[126379]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.3 MiB/s wr, 610 op/s 2023-07-21T22:35:50.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:50 smithi149 ceph-mon[129268]: pgmap v449: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.3 MiB/s wr, 610 op/s 2023-07-21T22:35:50.935 INFO:tasks.workunit.client.0.smithi066.stdout: 10 47088 35.15 MB/sec warmup 77 sec latency 118.592 ms 2023-07-21T22:35:51.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 48141 35.43 MB/sec warmup 78 sec latency 27.408 ms 2023-07-21T22:35:52.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:52 smithi104 ceph-mon[129021]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 824 op/s 2023-07-21T22:35:52.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:52 smithi066 ceph-mon[126379]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 824 op/s 2023-07-21T22:35:52.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:52 smithi149 ceph-mon[129268]: pgmap v450: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 824 op/s 2023-07-21T22:35:52.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 49105 35.81 MB/sec warmup 79 sec latency 34.140 ms 2023-07-21T22:35:53.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 49705 35.63 MB/sec warmup 80 sec latency 187.633 ms 2023-07-21T22:35:54.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:54 smithi066 ceph-mon[126379]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.4 MiB/s wr, 450 op/s 2023-07-21T22:35:54.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:54 smithi149 ceph-mon[129268]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.4 MiB/s wr, 450 op/s 2023-07-21T22:35:54.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:54 smithi104 ceph-mon[129021]: pgmap v451: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.4 MiB/s wr, 450 op/s 2023-07-21T22:35:54.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 50534 35.75 MB/sec warmup 81 sec latency 248.453 ms 2023-07-21T22:35:55.936 INFO:tasks.workunit.client.0.smithi066.stdout: 10 50782 35.53 MB/sec warmup 82 sec latency 586.335 ms 2023-07-21T22:35:56.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:56 smithi066 ceph-mon[126379]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 623 op/s 2023-07-21T22:35:56.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:56 smithi149 ceph-mon[129268]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 623 op/s 2023-07-21T22:35:56.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:56 smithi104 ceph-mon[129021]: pgmap v452: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.8 MiB/s wr, 623 op/s 2023-07-21T22:35:56.941 INFO:tasks.workunit.client.0.smithi066.stdout: 10 50929 35.15 MB/sec warmup 83 sec latency 786.519 ms 2023-07-21T22:35:57.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:35:57 smithi066 ceph-mon[126379]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2023-07-21T22:35:57.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:35:57 smithi149 ceph-mon[129268]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2023-07-21T22:35:57.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:35:57 smithi104 ceph-mon[129021]: pgmap v453: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.9 MiB/s wr, 517 op/s 2023-07-21T22:35:57.939 INFO:tasks.workunit.client.0.smithi066.stdout: 10 51333 34.96 MB/sec warmup 84 sec latency 87.036 ms 2023-07-21T22:35:58.939 INFO:tasks.workunit.client.0.smithi066.stdout: 10 51902 34.92 MB/sec warmup 85 sec latency 70.823 ms 2023-07-21T22:35:59.940 INFO:tasks.workunit.client.0.smithi066.stdout: 10 52713 35.05 MB/sec warmup 86 sec latency 40.939 ms 2023-07-21T22:36:00.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:00 smithi104 ceph-mon[129021]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2023-07-21T22:36:00.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:00 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:00.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:00 smithi066 ceph-mon[126379]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2023-07-21T22:36:00.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:00 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:00.502 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:00 smithi149 ceph-mon[129268]: pgmap v454: 129 pgs: 129 active+clean; 1.1 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 439 op/s 2023-07-21T22:36:00.502 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:00 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:36:00.941 INFO:tasks.workunit.client.0.smithi066.stdout: 10 53223 34.97 MB/sec warmup 87 sec latency 376.489 ms 2023-07-21T22:36:01.941 INFO:tasks.workunit.client.0.smithi066.stdout: 10 53468 34.68 MB/sec warmup 88 sec latency 823.494 ms 2023-07-21T22:36:02.459 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:02 smithi066 ceph-mon[126379]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 896 op/s 2023-07-21T22:36:02.459 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:02 smithi066 ceph-mon[126379]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T22:36:02.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:02 smithi149 ceph-mon[129268]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 896 op/s 2023-07-21T22:36:02.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:02 smithi149 ceph-mon[129268]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T22:36:02.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:02 smithi104 ceph-mon[129021]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 896 op/s 2023-07-21T22:36:02.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:02 smithi104 ceph-mon[129021]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T22:36:02.940 INFO:tasks.workunit.client.0.smithi066.stdout: 10 53657 34.35 MB/sec warmup 89 sec latency 449.163 ms 2023-07-21T22:36:03.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:03 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:36:03.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:03 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:36:03.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:03 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:36:03.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:03 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:36:03.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:03 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:36:03.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:03 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:36:03.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:03 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:36:03.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:03 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:36:03.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:03 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:36:03.940 INFO:tasks.workunit.client.0.smithi066.stdout: 10 53676 34.00 MB/sec warmup 90 sec latency 430.222 ms 2023-07-21T22:36:04.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:04 smithi066 ceph-mon[126379]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 18 MiB/s wr, 819 op/s 2023-07-21T22:36:04.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:04 smithi149 ceph-mon[129268]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 18 MiB/s wr, 819 op/s 2023-07-21T22:36:04.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:04 smithi104 ceph-mon[129021]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 18 MiB/s wr, 819 op/s 2023-07-21T22:36:04.941 INFO:tasks.workunit.client.0.smithi066.stdout: 10 53790 33.74 MB/sec warmup 91 sec latency 250.728 ms 2023-07-21T22:36:05.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 54113 33.53 MB/sec warmup 92 sec latency 280.249 ms 2023-07-21T22:36:06.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:06 smithi066 ceph-mon[126379]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 20 MiB/s wr, 961 op/s 2023-07-21T22:36:06.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:06 smithi149 ceph-mon[129268]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 20 MiB/s wr, 961 op/s 2023-07-21T22:36:06.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:06 smithi104 ceph-mon[129021]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 20 MiB/s wr, 961 op/s 2023-07-21T22:36:06.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 55030 33.81 MB/sec warmup 93 sec latency 178.319 ms 2023-07-21T22:36:07.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 55812 33.80 MB/sec warmup 94 sec latency 53.923 ms 2023-07-21T22:36:08.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:08 smithi066 ceph-mon[126379]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 999 op/s 2023-07-21T22:36:08.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:08 smithi066 ceph-mon[126379]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T22:36:08.493 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:08 smithi149 ceph-mon[129268]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 999 op/s 2023-07-21T22:36:08.494 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:08 smithi149 ceph-mon[129268]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T22:36:08.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:08 smithi104 ceph-mon[129021]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 22 MiB/s wr, 999 op/s 2023-07-21T22:36:08.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:08 smithi104 ceph-mon[129021]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T22:36:08.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 56670 33.95 MB/sec warmup 95 sec latency 65.033 ms 2023-07-21T22:36:09.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 57637 34.17 MB/sec warmup 96 sec latency 49.216 ms 2023-07-21T22:36:10.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:10 smithi066 ceph-mon[126379]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.8 MiB/s wr, 565 op/s 2023-07-21T22:36:10.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:10 smithi149 ceph-mon[129268]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.8 MiB/s wr, 565 op/s 2023-07-21T22:36:10.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:10 smithi104 ceph-mon[129021]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.8 MiB/s wr, 565 op/s 2023-07-21T22:36:10.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 58365 34.28 MB/sec warmup 97 sec latency 55.377 ms 2023-07-21T22:36:11.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 59126 34.28 MB/sec warmup 98 sec latency 43.473 ms 2023-07-21T22:36:12.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:12 smithi066 ceph-mon[126379]: pgmap v462: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 747 op/s 2023-07-21T22:36:12.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:12 smithi149 ceph-mon[129268]: pgmap v462: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 747 op/s 2023-07-21T22:36:12.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:12 smithi104 ceph-mon[129021]: pgmap v462: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 747 op/s 2023-07-21T22:36:12.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 59826 34.30 MB/sec warmup 99 sec latency 71.458 ms 2023-07-21T22:36:13.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 60974 34.54 MB/sec warmup 100 sec latency 59.895 ms 2023-07-21T22:36:14.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:14 smithi066 ceph-mon[126379]: pgmap v463: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 740 op/s 2023-07-21T22:36:14.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:14 smithi149 ceph-mon[129268]: pgmap v463: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 740 op/s 2023-07-21T22:36:14.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:14 smithi104 ceph-mon[129021]: pgmap v463: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 11 MiB/s wr, 740 op/s 2023-07-21T22:36:14.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 61862 34.77 MB/sec warmup 101 sec latency 69.281 ms 2023-07-21T22:36:15.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 62400 34.68 MB/sec warmup 102 sec latency 115.546 ms 2023-07-21T22:36:16.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:16 smithi066 ceph-mon[126379]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.7 MiB/s wr, 586 op/s 2023-07-21T22:36:16.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:16 smithi149 ceph-mon[129268]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.7 MiB/s wr, 586 op/s 2023-07-21T22:36:16.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:16 smithi104 ceph-mon[129021]: pgmap v464: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.7 MiB/s wr, 586 op/s 2023-07-21T22:36:16.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 63331 34.80 MB/sec warmup 103 sec latency 49.158 ms 2023-07-21T22:36:17.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:17 smithi066 ceph-mon[126379]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.8 MiB/s wr, 560 op/s 2023-07-21T22:36:17.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:17 smithi149 ceph-mon[129268]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.8 MiB/s wr, 560 op/s 2023-07-21T22:36:17.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:17 smithi104 ceph-mon[129021]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.8 MiB/s wr, 560 op/s 2023-07-21T22:36:17.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 64000 34.76 MB/sec warmup 104 sec latency 112.053 ms 2023-07-21T22:36:18.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 64777 34.91 MB/sec warmup 105 sec latency 229.854 ms 2023-07-21T22:36:19.942 INFO:tasks.workunit.client.0.smithi066.stdout: 10 65490 34.97 MB/sec warmup 106 sec latency 253.401 ms 2023-07-21T22:36:20.370 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:20 smithi104 ceph-mon[129021]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2023-07-21T22:36:20.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:20 smithi066 ceph-mon[126379]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2023-07-21T22:36:20.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:20 smithi149 ceph-mon[129268]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.1 MiB/s wr, 510 op/s 2023-07-21T22:36:20.945 INFO:tasks.workunit.client.0.smithi066.stdout: 10 66024 34.82 MB/sec warmup 107 sec latency 80.076 ms 2023-07-21T22:36:21.947 INFO:tasks.workunit.client.0.smithi066.stdout: 10 66388 34.70 MB/sec warmup 108 sec latency 389.179 ms 2023-07-21T22:36:22.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:22 smithi104 ceph-mon[129021]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 676 op/s 2023-07-21T22:36:22.451 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:22 smithi066 ceph-mon[126379]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 676 op/s 2023-07-21T22:36:22.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:22 smithi149 ceph-mon[129268]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 676 op/s 2023-07-21T22:36:22.947 INFO:tasks.workunit.client.0.smithi066.stdout: 10 66970 34.64 MB/sec warmup 109 sec latency 193.097 ms 2023-07-21T22:36:23.947 INFO:tasks.workunit.client.0.smithi066.stdout: 10 67792 34.82 MB/sec warmup 110 sec latency 72.364 ms 2023-07-21T22:36:24.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:24 smithi104 ceph-mon[129021]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.2 MiB/s wr, 436 op/s 2023-07-21T22:36:24.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:24 smithi066 ceph-mon[126379]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.2 MiB/s wr, 436 op/s 2023-07-21T22:36:24.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:24 smithi149 ceph-mon[129268]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.2 MiB/s wr, 436 op/s 2023-07-21T22:36:24.947 INFO:tasks.workunit.client.0.smithi066.stdout: 10 68741 35.01 MB/sec warmup 111 sec latency 29.056 ms 2023-07-21T22:36:25.949 INFO:tasks.workunit.client.0.smithi066.stdout: 10 69262 34.85 MB/sec warmup 112 sec latency 48.787 ms 2023-07-21T22:36:26.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:26 smithi104 ceph-mon[129021]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 649 op/s 2023-07-21T22:36:26.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:26 smithi066 ceph-mon[126379]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 649 op/s 2023-07-21T22:36:26.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:26 smithi149 ceph-mon[129268]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 649 op/s 2023-07-21T22:36:26.954 INFO:tasks.workunit.client.0.smithi066.stdout: 10 70193 35.02 MB/sec warmup 113 sec latency 53.116 ms 2023-07-21T22:36:27.954 INFO:tasks.workunit.client.0.smithi066.stdout: 10 70223 34.71 MB/sec warmup 114 sec latency 878.939 ms 2023-07-21T22:36:28.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:28 smithi066 ceph-mon[126379]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 596 op/s 2023-07-21T22:36:28.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:28 smithi149 ceph-mon[129268]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 596 op/s 2023-07-21T22:36:28.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:28 smithi104 ceph-mon[129021]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 596 op/s 2023-07-21T22:36:28.954 INFO:tasks.workunit.client.0.smithi066.stdout: 10 70820 34.61 MB/sec warmup 115 sec latency 221.234 ms 2023-07-21T22:36:29.954 INFO:tasks.workunit.client.0.smithi066.stdout: 10 71359 34.64 MB/sec warmup 116 sec latency 82.474 ms 2023-07-21T22:36:30.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:30 smithi066 ceph-mon[126379]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 533 op/s 2023-07-21T22:36:30.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:30 smithi149 ceph-mon[129268]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 533 op/s 2023-07-21T22:36:30.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:30 smithi104 ceph-mon[129021]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 11 MiB/s wr, 533 op/s 2023-07-21T22:36:30.954 INFO:tasks.workunit.client.0.smithi066.stdout: 10 72220 34.78 MB/sec warmup 117 sec latency 41.926 ms 2023-07-21T22:36:31.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 72834 34.72 MB/sec warmup 118 sec latency 56.450 ms 2023-07-21T22:36:32.449 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:32 smithi066 ceph-mon[126379]: pgmap v472: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 20 MiB/s wr, 881 op/s 2023-07-21T22:36:32.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:32 smithi149 ceph-mon[129268]: pgmap v472: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 20 MiB/s wr, 881 op/s 2023-07-21T22:36:32.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:32 smithi104 ceph-mon[129021]: pgmap v472: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 20 MiB/s wr, 881 op/s 2023-07-21T22:36:32.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 73631 34.80 MB/sec warmup 119 sec latency 105.583 ms 2023-07-21T22:36:34.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:34 smithi066 ceph-mon[126379]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 671 op/s 2023-07-21T22:36:34.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:34 smithi149 ceph-mon[129268]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 671 op/s 2023-07-21T22:36:34.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:34 smithi104 ceph-mon[129021]: pgmap v473: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 16 MiB/s wr, 671 op/s 2023-07-21T22:36:34.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 75305 58.60 MB/sec execute 1 sec latency 97.385 ms 2023-07-21T22:36:35.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 75983 43.69 MB/sec execute 2 sec latency 81.459 ms 2023-07-21T22:36:36.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:36 smithi066 ceph-mon[126379]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 811 op/s 2023-07-21T22:36:36.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:36 smithi149 ceph-mon[129268]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 811 op/s 2023-07-21T22:36:36.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:36 smithi104 ceph-mon[129021]: pgmap v474: 129 pgs: 129 active+clean; 1.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 811 op/s 2023-07-21T22:36:36.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 76623 39.69 MB/sec execute 3 sec latency 124.680 ms 2023-07-21T22:36:37.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 77475 40.01 MB/sec execute 4 sec latency 115.370 ms 2023-07-21T22:36:38.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:38 smithi066 ceph-mon[126379]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 677 op/s 2023-07-21T22:36:38.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:38 smithi149 ceph-mon[129268]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 677 op/s 2023-07-21T22:36:38.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:38 smithi104 ceph-mon[129021]: pgmap v475: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 677 op/s 2023-07-21T22:36:38.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 78520 44.79 MB/sec execute 5 sec latency 111.769 ms 2023-07-21T22:36:39.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 79189 43.52 MB/sec execute 6 sec latency 75.571 ms 2023-07-21T22:36:40.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:40 smithi066 ceph-mon[126379]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 566 op/s 2023-07-21T22:36:40.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:40 smithi149 ceph-mon[129268]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 566 op/s 2023-07-21T22:36:40.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:40 smithi104 ceph-mon[129021]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 566 op/s 2023-07-21T22:36:40.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 79934 43.43 MB/sec execute 7 sec latency 47.725 ms 2023-07-21T22:36:41.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 80556 41.20 MB/sec execute 8 sec latency 132.880 ms 2023-07-21T22:36:42.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:42 smithi066 ceph-mon[126379]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 780 op/s 2023-07-21T22:36:42.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:42 smithi149 ceph-mon[129268]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 780 op/s 2023-07-21T22:36:42.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:42 smithi104 ceph-mon[129021]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 780 op/s 2023-07-21T22:36:42.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 81391 41.53 MB/sec execute 9 sec latency 60.814 ms 2023-07-21T22:36:43.955 INFO:tasks.workunit.client.0.smithi066.stdout: 10 82426 43.13 MB/sec execute 10 sec latency 71.054 ms 2023-07-21T22:36:44.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:44 smithi104 ceph-mon[129021]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.9 MiB/s wr, 432 op/s 2023-07-21T22:36:44.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:44 smithi066 ceph-mon[126379]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.9 MiB/s wr, 432 op/s 2023-07-21T22:36:44.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:44 smithi149 ceph-mon[129268]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.9 MiB/s wr, 432 op/s 2023-07-21T22:36:44.957 INFO:tasks.workunit.client.0.smithi066.stdout: 10 83024 42.32 MB/sec execute 11 sec latency 276.808 ms 2023-07-21T22:36:45.957 INFO:tasks.workunit.client.0.smithi066.stdout: 10 83752 42.14 MB/sec execute 12 sec latency 58.092 ms 2023-07-21T22:36:46.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:46 smithi104 ceph-mon[129021]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 608 op/s 2023-07-21T22:36:46.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:46 smithi066 ceph-mon[126379]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 608 op/s 2023-07-21T22:36:46.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:46 smithi149 ceph-mon[129268]: pgmap v479: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 608 op/s 2023-07-21T22:36:46.957 INFO:tasks.workunit.client.0.smithi066.stdout: 10 84402 41.16 MB/sec execute 13 sec latency 60.836 ms 2023-07-21T22:36:47.957 INFO:tasks.workunit.client.0.smithi066.stdout: 10 85369 42.31 MB/sec execute 14 sec latency 34.916 ms 2023-07-21T22:36:48.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:48 smithi104 ceph-mon[129021]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.6 MiB/s wr, 502 op/s 2023-07-21T22:36:48.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:48 smithi066 ceph-mon[126379]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.6 MiB/s wr, 502 op/s 2023-07-21T22:36:48.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:48 smithi149 ceph-mon[129268]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.6 MiB/s wr, 502 op/s 2023-07-21T22:36:48.957 INFO:tasks.workunit.client.0.smithi066.stdout: 10 86213 42.46 MB/sec execute 15 sec latency 79.942 ms 2023-07-21T22:36:49.958 INFO:tasks.workunit.client.0.smithi066.stdout: 10 87002 42.46 MB/sec execute 16 sec latency 49.554 ms 2023-07-21T22:36:50.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:50 smithi104 ceph-mon[129021]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.5 MiB/s wr, 424 op/s 2023-07-21T22:36:50.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:50 smithi066 ceph-mon[126379]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.5 MiB/s wr, 424 op/s 2023-07-21T22:36:50.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:50 smithi149 ceph-mon[129268]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.5 MiB/s wr, 424 op/s 2023-07-21T22:36:50.957 INFO:tasks.workunit.client.0.smithi066.stdout: 10 87827 42.19 MB/sec execute 17 sec latency 51.547 ms 2023-07-21T22:36:51.958 INFO:tasks.workunit.client.0.smithi066.stdout: 10 88797 43.36 MB/sec execute 18 sec latency 39.393 ms 2023-07-21T22:36:52.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:52 smithi104 ceph-mon[129021]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 636 op/s 2023-07-21T22:36:52.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:52 smithi066 ceph-mon[126379]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 636 op/s 2023-07-21T22:36:52.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:52 smithi149 ceph-mon[129268]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 636 op/s 2023-07-21T22:36:52.958 INFO:tasks.workunit.client.0.smithi066.stdout: 10 89450 42.76 MB/sec execute 19 sec latency 70.499 ms 2023-07-21T22:36:53.960 INFO:tasks.workunit.client.0.smithi066.stdout: 10 89530 40.84 MB/sec execute 20 sec latency 743.931 ms 2023-07-21T22:36:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:54 smithi149 ceph-mon[129268]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 423 op/s 2023-07-21T22:36:54.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:54 smithi104 ceph-mon[129021]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 423 op/s 2023-07-21T22:36:54.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:54 smithi066 ceph-mon[126379]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.0 MiB/s wr, 423 op/s 2023-07-21T22:36:54.959 INFO:tasks.workunit.client.0.smithi066.stdout: 10 89618 39.13 MB/sec execute 21 sec latency 1161.263 ms 2023-07-21T22:36:55.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:55 smithi149 ceph-mon[129268]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 684 op/s 2023-07-21T22:36:55.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:55 smithi104 ceph-mon[129021]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 684 op/s 2023-07-21T22:36:55.699 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:55 smithi066 ceph-mon[126379]: pgmap v484: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 684 op/s 2023-07-21T22:36:55.959 INFO:tasks.workunit.client.0.smithi066.stdout: 10 89963 38.04 MB/sec execute 22 sec latency 123.189 ms 2023-07-21T22:36:56.961 INFO:tasks.workunit.client.0.smithi066.stdout: 10 90365 37.35 MB/sec execute 23 sec latency 80.911 ms 2023-07-21T22:36:57.961 INFO:tasks.workunit.client.0.smithi066.stdout: 10 90961 36.91 MB/sec execute 24 sec latency 98.453 ms 2023-07-21T22:36:58.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:36:58 smithi066 ceph-mon[126379]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-21T22:36:58.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:36:58 smithi149 ceph-mon[129268]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-21T22:36:58.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:36:58 smithi104 ceph-mon[129021]: pgmap v485: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 17 MiB/s wr, 585 op/s 2023-07-21T22:36:58.962 INFO:tasks.workunit.client.0.smithi066.stdout: 10 91683 37.19 MB/sec execute 25 sec latency 304.506 ms 2023-07-21T22:36:59.962 INFO:tasks.workunit.client.0.smithi066.stdout: 10 91840 36.07 MB/sec execute 26 sec latency 576.062 ms 2023-07-21T22:37:00.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:00 smithi149 ceph-mon[129268]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 551 op/s 2023-07-21T22:37:00.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:00 smithi104 ceph-mon[129021]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 551 op/s 2023-07-21T22:37:00.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:00 smithi066 ceph-mon[126379]: pgmap v486: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 17 MiB/s wr, 551 op/s 2023-07-21T22:37:00.963 INFO:tasks.workunit.client.0.smithi066.stdout: 10 92137 35.43 MB/sec execute 27 sec latency 172.936 ms 2023-07-21T22:37:01.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:01 smithi104 ceph-mon[129021]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 945 op/s 2023-07-21T22:37:01.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:01 smithi066 ceph-mon[126379]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 945 op/s 2023-07-21T22:37:01.742 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:01 smithi149 ceph-mon[129268]: pgmap v487: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 25 MiB/s wr, 945 op/s 2023-07-21T22:37:01.964 INFO:tasks.workunit.client.0.smithi066.stdout: 10 92423 34.60 MB/sec execute 28 sec latency 455.081 ms 2023-07-21T22:37:02.964 INFO:tasks.workunit.client.0.smithi066.stdout: 10 92570 33.58 MB/sec execute 29 sec latency 329.408 ms 2023-07-21T22:37:03.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:03 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:37:03.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:03 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:37:03.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:03 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T22:37:03.964 INFO:tasks.workunit.client.0.smithi066.stdout: 10 92586 32.56 MB/sec execute 30 sec latency 399.517 ms 2023-07-21T22:37:04.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:04 smithi066 ceph-mon[126379]: pgmap v488: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 22 MiB/s wr, 733 op/s 2023-07-21T22:37:04.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:04 smithi149 ceph-mon[129268]: pgmap v488: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 22 MiB/s wr, 733 op/s 2023-07-21T22:37:04.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:04 smithi104 ceph-mon[129021]: pgmap v488: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 22 MiB/s wr, 733 op/s 2023-07-21T22:37:04.964 INFO:tasks.workunit.client.0.smithi066.stdout: 10 92620 31.61 MB/sec execute 31 sec latency 354.234 ms 2023-07-21T22:37:05.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:05 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:05.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:05 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:05.447 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:05 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:37:05.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:05 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:05.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:05 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:05.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:05 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:37:05.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:05 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T22:37:05.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:05 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T22:37:05.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:05 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' 2023-07-21T22:37:05.964 INFO:tasks.workunit.client.0.smithi066.stdout: 10 93160 31.55 MB/sec execute 32 sec latency 251.188 ms 2023-07-21T22:37:06.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:06 smithi066 ceph-mon[126379]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 1.01k op/s 2023-07-21T22:37:06.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:06 smithi149 ceph-mon[129268]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 1.01k op/s 2023-07-21T22:37:06.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:06 smithi104 ceph-mon[129021]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 23 MiB/s wr, 1.01k op/s 2023-07-21T22:37:06.965 INFO:tasks.workunit.client.0.smithi066.stdout: 10 93768 31.67 MB/sec execute 33 sec latency 171.813 ms 2023-07-21T22:37:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:07 smithi104 ceph-mon[129021]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T22:37:07.869 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:07 smithi104 ceph-mon[129021]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 877 op/s 2023-07-21T22:37:07.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:07 smithi066 ceph-mon[126379]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T22:37:07.946 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:07 smithi066 ceph-mon[126379]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 877 op/s 2023-07-21T22:37:07.965 INFO:tasks.workunit.client.0.smithi066.stdout: 10 94664 32.17 MB/sec execute 34 sec latency 58.056 ms 2023-07-21T22:37:07.989 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:07 smithi149 ceph-mon[129268]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T22:37:07.990 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:07 smithi149 ceph-mon[129268]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 877 op/s 2023-07-21T22:37:08.965 INFO:tasks.workunit.client.0.smithi066.stdout: 10 95441 32.46 MB/sec execute 35 sec latency 73.801 ms 2023-07-21T22:37:09.295 INFO:tasks.workunit.client.0.smithi066.stdout:[100886] write failed on handle 10089 (Resource temporarily unavailable) 2023-07-21T22:37:09.296 INFO:tasks.workunit.client.0.smithi066.stdout:Child failed with status 1 2023-07-21T22:37:09.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:37:09.315 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2023-07-21T22:37:09.316 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-07-21T22:37:10.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:10 smithi066 ceph-mon[126379]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 877 op/s 2023-07-21T22:37:10.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:10 smithi149 ceph-mon[129268]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 877 op/s 2023-07-21T22:37:10.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:10 smithi104 ceph-mon[129021]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 877 op/s 2023-07-21T22:37:10.623 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi066 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-07-21T22:37:10.745 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6cec760fdafc45fa8d9a524f1eaffce0 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi066 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2023-07-21T22:37:10.748 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-21T22:37:10.765 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-21T22:37:10.765 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T22:37:10.766 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:37:10.766 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-21T22:37:10.844 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-21T22:37:10.937 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T22:37:11.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 -- 172.21.15.66:0/183403828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 msgr2=0x7f6c5c10e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.143 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 --2- 172.21.15.66:0/183403828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c10e710 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6c5400b0a0 tx=0x7f6c5400b3b0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 -- 172.21.15.66:0/183403828 shutdown_connections 2023-07-21T22:37:11.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 --2- 172.21.15.66:0/183403828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c10e710 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 --2- 172.21.15.66:0/183403828 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5c072850 0x7f6c5c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.144 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 --2- 172.21.15.66:0/183403828 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c073160 0x7f6c5c10bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.097+0000 7f6c63fff700 1 -- 172.21.15.66:0/183403828 >> 172.21.15.66:0/183403828 conn(0x7f6c5c0fb5b0 msgr2=0x7f6c5c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.098+0000 7f6c63fff700 1 -- 172.21.15.66:0/183403828 shutdown_connections 2023-07-21T22:37:11.145 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.098+0000 7f6c63fff700 1 -- 172.21.15.66:0/183403828 wait complete. 2023-07-21T22:37:11.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 Processor -- start 2023-07-21T22:37:11.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 -- start start 2023-07-21T22:37:11.147 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 0x7f6c5c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5c073160 0x7f6c5c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c19cf20 con 0x7f6c5c073160 2023-07-21T22:37:11.148 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c19d090 con 0x7f6c5c10c330 2023-07-21T22:37:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.099+0000 7f6c63fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c19d250 con 0x7f6c5c072850 2023-07-21T22:37:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 0x7f6c5c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 0x7f6c5c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:37992/0 (socket says 172.21.15.66:37992) 2023-07-21T22:37:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c6259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.149 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 -- 172.21.15.66:0/2081487065 learned_addr learned my addr 172.21.15.66:0/2081487065 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c6159a700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5c073160 0x7f6c5c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 -- 172.21.15.66:0/2081487065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 msgr2=0x7f6c5c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 -- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5c073160 msgr2=0x7f6c5c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.150 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5c073160 0x7f6c5c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 -- 172.21.15.66:0/2081487065 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c54009d00 con 0x7f6c5c072850 2023-07-21T22:37:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c6259c700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.100+0000 7f6c61d9b700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 0x7f6c5c197f40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800e9a0 tx=0x7f6c5800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.101+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58009790 con 0x7f6c5c072850 2023-07-21T22:37:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.101+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c19d530 con 0x7f6c5c072850 2023-07-21T22:37:11.151 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.101+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c58010430 con 0x7f6c5c072850 2023-07-21T22:37:11.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.101+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58018680 con 0x7f6c5c072850 2023-07-21T22:37:11.152 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.101+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c1a4d70 con 0x7f6c5c072850 2023-07-21T22:37:11.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.102+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c04e210 con 0x7f6c5c072850 2023-07-21T22:37:11.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.104+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6c580105a0 con 0x7f6c5c072850 2023-07-21T22:37:11.153 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.105+0000 7f6c52ffd700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6c480a12c0 0x7f6c480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.105+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f6c58014070 con 0x7f6c5c072850 2023-07-21T22:37:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.105+0000 7f6c6159a700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6c480a12c0 0x7f6c480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.106+0000 7f6c6159a700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6c480a12c0 0x7f6c480a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c199550 tx=0x7f6c4c009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:11.154 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.108+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6c580962a0 con 0x7f6c5c072850 2023-07-21T22:37:11.390 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.345+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 --> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c5c0fd220 con 0x7f6c480a12c0 2023-07-21T22:37:11.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.356+0000 7f6c52ffd700 1 -- 172.21.15.66:0/2081487065 <== mgr.15483 v2:172.21.15.66:6800/3475780841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6c5c0fd220 con 0x7f6c480a12c0 2023-07-21T22:37:11.402 INFO:teuthology.orchestra.run.smithi066.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T22:21:29", "first": "2023-07-21T22:22:00", "last": "2023-07-21T22:37:00", "last_pruned": "2023-07-21T22:37:01", "created_count": 16, "pruned_count": 10, "active": true} 2023-07-21T22:37:11.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6c480a12c0 msgr2=0x7f6c480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.405 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6c480a12c0 0x7f6c480a3770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c199550 tx=0x7f6c4c009260 comp rx=0 tx=0).stop 2023-07-21T22:37:11.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 msgr2=0x7f6c5c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.406 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 0x7f6c5c197f40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800e9a0 tx=0x7f6c5800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 shutdown_connections 2023-07-21T22:37:11.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c5c10c330 0x7f6c5c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.407 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f6c480a12c0 0x7f6c480a3770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c5c072850 0x7f6c5c197f40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.408 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 --2- 172.21.15.66:0/2081487065 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f6c5c073160 0x7f6c5c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 >> 172.21.15.66:0/2081487065 conn(0x7f6c5c0fb5b0 msgr2=0x7f6c5c10e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:11.409 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 shutdown_connections 2023-07-21T22:37:11.410 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.359+0000 7f6c63fff700 1 -- 172.21.15.66:0/2081487065 wait complete. 2023-07-21T22:37:11.424 DEBUG:teuthology.orchestra.run.smithi066:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T22:37:11.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 -- 172.21.15.66:0/2273155812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 msgr2=0x7f541c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 --2- 172.21.15.66:0/2273155812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c105ee0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5410009990 tx=0x7f5410009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 -- 172.21.15.66:0/2273155812 shutdown_connections 2023-07-21T22:37:11.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 --2- 172.21.15.66:0/2273155812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c1004b0 0x7f541c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 --2- 172.21.15.66:0/2273155812 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c105ee0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 --2- 172.21.15.66:0/2273155812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c0ffb00 0x7f541c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 -- 172.21.15.66:0/2273155812 >> 172.21.15.66:0/2273155812 conn(0x7f541c0fb600 msgr2=0x7f541c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:11.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 -- 172.21.15.66:0/2273155812 shutdown_connections 2023-07-21T22:37:11.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.585+0000 7f54231ff700 1 -- 172.21.15.66:0/2273155812 wait complete. 2023-07-21T22:37:11.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.586+0000 7f54231ff700 1 Processor -- start 2023-07-21T22:37:11.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.586+0000 7f54231ff700 1 -- start start 2023-07-21T22:37:11.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f54231ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 0x7f541c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f54231ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c1004b0 0x7f541c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f54231ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c1a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f54231ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c1a4350 con 0x7f541c105b10 2023-07-21T22:37:11.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f54231ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c1a4490 con 0x7f541c1004b0 2023-07-21T22:37:11.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f54231ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c1a4600 con 0x7f541c0ffb00 2023-07-21T22:37:11.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f5420f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 0x7f541c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f5420f9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 0x7f541c0729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:38008/0 (socket says 172.21.15.66:38008) 2023-07-21T22:37:11.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f5420f9b700 1 -- 172.21.15.66:0/376126918 learned_addr learned my addr 172.21.15.66:0/376126918 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:11.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f542179c700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c1a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.587+0000 7f541bfff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c1004b0 0x7f541c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f542179c700 1 -- 172.21.15.66:0/376126918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 msgr2=0x7f541c0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f542179c700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 0x7f541c0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f542179c700 1 -- 172.21.15.66:0/376126918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c1004b0 msgr2=0x7f541c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f542179c700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c1004b0 0x7f541c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f542179c700 1 -- 172.21.15.66:0/376126918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54100095f0 con 0x7f541c105b10 2023-07-21T22:37:11.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f541bfff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c1004b0 0x7f541c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:11.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f5420f9b700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 0x7f541c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:11.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f542179c700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c1a3e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f541400efc0 tx=0x7f541400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:11.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5414018070 con 0x7f541c105b10 2023-07-21T22:37:11.639 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c1a4880 con 0x7f541c105b10 2023-07-21T22:37:11.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541400f040 con 0x7f541c105b10 2023-07-21T22:37:11.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541400cd40 con 0x7f541c105b10 2023-07-21T22:37:11.640 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.588+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c1a4de0 con 0x7f541c105b10 2023-07-21T22:37:11.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.590+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c0ff240 con 0x7f541c105b10 2023-07-21T22:37:11.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.592+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5414003a00 con 0x7f541c105b10 2023-07-21T22:37:11.641 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.593+0000 7f5419ffb700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f540c0a1340 0x7f540c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.593+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f54140ccbf0 con 0x7f541c105b10 2023-07-21T22:37:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.593+0000 7f5420f9b700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f540c0a1340 0x7f540c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:11.642 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.594+0000 7f5420f9b700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f540c0a1340 0x7f540c0a37f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5410006010 tx=0x7f541000b260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:11.643 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.598+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5414095c30 con 0x7f541c105b10 2023-07-21T22:37:11.883 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.838+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 --> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f541c0fc2d0 con 0x7f540c0a1340 2023-07-21T22:37:11.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.848+0000 7f5419ffb700 1 -- 172.21.15.66:0/376126918 <== mgr.15483 v2:172.21.15.66:6800/3475780841 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f541c0fc2d0 con 0x7f540c0a1340 2023-07-21T22:37:11.893 INFO:teuthology.orchestra.run.smithi066.stdout:/ 1M 6M3h 2023-07-21T22:37:11.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f540c0a1340 msgr2=0x7f540c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f540c0a1340 0x7f540c0a37f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5410006010 tx=0x7f541000b260 comp rx=0 tx=0).stop 2023-07-21T22:37:11.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 msgr2=0x7f541c1a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:11.896 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c1a3e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f541400efc0 tx=0x7f541400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 shutdown_connections 2023-07-21T22:37:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f540c0a1340 0x7f540c0a37f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f541c105b10 0x7f541c1a3e10 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f541c0ffb00 0x7f541c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.897 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 --2- 172.21.15.66:0/376126918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f541c1004b0 0x7f541c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:11.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 >> 172.21.15.66:0/376126918 conn(0x7f541c0fb600 msgr2=0x7f541c0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:11.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 shutdown_connections 2023-07-21T22:37:11.898 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:11.850+0000 7f54231ff700 1 -- 172.21.15.66:0/376126918 wait complete. 2023-07-21T22:37:11.915 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-21T22:37:11.931 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-21T22:37:11.931 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-07-21T22:37:11.940 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-21T22:37:11.958 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:37:12.169 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 -- 172.21.15.66:0/3776506364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 msgr2=0x7f0c640fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 --2- 172.21.15.66:0/3776506364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c640fff40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009990 tx=0x7f0c50009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 -- 172.21.15.66:0/3776506364 shutdown_connections 2023-07-21T22:37:12.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 --2- 172.21.15.66:0/3776506364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c64100480 0x7f0c6410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.170 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 --2- 172.21.15.66:0/3776506364 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c64105ae0 0x7f0c64105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 --2- 172.21.15.66:0/3776506364 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c640fff40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 -- 172.21.15.66:0/3776506364 >> 172.21.15.66:0/3776506364 conn(0x7f0c640fb590 msgr2=0x7f0c640fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:12.171 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 -- 172.21.15.66:0/3776506364 shutdown_connections 2023-07-21T22:37:12.172 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.124+0000 7f0c6abff700 1 -- 172.21.15.66:0/3776506364 wait complete. 2023-07-21T22:37:12.173 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.125+0000 7f0c6abff700 1 Processor -- start 2023-07-21T22:37:12.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.125+0000 7f0c6abff700 1 -- start start 2023-07-21T22:37:12.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.125+0000 7f0c6abff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c64072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.125+0000 7f0c6abff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c64100480 0x7f0c64072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.174 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6abff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c64105ae0 0x7f0c641a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6abff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64073580 con 0x7f0c640ffad0 2023-07-21T22:37:12.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6abff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641a42e0 con 0x7f0c64100480 2023-07-21T22:37:12.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6abff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641a4420 con 0x7f0c64105ae0 2023-07-21T22:37:12.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c64072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c64072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41320/0 (socket says 172.21.15.66:41320) 2023-07-21T22:37:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 -- 172.21.15.66:0/482215530 learned_addr learned my addr 172.21.15.66:0/482215530 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6919c700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c64105ae0 0x7f0c641a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c63fff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c64100480 0x7f0c64072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 -- 172.21.15.66:0/482215530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c64105ae0 msgr2=0x7f0c641a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c64105ae0 0x7f0c641a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 -- 172.21.15.66:0/482215530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c64100480 msgr2=0x7f0c64072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c64100480 0x7f0c64072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.126+0000 7f0c6899b700 1 -- 172.21.15.66:0/482215530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c500095f0 con 0x7f0c640ffad0 2023-07-21T22:37:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.127+0000 7f0c6899b700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c64072810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9a0 tx=0x7f0c5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:12.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.127+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58009790 con 0x7f0c640ffad0 2023-07-21T22:37:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.127+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c58010430 con 0x7f0c640ffad0 2023-07-21T22:37:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.127+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c641a4680 con 0x7f0c640ffad0 2023-07-21T22:37:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.127+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58018680 con 0x7f0c640ffad0 2023-07-21T22:37:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.127+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c641a4bb0 con 0x7f0c640ffad0 2023-07-21T22:37:12.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.128+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c64067300 con 0x7f0c640ffad0 2023-07-21T22:37:12.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.130+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0c580105a0 con 0x7f0c640ffad0 2023-07-21T22:37:12.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.132+0000 7f0c61ffb700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c540a1380 0x7f0c540a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.132+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0c58014070 con 0x7f0c640ffad0 2023-07-21T22:37:12.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.133+0000 7f0c63fff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c540a1380 0x7f0c540a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.133+0000 7f0c63fff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c540a1380 0x7f0c540a3830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c64074000 tx=0x7f0c50005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:12.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.136+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0c58051770 con 0x7f0c640ffad0 2023-07-21T22:37:12.195 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:12 smithi066 ceph-mon[126379]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 633 op/s 2023-07-21T22:37:12.395 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.350+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c641a4ee0 con 0x7f0c640ffad0 2023-07-21T22:37:12.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.352+0000 7f0c61ffb700 1 -- 172.21.15.66:0/482215530 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+25286 (secure 0 0 0) 0x7f0c58096250 con 0x7f0c640ffad0 2023-07-21T22:37:12.397 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:37:12.398 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":113,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:37:06.440755+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/4012457980":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/4235245168":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1497274919":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6801/1262827329":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6800/1262827329":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3171200415":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:37:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c540a1380 msgr2=0x7f0c540a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c540a1380 0x7f0c540a3830 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0c64074000 tx=0x7f0c50005fb0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 msgr2=0x7f0c64072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c64072810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9a0 tx=0x7f0c5800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 shutdown_connections 2023-07-21T22:37:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c540a1380 0x7f0c540a3830 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c64105ae0 0x7f0c641a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c640ffad0 0x7f0c64072810 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 --2- 172.21.15.66:0/482215530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c64100480 0x7f0c64072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 >> 172.21.15.66:0/482215530 conn(0x7f0c640fb590 msgr2=0x7f0c640fba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:12.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 shutdown_connections 2023-07-21T22:37:12.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.354+0000 7f0c6abff700 1 -- 172.21.15.66:0/482215530 wait complete. 2023-07-21T22:37:12.419 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:37:12.420 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:37:12.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:12 smithi149 ceph-mon[129268]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 633 op/s 2023-07-21T22:37:12.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:12 smithi104 ceph-mon[129021]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 633 op/s 2023-07-21T22:37:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.646+0000 7fe8577ff700 1 -- 172.21.15.66:0/3487198992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8500728b0 msgr2=0x7fe850072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.692 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.646+0000 7fe8577ff700 1 --2- 172.21.15.66:0/3487198992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8500728b0 0x7fe850072c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0098f0 tx=0x7fe84c009c00 comp rx=0 tx=0).stop 2023-07-21T22:37:12.693 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 -- 172.21.15.66:0/3487198992 shutdown_connections 2023-07-21T22:37:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 --2- 172.21.15.66:0/3487198992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe8500728b0 0x7fe850072c80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 --2- 172.21.15.66:0/3487198992 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe85010c310 0x7fe85010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 --2- 172.21.15.66:0/3487198992 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500731c0 0x7fe85010bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 -- 172.21.15.66:0/3487198992 >> 172.21.15.66:0/3487198992 conn(0x7fe8500fb5f0 msgr2=0x7fe8500fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:12.694 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 -- 172.21.15.66:0/3487198992 shutdown_connections 2023-07-21T22:37:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.647+0000 7fe8577ff700 1 -- 172.21.15.66:0/3487198992 wait complete. 2023-07-21T22:37:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.648+0000 7fe8577ff700 1 Processor -- start 2023-07-21T22:37:12.695 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.648+0000 7fe8577ff700 1 -- start start 2023-07-21T22:37:12.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.648+0000 7fe8577ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 0x7fe850197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.648+0000 7fe8577ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500731c0 0x7fe8501983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe8577ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85010c310 0x7fe85019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe8577ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019ce40 con 0x7fe8500731c0 2023-07-21T22:37:12.697 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe8577ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019cfb0 con 0x7fe85010c310 2023-07-21T22:37:12.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe8577ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85019d170 con 0x7fe8500728b0 2023-07-21T22:37:12.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 0x7fe850197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 0x7fe850197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:51528/0 (socket says 172.21.15.66:51528) 2023-07-21T22:37:12.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 -- 172.21.15.66:0/4001862665 learned_addr learned my addr 172.21.15.66:0/4001862665 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:12.698 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe855d9c700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85010c310 0x7fe85019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe854d9a700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500731c0 0x7fe8501983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 -- 172.21.15.66:0/4001862665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85010c310 msgr2=0x7fe85019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85010c310 0x7fe85019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 -- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500731c0 msgr2=0x7fe8501983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.699 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500731c0 0x7fe8501983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.649+0000 7fe85559b700 1 -- 172.21.15.66:0/4001862665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400095f0 con 0x7fe8500728b0 2023-07-21T22:37:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.650+0000 7fe85559b700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 0x7fe850197e60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009810 tx=0x7fe84c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.650+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c01c070 con 0x7fe8500728b0 2023-07-21T22:37:12.700 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.650+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c009520 con 0x7fe8500728b0 2023-07-21T22:37:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.650+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8501a4ab0 con 0x7fe8500728b0 2023-07-21T22:37:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.650+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe84c00bbd0 con 0x7fe8500728b0 2023-07-21T22:37:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.650+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0049c0 con 0x7fe8500728b0 2023-07-21T22:37:12.701 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.651+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe834005320 con 0x7fe8500728b0 2023-07-21T22:37:12.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.654+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe84c020070 con 0x7fe8500728b0 2023-07-21T22:37:12.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.655+0000 7fe8467fc700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe83c0a1340 0x7fe83c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:12.702 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.655+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe84c0ce0f0 con 0x7fe8500728b0 2023-07-21T22:37:12.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.655+0000 7fe854d9a700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe83c0a1340 0x7fe83c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:12.703 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.656+0000 7fe854d9a700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe83c0a1340 0x7fe83c0a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe850199470 tx=0x7fe840009310 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:12.704 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.659+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe84c097130 con 0x7fe8500728b0 2023-07-21T22:37:12.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.930+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe834005f70 con 0x7fe8500728b0 2023-07-21T22:37:12.977 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:37:12.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.931+0000 7fe8467fc700 1 -- 172.21.15.66:0/4001862665 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe84c0270a0 con 0x7fe8500728b0 2023-07-21T22:37:12.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.938+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe83c0a1340 msgr2=0x7fe83c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.938+0000 7fe8577ff700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe83c0a1340 0x7fe83c0a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe850199470 tx=0x7fe840009310 comp rx=0 tx=0).stop 2023-07-21T22:37:12.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.938+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 msgr2=0x7fe850197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:12.984 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.938+0000 7fe8577ff700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 0x7fe850197e60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009810 tx=0x7fe84c00fa00 comp rx=0 tx=0).stop 2023-07-21T22:37:12.987 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.942+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 shutdown_connections 2023-07-21T22:37:12.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.942+0000 7fe8577ff700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe85010c310 0x7fe85019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.942+0000 7fe8577ff700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fe83c0a1340 0x7fe83c0a37f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.942+0000 7fe8577ff700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8500728b0 0x7fe850197e60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.988 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.942+0000 7fe8577ff700 1 --2- 172.21.15.66:0/4001862665 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fe8500731c0 0x7fe8501983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:12.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.942+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 >> 172.21.15.66:0/4001862665 conn(0x7fe8500fb5f0 msgr2=0x7fe85010e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:12.989 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.944+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 shutdown_connections 2023-07-21T22:37:12.990 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:12.944+0000 7fe8577ff700 1 -- 172.21.15.66:0/4001862665 wait complete. 2023-07-21T22:37:13.023 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:37:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 -- 172.21.15.66:0/2343484256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00fe820 msgr2=0x7fb5a00febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/2343484256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00fe820 0x7fb5a00febf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009990 tx=0x7fb59c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 -- 172.21.15.66:0/2343484256 shutdown_connections 2023-07-21T22:37:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/2343484256 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a010c3d0 0x7fb5a010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/2343484256 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00fe820 0x7fb5a00febf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/2343484256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.205+0000 7fb5a81ff700 1 -- 172.21.15.66:0/2343484256 >> 172.21.15.66:0/2343484256 conn(0x7fb5a00fa4b0 msgr2=0x7fb5a00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:13.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.206+0000 7fb5a81ff700 1 -- 172.21.15.66:0/2343484256 shutdown_connections 2023-07-21T22:37:13.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.206+0000 7fb5a81ff700 1 -- 172.21.15.66:0/2343484256 wait complete. 2023-07-21T22:37:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.206+0000 7fb5a81ff700 1 Processor -- start 2023-07-21T22:37:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 -- start start 2023-07-21T22:37:13.255 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a00fe820 0x7fb5a0197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5a010c3d0 0x7fb5a019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019ce50 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019cfc0 con 0x7fb5a010c3d0 2023-07-21T22:37:13.256 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a81ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a019d180 con 0x7fb5a00fe820 2023-07-21T22:37:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a579a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a579a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41360/0 (socket says 172.21.15.66:41360) 2023-07-21T22:37:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a579a700 1 -- 172.21.15.66:0/3823027267 learned_addr learned my addr 172.21.15.66:0/3823027267 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:13.257 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a5f9b700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a00fe820 0x7fb5a0197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.207+0000 7fb5a679c700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5a010c3d0 0x7fb5a019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a579a700 1 -- 172.21.15.66:0/3823027267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a00fe820 msgr2=0x7fb5a0197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a579a700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a00fe820 0x7fb5a0197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a579a700 1 -- 172.21.15.66:0/3823027267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5a010c3d0 msgr2=0x7fb5a019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.258 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a579a700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5a010c3d0 0x7fb5a019c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a579a700 1 -- 172.21.15.66:0/3823027267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0095f0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a579a700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a01983b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb59000e9f0 tx=0x7fb59000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590004a30 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5900054e0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.259 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a019d460 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59000cd00 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.208+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a01a4ca0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.260 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.209+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a01075f0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.212+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb590004fa0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.261 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.213+0000 7fb596ffd700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb58c0a1340 0x7fb58c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.213+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb590014070 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.213+0000 7fb5a5f9b700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb58c0a1340 0x7fb58c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.214+0000 7fb5a5f9b700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb58c0a1340 0x7fb58c0a37f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00b2e0 tx=0x7fb59c005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:13.262 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.217+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb5900962d0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:13 smithi066 ceph-mon[126379]: from='client.27598 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:37:13.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:13 smithi066 ceph-mon[126379]: from='client.17559 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:37:13.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/482215530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:13.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:13 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4001862665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:37:13.476 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.431+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5a002c910 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.477 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.431+0000 7fb596ffd700 1 -- 172.21.15.66:0/3823027267 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5900998f0 con 0x7fb5a00ff1c0 2023-07-21T22:37:13.477 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:37:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.433+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb58c0a1340 msgr2=0x7fb58c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.433+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb58c0a1340 0x7fb58c0a37f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00b2e0 tx=0x7fb59c005fb0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.479 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 msgr2=0x7fb5a01983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a01983b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb59000e9f0 tx=0x7fb59000ed00 comp rx=0 tx=0).stop 2023-07-21T22:37:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 shutdown_connections 2023-07-21T22:37:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5a010c3d0 0x7fb5a019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fb58c0a1340 0x7fb58c0a37f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb5a00fe820 0x7fb5a0197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 --2- 172.21.15.66:0/3823027267 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fb5a00ff1c0 0x7fb5a01983b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 >> 172.21.15.66:0/3823027267 conn(0x7fb5a00fa4b0 msgr2=0x7fb5a010e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:13.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 shutdown_connections 2023-07-21T22:37:13.482 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.434+0000 7fb5a81ff700 1 -- 172.21.15.66:0/3823027267 wait complete. 2023-07-21T22:37:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:13 smithi149 ceph-mon[129268]: from='client.27598 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:37:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:13 smithi149 ceph-mon[129268]: from='client.17559 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:37:13.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/482215530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:13.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:13 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4001862665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:37:13.498 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:37:13.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:13 smithi104 ceph-mon[129021]: from='client.27598 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:37:13.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:13 smithi104 ceph-mon[129021]: from='client.17559 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T22:37:13.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/482215530' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:13.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:13 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4001862665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:37:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 -- 172.21.15.66:0/2146932727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc1015b0 msgr2=0x7f94cc109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.714 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 --2- 172.21.15.66:0/2146932727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc1015b0 0x7f94cc109aa0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009990 tx=0x7f94c0009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 -- 172.21.15.66:0/2146932727 shutdown_connections 2023-07-21T22:37:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 --2- 172.21.15.66:0/2146932727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94cc10a070 0x7f94cc10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 --2- 172.21.15.66:0/2146932727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94cc100ca0 0x7f94cc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.715 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 --2- 172.21.15.66:0/2146932727 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc1015b0 0x7f94cc109aa0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 -- 172.21.15.66:0/2146932727 >> 172.21.15.66:0/2146932727 conn(0x7f94cc0fa4b0 msgr2=0x7f94cc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 -- 172.21.15.66:0/2146932727 shutdown_connections 2023-07-21T22:37:13.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.668+0000 7f94d25ff700 1 -- 172.21.15.66:0/2146932727 wait complete. 2023-07-21T22:37:13.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.669+0000 7f94d25ff700 1 Processor -- start 2023-07-21T22:37:13.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 -- start start 2023-07-21T22:37:13.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 0x7f94cc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94cc1015b0 0x7f94cc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94cc10a070 0x7f94cc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc19ce00 con 0x7f94cc100ca0 2023-07-21T22:37:13.719 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc19cf70 con 0x7f94cc1015b0 2023-07-21T22:37:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d25ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc19d130 con 0x7f94cc10a070 2023-07-21T22:37:13.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94cbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 0x7f94cc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94cbfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 0x7f94cc197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41374/0 (socket says 172.21.15.66:41374) 2023-07-21T22:37:13.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94cbfff700 1 -- 172.21.15.66:0/28186326 learned_addr learned my addr 172.21.15.66:0/28186326 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:13.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94d0b9c700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94cc10a070 0x7f94cc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.670+0000 7f94cb7fe700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94cc1015b0 0x7f94cc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94cbfff700 1 -- 172.21.15.66:0/28186326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94cc10a070 msgr2=0x7f94cc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94cbfff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94cc10a070 0x7f94cc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94cbfff700 1 -- 172.21.15.66:0/28186326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94cc1015b0 msgr2=0x7f94cc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94cbfff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94cc1015b0 0x7f94cc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94cbfff700 1 -- 172.21.15.66:0/28186326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c00095f0 con 0x7f94cc100ca0 2023-07-21T22:37:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94cbfff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 0x7f94cc197e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00e9a0 tx=0x7f94bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc009790 con 0x7f94cc100ca0 2023-07-21T22:37:13.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94bc010430 con 0x7f94cc100ca0 2023-07-21T22:37:13.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc018680 con 0x7f94cc100ca0 2023-07-21T22:37:13.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.671+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc19d410 con 0x7f94cc100ca0 2023-07-21T22:37:13.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.672+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc1a4c50 con 0x7f94cc100ca0 2023-07-21T22:37:13.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.673+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc067290 con 0x7f94cc100ca0 2023-07-21T22:37:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.674+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f94bc0105a0 con 0x7f94cc100ca0 2023-07-21T22:37:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.676+0000 7f94c97fa700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f94b40a1340 0x7f94b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:13.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.676+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f94bc014070 con 0x7f94cc100ca0 2023-07-21T22:37:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.676+0000 7f94cb7fe700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f94b40a1340 0x7f94b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.677+0000 7f94cb7fe700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f94b40a1340 0x7f94b40a37f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94cc199430 tx=0x7f94c0005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:13.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.680+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f94bc051770 con 0x7f94cc100ca0 2023-07-21T22:37:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.896+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f94cc1a4f10 con 0x7f94cc100ca0 2023-07-21T22:37:13.942 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.897+0000 7f94c97fa700 1 -- 172.21.15.66:0/28186326 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94bc0962a0 con 0x7f94cc100ca0 2023-07-21T22:37:13.943 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:37:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.899+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f94b40a1340 msgr2=0x7f94b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.899+0000 7f94d25ff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f94b40a1340 0x7f94b40a37f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94cc199430 tx=0x7f94c0005fb0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.899+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 msgr2=0x7f94cc197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:13.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.899+0000 7f94d25ff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 0x7f94cc197e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00e9a0 tx=0x7f94bc00ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 shutdown_connections 2023-07-21T22:37:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f94b40a1340 0x7f94b40a37f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94cc10a070 0x7f94cc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f94cc100ca0 0x7f94cc197e20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 --2- 172.21.15.66:0/28186326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94cc1015b0 0x7f94cc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 >> 172.21.15.66:0/28186326 conn(0x7f94cc0fa4b0 msgr2=0x7f94cc10c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 shutdown_connections 2023-07-21T22:37:13.947 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:13.900+0000 7f94d25ff700 1 -- 172.21.15.66:0/28186326 wait complete. 2023-07-21T22:37:13.964 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:37:14.175 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 -- 172.21.15.66:0/246150920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc105ae0 msgr2=0x7f62fc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 --2- 172.21.15.66:0/246150920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc105ae0 0x7f62fc105eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009990 tx=0x7f62ec009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 -- 172.21.15.66:0/246150920 shutdown_connections 2023-07-21T22:37:14.176 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 --2- 172.21.15.66:0/246150920 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc100480 0x7f62fc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 --2- 172.21.15.66:0/246150920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc105ae0 0x7f62fc105eb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 --2- 172.21.15.66:0/246150920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62fc0ffad0 0x7f62fc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 -- 172.21.15.66:0/246150920 >> 172.21.15.66:0/246150920 conn(0x7f62fc0fb5f0 msgr2=0x7f62fc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:14.177 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 -- 172.21.15.66:0/246150920 shutdown_connections 2023-07-21T22:37:14.178 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.130+0000 7f63011ff700 1 -- 172.21.15.66:0/246150920 wait complete. 2023-07-21T22:37:14.179 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.131+0000 7f63011ff700 1 Processor -- start 2023-07-21T22:37:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.131+0000 7f63011ff700 1 -- start start 2023-07-21T22:37:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f63011ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 0x7f62fc19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f63011ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc100480 0x7f62fc19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f63011ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62fc105ae0 0x7f62fc195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.180 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f63011ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc1960a0 con 0x7f62fc0ffad0 2023-07-21T22:37:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f63011ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc196210 con 0x7f62fc105ae0 2023-07-21T22:37:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f63011ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc1963d0 con 0x7f62fc100480 2023-07-21T22:37:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62fad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 0x7f62fc19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.181 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62fad9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 0x7f62fc19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41398/0 (socket says 172.21.15.66:41398) 2023-07-21T22:37:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62fad9d700 1 -- 172.21.15.66:0/1892199429 learned_addr learned my addr 172.21.15.66:0/1892199429 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62f3fff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc100480 0x7f62fc19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62fb59e700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62fc105ae0 0x7f62fc195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62fad9d700 1 -- 172.21.15.66:0/1892199429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc100480 msgr2=0x7f62fc19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.182 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.132+0000 7f62fad9d700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc100480 0x7f62fc19bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62fad9d700 1 -- 172.21.15.66:0/1892199429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62fc105ae0 msgr2=0x7f62fc195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62fad9d700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62fc105ae0 0x7f62fc195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62fad9d700 1 -- 172.21.15.66:0/1892199429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4009c40 con 0x7f62fc0ffad0 2023-07-21T22:37:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62fad9d700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 0x7f62fc19b9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005910 tx=0x7f62ec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:14.183 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec01d070 con 0x7f62fc0ffad0 2023-07-21T22:37:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec022470 con 0x7f62fc0ffad0 2023-07-21T22:37:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec0095f0 con 0x7f62fc0ffad0 2023-07-21T22:37:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec017650 con 0x7f62fc0ffad0 2023-07-21T22:37:14.184 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.133+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc1a27d0 con 0x7f62fc0ffad0 2023-07-21T22:37:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.134+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc067290 con 0x7f62fc0ffad0 2023-07-21T22:37:14.185 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.136+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f62ec00fad0 con 0x7f62fc0ffad0 2023-07-21T22:37:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.138+0000 7f62f8d99700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f62e80a12c0 0x7f62e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.138+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f62ec0ce690 con 0x7f62fc0ffad0 2023-07-21T22:37:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.138+0000 7f62f3fff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f62e80a12c0 0x7f62e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.186 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.139+0000 7f62f3fff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f62e80a12c0 0x7f62e80a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f62fc105260 tx=0x7f62e40073d0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:14.187 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.141+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f62ec097730 con 0x7f62fc0ffad0 2023-07-21T22:37:14.396 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.351+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f62fc1a2a20 con 0x7f62fc0ffad0 2023-07-21T22:37:14.397 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.352+0000 7f62f8d99700 1 -- 172.21.15.66:0/1892199429 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v114) v1 ==== 103+0+11 (secure 0 0 0) 0x7f62ec09ad50 con 0x7f62fc0ffad0 2023-07-21T22:37:14.397 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:37:14.399 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f62e80a12c0 msgr2=0x7f62e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f62e80a12c0 0x7f62e80a3770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f62fc105260 tx=0x7f62e40073d0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 msgr2=0x7f62fc19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.400 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 0x7f62fc19b9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005910 tx=0x7f62ec005740 comp rx=0 tx=0).stop 2023-07-21T22:37:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 shutdown_connections 2023-07-21T22:37:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f62fc105ae0 0x7f62fc195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f62e80a12c0 0x7f62e80a3770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f62fc0ffad0 0x7f62fc19b9c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.401 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 --2- 172.21.15.66:0/1892199429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62fc100480 0x7f62fc19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 >> 172.21.15.66:0/1892199429 conn(0x7f62fc0fb5f0 msgr2=0x7f62fc0fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 shutdown_connections 2023-07-21T22:37:14.402 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.354+0000 7f63011ff700 1 -- 172.21.15.66:0/1892199429 wait complete. 2023-07-21T22:37:14.417 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:14 smithi066 ceph-mon[126379]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 633 op/s 2023-07-21T22:37:14.417 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:14 smithi066 ceph-mon[126379]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T22:37:14.417 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3823027267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:37:14.418 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:14 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/28186326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:37:14.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-21T22:37:14.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:14 smithi149 ceph-mon[129268]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 633 op/s 2023-07-21T22:37:14.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:14 smithi149 ceph-mon[129268]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T22:37:14.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3823027267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:37:14.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:14 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/28186326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:37:14.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:14 smithi104 ceph-mon[129021]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.3 MiB/s wr, 633 op/s 2023-07-21T22:37:14.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:14 smithi104 ceph-mon[129021]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T22:37:14.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3823027267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:37:14.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:14 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/28186326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:37:14.627 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 -- 172.21.15.66:0/3942330963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 msgr2=0x7f18100fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 --2- 172.21.15.66:0/3942330963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f18100fff40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f17fc009990 tx=0x7f17fc009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 -- 172.21.15.66:0/3942330963 shutdown_connections 2023-07-21T22:37:14.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 --2- 172.21.15.66:0/3942330963 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f181010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.628 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 --2- 172.21.15.66:0/3942330963 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1810105ae0 0x7f1810105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 --2- 172.21.15.66:0/3942330963 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f18100fff40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 -- 172.21.15.66:0/3942330963 >> 172.21.15.66:0/3942330963 conn(0x7f18100fb590 msgr2=0x7f18100fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:14.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 -- 172.21.15.66:0/3942330963 shutdown_connections 2023-07-21T22:37:14.629 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.582+0000 7f1816fff700 1 -- 172.21.15.66:0/3942330963 wait complete. 2023-07-21T22:37:14.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.583+0000 7f1816fff700 1 Processor -- start 2023-07-21T22:37:14.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 -- start start 2023-07-21T22:37:14.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f1810197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.631 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f18101982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1810105ae0 0x7f181019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181019cd70 con 0x7f1810100480 2023-07-21T22:37:14.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181019cee0 con 0x7f18100ffad0 2023-07-21T22:37:14.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1816fff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181019d0a0 con 0x7f1810105ae0 2023-07-21T22:37:14.632 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f180ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f18101982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f1814d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f1810197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f180ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f18101982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:41412/0 (socket says 172.21.15.66:41412) 2023-07-21T22:37:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f180ffff700 1 -- 172.21.15.66:0/3022337275 learned_addr learned my addr 172.21.15.66:0/3022337275 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.584+0000 7f181559c700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1810105ae0 0x7f181019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.633 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180ffff700 1 -- 172.21.15.66:0/3022337275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1810105ae0 msgr2=0x7f181019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180ffff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1810105ae0 0x7f181019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180ffff700 1 -- 172.21.15.66:0/3022337275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 msgr2=0x7f1810197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180ffff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f1810197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180ffff700 1 -- 172.21.15.66:0/3022337275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0095f0 con 0x7f1810100480 2023-07-21T22:37:14.634 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f1814d9b700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f1810197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180ffff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f18101982d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005910 tx=0x7f17fc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc01d070 con 0x7f1810100480 2023-07-21T22:37:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17fc022470 con 0x7f1810100480 2023-07-21T22:37:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc017650 con 0x7f1810100480 2023-07-21T22:37:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181019d320 con 0x7f1810100480 2023-07-21T22:37:14.635 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.585+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181019d5a0 con 0x7f1810100480 2023-07-21T22:37:14.636 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.586+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1810067300 con 0x7f1810100480 2023-07-21T22:37:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.588+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f17fc0225e0 con 0x7f1810100480 2023-07-21T22:37:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.590+0000 7f180dffb700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18000a1300 0x7f18000a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.590+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f17fc0ce980 con 0x7f1810100480 2023-07-21T22:37:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.590+0000 7f1814d9b700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18000a1300 0x7f18000a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:14.637 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.591+0000 7f1814d9b700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18000a1300 0x7f18000a37b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1804009fd0 tx=0x7f1804009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:14.638 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.591+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f17fc097a20 con 0x7f1810100480 2023-07-21T22:37:14.908 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.863+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f181019d4b0 con 0x7f1810100480 2023-07-21T22:37:14.919 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.874+0000 7f180dffb700 1 -- 172.21.15.66:0/3022337275 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+733774 (secure 0 0 0) 0x7f17fc09b040 con 0x7f1810100480 2023-07-21T22:37:14.928 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:37:14.929 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:37:14.929 INFO:teuthology.orchestra.run.smithi066.stdout: "epoch": 26, 2023-07-21T22:37:14.929 INFO:teuthology.orchestra.run.smithi066.stdout: "active_gid": 15483, 2023-07-21T22:37:14.929 INFO:teuthology.orchestra.run.smithi066.stdout: "active_name": "x", 2023-07-21T22:37:14.930 INFO:teuthology.orchestra.run.smithi066.stdout: "active_addrs": { 2023-07-21T22:37:14.930 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:14.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:14.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:14.930 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:6800", 2023-07-21T22:37:14.931 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3475780841 2023-07-21T22:37:14.931 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.931 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:14.931 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v1", 2023-07-21T22:37:14.931 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:6801", 2023-07-21T22:37:14.931 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3475780841 2023-07-21T22:37:14.932 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:14.932 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:14.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.932 INFO:teuthology.orchestra.run.smithi066.stdout: "active_addr": "172.21.15.66:6801/3475780841", 2023-07-21T22:37:14.932 INFO:teuthology.orchestra.run.smithi066.stdout: "active_change": "2023-07-21T22:21:28.704308+0000", 2023-07-21T22:37:14.933 INFO:teuthology.orchestra.run.smithi066.stdout: "active_mgr_features": 4540138322906710015, 2023-07-21T22:37:14.933 INFO:teuthology.orchestra.run.smithi066.stdout: "available": true, 2023-07-21T22:37:14.933 INFO:teuthology.orchestra.run.smithi066.stdout: "standbys": [ 2023-07-21T22:37:14.933 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:14.933 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 25522, 2023-07-21T22:37:14.934 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "z", 2023-07-21T22:37:14.934 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr_features": 4540138322906710015, 2023-07-21T22:37:14.934 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2023-07-21T22:37:14.934 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:14.934 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2023-07-21T22:37:14.935 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:14.935 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:14.935 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:14.935 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:14.936 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:14.936 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:14.936 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.936 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.936 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:14.937 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.937 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.937 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:37:14.937 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.938 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.938 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:14.938 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:14.939 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.939 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.939 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.939 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:14.939 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.940 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.940 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:14.940 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:14.940 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:14.940 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:14.941 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:14.941 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:14.941 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:14.941 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:14.941 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:14.942 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.942 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.943 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:14.943 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:14.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:14.943 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:14.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.944 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.944 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.945 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:14.945 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.945 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.945 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.945 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.945 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:14.946 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:14.946 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.946 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.947 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:14.947 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.947 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.947 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:14.947 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:14.948 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:14.948 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:14.948 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:14.948 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:14.949 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:14.949 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:14.949 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:14.949 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.950 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.950 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:14.950 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:14.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:14.951 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:14.951 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.952 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.952 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:14.952 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.952 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.952 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2023-07-21T22:37:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2023-07-21T22:37:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.953 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.954 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.954 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:14.954 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.955 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.955 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:37:14.955 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.955 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.956 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2023-07-21T22:37:14.956 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2023-07-21T22:37:14.957 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.957 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.957 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.957 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2023-07-21T22:37:14.957 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.958 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.958 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.958 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2023-07-21T22:37:14.958 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.958 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.959 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.959 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.959 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2023-07-21T22:37:14.959 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2023-07-21T22:37:14.959 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.960 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.960 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.960 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:14.960 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.961 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.961 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.961 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2023-07-21T22:37:14.961 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.961 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.962 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.962 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2023-07-21T22:37:14.962 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2023-07-21T22:37:14.962 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.963 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.963 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:14.963 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.963 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.964 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.964 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2023-07-21T22:37:14.964 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.964 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.964 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.965 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2023-07-21T22:37:14.965 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2023-07-21T22:37:14.965 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:14.965 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.966 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2023-07-21T22:37:14.966 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.966 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.966 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.967 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2023-07-21T22:37:14.967 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.968 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.968 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2023-07-21T22:37:14.968 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2023-07-21T22:37:14.968 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.969 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:14.969 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.969 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.970 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.970 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:37:14.970 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.970 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.971 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2023-07-21T22:37:14.971 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2023-07-21T22:37:14.971 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:14.971 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.972 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.972 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:14.972 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.972 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.973 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.973 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:37:14.973 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.973 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.973 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.974 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2023-07-21T22:37:14.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2023-07-21T22:37:14.974 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.974 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.975 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:14.975 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2023-07-21T22:37:14.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.976 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:14.977 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:14.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.978 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:14.978 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2023-07-21T22:37:14.979 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:14.979 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:14.979 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:14.980 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2023-07-21T22:37:14.980 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2023-07-21T22:37:14.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:14.981 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.982 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:14.982 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.983 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:37:14.983 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.983 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.983 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.984 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2023-07-21T22:37:14.984 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2023-07-21T22:37:14.984 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.984 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2023-07-21T22:37:14.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.985 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.985 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.986 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:37:14.986 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:37:14.986 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.986 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.986 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.987 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2023-07-21T22:37:14.987 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2023-07-21T22:37:14.987 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:14.987 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.987 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.987 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:14.988 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-07-21T22:37:14.988 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-07-21T22:37:14.988 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.988 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:37:14.989 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:37:14.989 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.989 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.989 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.989 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2023-07-21T22:37:14.989 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:37:14.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:14.990 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.990 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.990 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2023-07-21T22:37:14.990 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:14.991 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2023-07-21T22:37:14.991 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.991 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:37:14.991 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:14.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.992 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.992 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2023-07-21T22:37:14.992 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2023-07-21T22:37:14.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:37:14.993 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:14.993 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:14.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.994 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:37:14.994 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:37:14.994 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.994 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.994 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.995 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2023-07-21T22:37:14.995 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2023-07-21T22:37:14.995 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:14.995 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.996 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.996 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2023-07-21T22:37:14.996 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2023-07-21T22:37:14.996 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2023-07-21T22:37:14.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:14.997 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:37:14.997 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:37:14.997 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:14.997 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:14.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:14.998 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2023-07-21T22:37:14.998 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2023-07-21T22:37:14.998 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:14.998 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:14.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:14.999 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2023-07-21T22:37:14.999 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.000 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.000 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:37:15.000 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:37:15.000 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.001 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.001 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2023-07-21T22:37:15.001 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2023-07-21T22:37:15.002 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:15.002 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.002 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.002 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:15.002 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-07-21T22:37:15.003 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-07-21T22:37:15.003 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.003 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:37:15.003 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:37:15.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.004 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.004 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.004 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.004 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.004 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.005 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.005 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.005 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.005 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.005 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.006 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.006 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.006 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.006 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.006 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.007 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.007 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.007 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.007 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.007 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.008 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.008 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.008 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.008 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.009 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.009 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.009 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.010 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.010 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.010 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.011 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.011 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.011 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.011 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.011 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.012 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.012 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.012 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.012 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.012 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.013 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.013 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.013 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.013 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.013 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.014 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.014 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.014 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.014 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.014 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.015 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.015 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.015 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.015 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.016 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.016 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.016 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.016 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.016 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.017 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.017 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.017 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.017 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.017 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2023-07-21T22:37:15.018 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2023-07-21T22:37:15.019 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.019 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.019 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.019 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:15.019 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.020 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.020 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.020 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:37:15.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.020 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.021 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.021 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-07-21T22:37:15.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-07-21T22:37:15.021 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2023-07-21T22:37:15.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.022 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.023 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2023-07-21T22:37:15.023 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2023-07-21T22:37:15.023 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap" 2023-07-21T22:37:15.023 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.024 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2023-07-21T22:37:15.024 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.024 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.024 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.025 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2023-07-21T22:37:15.025 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2023-07-21T22:37:15.025 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.025 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.026 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.026 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.026 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:37:15.026 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.027 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.027 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.027 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.027 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:15.027 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:15.028 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.028 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.028 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.028 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:15.028 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.029 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.029 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.029 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:37:15.029 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.029 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.030 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.030 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2023-07-21T22:37:15.030 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2023-07-21T22:37:15.030 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.031 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.031 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:15.031 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:15.031 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.031 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:37:15.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:37:15.032 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.032 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.033 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2023-07-21T22:37:15.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:37:15.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:15.033 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.034 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.034 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:37:15.035 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.035 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2023-07-21T22:37:15.036 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.036 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.037 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.037 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2023-07-21T22:37:15.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2023-07-21T22:37:15.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.037 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.038 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-07-21T22:37:15.038 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.038 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.038 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.039 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:37:15.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.040 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2023-07-21T22:37:15.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2023-07-21T22:37:15.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.040 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.041 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2023-07-21T22:37:15.041 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.041 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:37:15.042 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.042 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.042 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.042 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.042 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2023-07-21T22:37:15.043 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2023-07-21T22:37:15.043 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.043 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.043 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.043 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2023-07-21T22:37:15.044 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.044 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.044 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.044 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:37:15.044 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.045 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.045 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.045 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.045 INFO:teuthology.orchestra.run.smithi066.stdout: "alertmanager_web_password": { 2023-07-21T22:37:15.046 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alertmanager_web_password", 2023-07-21T22:37:15.046 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.046 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.046 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.046 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.047 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.047 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.047 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Alertmanager web password", 2023-07-21T22:37:15.047 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.048 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.048 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.048 INFO:teuthology.orchestra.run.smithi066.stdout: "alertmanager_web_user": { 2023-07-21T22:37:15.049 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alertmanager_web_user", 2023-07-21T22:37:15.049 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.049 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.050 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.050 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.050 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.050 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.050 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Alertmanager web user", 2023-07-21T22:37:15.051 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.051 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.051 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.051 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.051 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2023-07-21T22:37:15.052 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2023-07-21T22:37:15.052 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.052 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.052 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.052 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.053 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.053 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.053 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.053 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:37:15.053 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:37:15.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.054 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2023-07-21T22:37:15.054 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2023-07-21T22:37:15.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.055 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.055 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:15.055 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.056 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:37:15.056 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.057 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:37:15.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:37:15.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.058 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2023-07-21T22:37:15.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:37:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.059 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.060 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2023-07-21T22:37:15.060 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2023-07-21T22:37:15.060 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.060 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.061 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.061 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.061 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.061 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:37:15.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.062 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.063 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2023-07-21T22:37:15.063 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2023-07-21T22:37:15.063 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.063 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.063 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.064 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.064 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.064 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:37:15.065 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.065 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.065 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.065 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2023-07-21T22:37:15.066 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2023-07-21T22:37:15.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.066 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.067 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.067 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.067 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:37:15.067 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.067 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.068 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.068 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2023-07-21T22:37:15.068 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2023-07-21T22:37:15.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.069 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.069 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:37:15.069 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.069 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.070 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.070 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:15.070 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.070 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.070 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.071 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2023-07-21T22:37:15.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2023-07-21T22:37:15.071 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.071 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.072 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:37:15.072 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.072 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.072 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.073 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:37:15.073 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.073 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.073 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.074 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2023-07-21T22:37:15.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:37:15.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.074 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.075 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:37:15.075 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.075 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.075 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.075 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2023-07-21T22:37:15.076 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.076 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.076 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.076 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2023-07-21T22:37:15.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2023-07-21T22:37:15.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.077 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.077 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.077 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:37:15.077 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.078 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.078 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.078 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:15.078 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.078 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.079 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.079 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.079 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2023-07-21T22:37:15.079 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2023-07-21T22:37:15.080 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.080 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.080 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.080 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:37:15.080 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.081 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.081 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.081 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2023-07-21T22:37:15.081 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.081 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.082 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.082 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.082 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2023-07-21T22:37:15.082 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:37:15.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.083 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.083 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.083 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:37:15.083 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.083 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.083 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.084 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:37:15.084 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.084 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.084 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.085 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.085 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2023-07-21T22:37:15.085 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:37:15.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.085 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.086 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:37:15.086 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.086 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.086 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.086 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:37:15.087 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.087 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.087 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.087 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2023-07-21T22:37:15.088 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:37:15.088 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.088 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.088 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.088 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:37:15.089 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.089 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.089 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.089 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2023-07-21T22:37:15.089 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.090 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.090 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.090 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2023-07-21T22:37:15.090 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2023-07-21T22:37:15.091 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.091 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.091 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:37:15.091 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.092 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.092 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.092 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2023-07-21T22:37:15.092 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.092 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.093 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.093 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.093 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2023-07-21T22:37:15.093 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2023-07-21T22:37:15.093 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.094 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.094 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.094 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:37:15.094 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.095 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.095 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.095 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2023-07-21T22:37:15.096 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.097 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.098 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2023-07-21T22:37:15.098 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2023-07-21T22:37:15.099 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.099 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.099 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.100 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:37:15.100 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.101 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.101 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.102 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:15.102 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.102 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.103 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2023-07-21T22:37:15.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2023-07-21T22:37:15.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.105 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.105 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:37:15.105 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.105 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.105 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.106 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:15.106 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.106 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.106 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.107 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2023-07-21T22:37:15.107 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2023-07-21T22:37:15.107 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.107 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.108 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:37:15.108 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.108 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.108 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.108 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2023-07-21T22:37:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.109 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2023-07-21T22:37:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:37:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:37:15.110 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.111 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.111 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.111 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:37:15.111 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.111 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.112 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.112 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.112 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2023-07-21T22:37:15.112 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2023-07-21T22:37:15.112 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.113 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.113 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.113 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.113 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.113 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.114 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.114 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:37:15.114 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.114 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.114 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.115 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2023-07-21T22:37:15.115 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:37:15.115 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.115 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.115 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.116 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:15.116 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.116 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.116 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:37:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.118 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:37:15.118 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:37:15.118 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.118 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.119 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2023-07-21T22:37:15.119 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.119 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.120 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:37:15.120 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.120 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.121 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2023-07-21T22:37:15.121 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2023-07-21T22:37:15.121 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.121 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.121 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.122 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2023-07-21T22:37:15.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.122 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.122 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.122 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:37:15.122 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.123 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.123 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.123 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2023-07-21T22:37:15.123 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2023-07-21T22:37:15.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.124 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.124 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2023-07-21T22:37:15.125 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.126 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.126 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.126 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:37:15.127 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.127 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.128 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.128 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.128 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2023-07-21T22:37:15.129 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2023-07-21T22:37:15.129 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.130 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.130 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.131 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.131 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.131 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.132 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.132 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:37:15.133 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.133 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.133 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.134 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2023-07-21T22:37:15.134 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2023-07-21T22:37:15.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.134 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.134 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:15.135 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.135 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.135 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.135 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:37:15.136 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.136 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.136 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.136 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2023-07-21T22:37:15.137 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2023-07-21T22:37:15.137 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.137 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.137 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.137 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:15.138 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.138 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.138 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.138 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:37:15.138 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.139 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.139 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.139 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.139 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.140 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.140 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.140 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.141 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.141 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.141 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.141 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.141 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.141 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.142 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.142 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.142 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.142 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.142 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.143 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.143 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.143 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2023-07-21T22:37:15.144 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2023-07-21T22:37:15.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.144 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.144 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.145 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.145 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.145 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.145 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:37:15.145 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.146 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.146 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.146 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.147 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.147 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.148 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.148 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:37:15.148 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.149 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.149 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.150 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.150 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.150 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.151 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.151 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.151 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.151 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.151 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.152 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.152 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.152 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.152 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.152 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.153 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.153 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.153 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.153 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.154 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.154 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.154 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.155 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.155 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.155 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.155 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.156 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.156 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.156 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.156 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.156 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.157 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.157 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:37:15.157 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:37:15.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.158 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.158 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.158 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.158 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.159 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.159 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:37:15.159 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.159 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.159 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.160 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.160 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:37:15.160 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:37:15.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.161 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.161 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.161 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2023-07-21T22:37:15.161 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.161 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.161 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.162 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:37:15.162 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.162 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.162 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.163 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2023-07-21T22:37:15.163 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2023-07-21T22:37:15.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.163 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.164 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.164 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.164 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.165 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:37:15.165 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.165 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.165 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.165 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.166 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2023-07-21T22:37:15.166 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2023-07-21T22:37:15.166 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.166 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.166 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.167 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.167 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.167 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.167 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:37:15.168 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.168 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.168 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.168 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2023-07-21T22:37:15.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2023-07-21T22:37:15.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.169 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.169 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.170 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.170 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2023-07-21T22:37:15.170 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.170 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.171 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.171 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.171 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-07-21T22:37:15.171 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-07-21T22:37:15.171 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.172 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.172 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.172 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2023-07-21T22:37:15.172 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.172 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.173 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.173 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2023-07-21T22:37:15.173 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2023-07-21T22:37:15.173 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.173 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:37:15.174 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.174 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.174 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.174 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.175 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2023-07-21T22:37:15.175 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:37:15.175 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.176 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.176 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.177 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:37:15.177 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.177 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.178 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.178 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:37:15.178 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.179 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.180 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_web_password": { 2023-07-21T22:37:15.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_web_password", 2023-07-21T22:37:15.181 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.181 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.182 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.182 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.182 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.183 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.183 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.183 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus web password", 2023-07-21T22:37:15.184 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.184 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.184 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.185 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_web_user": { 2023-07-21T22:37:15.185 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_web_user", 2023-07-21T22:37:15.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.186 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.187 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.187 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.187 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.188 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.188 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.189 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus web user", 2023-07-21T22:37:15.189 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.189 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.190 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.190 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2023-07-21T22:37:15.191 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2023-07-21T22:37:15.191 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.192 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.192 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.192 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.193 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.193 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.194 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.194 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:37:15.194 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.195 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.195 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.196 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2023-07-21T22:37:15.196 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2023-07-21T22:37:15.197 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.197 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.198 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.198 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.198 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.198 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.198 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.199 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:37:15.199 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.199 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.199 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.200 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2023-07-21T22:37:15.200 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2023-07-21T22:37:15.200 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.200 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.201 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.201 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.201 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.201 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.201 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.201 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:37:15.202 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.202 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.202 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.202 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.203 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2023-07-21T22:37:15.203 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2023-07-21T22:37:15.203 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.203 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.204 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.204 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.204 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.204 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:37:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.205 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.206 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2023-07-21T22:37:15.206 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:37:15.207 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.207 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.207 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.207 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.207 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.208 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.208 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.208 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:37:15.208 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.208 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.209 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.209 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2023-07-21T22:37:15.209 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2023-07-21T22:37:15.209 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.210 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.210 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.210 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2023-07-21T22:37:15.210 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.210 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.211 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.211 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:37:15.211 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.211 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.211 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.212 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.212 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2023-07-21T22:37:15.212 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2023-07-21T22:37:15.212 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.213 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.213 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.213 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.213 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.213 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.214 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.214 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:37:15.214 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.214 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.214 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.214 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.215 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2023-07-21T22:37:15.215 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2023-07-21T22:37:15.215 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.215 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.215 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.216 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.216 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.216 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.216 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.217 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:37:15.217 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.217 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.217 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.218 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2023-07-21T22:37:15.218 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2023-07-21T22:37:15.218 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.218 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.218 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.219 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.219 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.219 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.219 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.219 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:37:15.220 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.220 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.220 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.220 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2023-07-21T22:37:15.221 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:37:15.221 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.221 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.221 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.221 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.222 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.222 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.222 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.222 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:37:15.222 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.222 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.223 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.223 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2023-07-21T22:37:15.223 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:37:15.223 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.224 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.224 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.224 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.224 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.224 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.225 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.225 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:37:15.225 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.225 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.225 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.226 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2023-07-21T22:37:15.226 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:37:15.226 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.226 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.227 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.227 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.227 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.227 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.228 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.228 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:37:15.228 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.228 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.228 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.229 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.229 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.229 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.229 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.229 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2023-07-21T22:37:15.230 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.230 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.230 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.230 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.230 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.231 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.231 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.231 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.231 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.232 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.232 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.232 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.232 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.232 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.233 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.233 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.233 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.233 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.233 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.233 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.234 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.234 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.234 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.234 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.235 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.235 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.235 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.235 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.235 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.236 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.236 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.236 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.236 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.236 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.237 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.237 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.237 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.238 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.238 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.238 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.238 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.239 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.239 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.239 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.239 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.239 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.240 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.240 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.240 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.240 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.240 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.241 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.241 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.241 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.241 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.242 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.242 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.242 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.242 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.243 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.243 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.243 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.243 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.243 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.244 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.244 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.244 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.244 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2023-07-21T22:37:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2023-07-21T22:37:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.245 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2023-07-21T22:37:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.246 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:37:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.247 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2023-07-21T22:37:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2023-07-21T22:37:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.248 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2023-07-21T22:37:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.249 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:37:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.250 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2023-07-21T22:37:15.251 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:37:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:37:15.252 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.253 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.254 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:37:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:37:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.255 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.256 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.257 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.257 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.257 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.257 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.257 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.257 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:37:15.258 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:37:15.258 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.258 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.259 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.259 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.259 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.259 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.259 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.260 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.260 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.260 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.260 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.260 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:37:15.261 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:37:15.261 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.261 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.261 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.261 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.262 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.262 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.262 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.262 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.262 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.263 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.263 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.263 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:37:15.263 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:37:15.264 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.264 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.264 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.264 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.265 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.265 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.265 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.265 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.265 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.265 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.266 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.266 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:37:15.266 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:37:15.266 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.267 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.267 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.267 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.267 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.267 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.268 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.268 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.268 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.268 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.268 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.269 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.269 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:37:15.269 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:37:15.269 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.270 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.270 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.270 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.270 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.270 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.271 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.271 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.271 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.271 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.271 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.272 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.272 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:37:15.272 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:37:15.272 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.272 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.273 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.273 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.273 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.273 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.273 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.274 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.274 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.274 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.274 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.274 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:37:15.275 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:37:15.275 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.275 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.275 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.275 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.276 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.276 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.276 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.276 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.276 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.277 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.277 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.277 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:37:15.277 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:37:15.278 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.278 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.278 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.278 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.278 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.279 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.279 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.279 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.279 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.280 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.280 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.280 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.280 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:37:15.281 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:37:15.281 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.281 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.281 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.281 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.281 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.282 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.282 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.282 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.282 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.282 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.283 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.283 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:37:15.283 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:37:15.283 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.284 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.284 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.284 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.284 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.284 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.285 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.285 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.285 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.285 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.285 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.286 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.286 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:37:15.286 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:37:15.286 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.286 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.287 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.287 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.287 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.287 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.287 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.287 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.288 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.288 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.288 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.288 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:37:15.289 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:37:15.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.289 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.289 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.290 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.290 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.290 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.291 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.291 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.291 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.291 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:37:15.292 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:37:15.292 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.292 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.292 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.292 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.293 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.293 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.293 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.293 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.293 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.294 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.294 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.294 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:37:15.294 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:37:15.294 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.295 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.295 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.295 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.295 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.295 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.296 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.296 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.296 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.296 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.296 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.297 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.297 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2023-07-21T22:37:15.297 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:37:15.297 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.297 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.298 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.298 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.298 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.298 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.298 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.299 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.299 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.299 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.299 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.299 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.300 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:37:15.300 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:37:15.300 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.300 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.300 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.301 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.301 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.301 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.301 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.301 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.302 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.302 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.302 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.302 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.302 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:37:15.303 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:37:15.303 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.303 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.303 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.303 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.304 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.304 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.304 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.304 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.305 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.305 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.305 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.305 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:37:15.305 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:37:15.306 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.306 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.306 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.306 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.306 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.307 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.307 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.307 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.307 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.308 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.308 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.308 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:37:15.308 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:37:15.309 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.309 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.309 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.309 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.309 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.310 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.310 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.310 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.310 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.310 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.310 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.311 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:37:15.311 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:37:15.311 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.311 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.312 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.312 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.312 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.312 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.313 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.313 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.313 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.313 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.313 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.314 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:37:15.314 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:37:15.314 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.314 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.315 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.315 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.315 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.315 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.315 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.315 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.316 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.316 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.316 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.317 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:37:15.317 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:37:15.317 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.317 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.317 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.318 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.318 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.318 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.318 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.318 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.318 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.319 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.319 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.319 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:37:15.319 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:37:15.320 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.320 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.320 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.320 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.320 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.321 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.321 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.321 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.321 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.322 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.322 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:37:15.322 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:37:15.322 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.323 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.323 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.323 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.323 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.323 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.324 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.324 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.324 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.324 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.324 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.325 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.325 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:37:15.325 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:37:15.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.325 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.325 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.326 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.326 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.326 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.326 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.326 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.327 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.327 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.327 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.327 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:37:15.328 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:37:15.328 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.328 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.328 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.328 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.328 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.329 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.329 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.329 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.329 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.329 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.330 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.330 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:37:15.330 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:37:15.330 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.331 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.331 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.331 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.331 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.331 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.331 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.332 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.332 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.332 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.333 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.333 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.334 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:37:15.334 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:37:15.335 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.335 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.335 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.336 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.336 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.337 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.337 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.338 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.338 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.338 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.339 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.340 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:37:15.340 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:37:15.340 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.340 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.341 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.341 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.342 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.342 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.342 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.343 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.343 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.343 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.343 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.344 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:37:15.344 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:37:15.344 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.344 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.345 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.345 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.345 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.345 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.345 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.346 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.346 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.346 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.346 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.346 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:37:15.347 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:37:15.347 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.347 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.347 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.347 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:37:15.348 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.348 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.348 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.348 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.348 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.349 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.349 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.349 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:37:15.349 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:37:15.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.350 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.350 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.350 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.351 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.351 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.351 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.351 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.351 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:37:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:37:15.352 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2023-07-21T22:37:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.353 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.354 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:37:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:37:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.355 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2023-07-21T22:37:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.356 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.357 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:37:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:37:15.358 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.359 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.360 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:37:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:37:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.361 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.362 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.363 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.363 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.363 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.363 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.364 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:37:15.364 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:37:15.364 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.364 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.364 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.365 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.365 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.365 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.365 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.365 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.365 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.366 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.366 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.366 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:37:15.366 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:37:15.367 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.367 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.367 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.367 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.367 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.368 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.368 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.368 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.368 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.368 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.369 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.369 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:37:15.369 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:37:15.369 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.370 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.370 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.370 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:15.370 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.370 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.372 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:37:15.372 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:37:15.372 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.372 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.373 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.373 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.373 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.373 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.373 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.374 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.374 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.374 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.374 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.375 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:37:15.375 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:37:15.375 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.375 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.376 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.376 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:15.376 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.376 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.376 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.377 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.377 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.377 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.377 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.377 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.378 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2023-07-21T22:37:15.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2023-07-21T22:37:15.378 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.378 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.378 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.378 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.379 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.379 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.379 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.379 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.379 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.380 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.380 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.380 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.380 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2023-07-21T22:37:15.380 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2023-07-21T22:37:15.381 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.381 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.381 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.381 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.381 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.382 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.382 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.382 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.382 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.382 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.382 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.383 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.383 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2023-07-21T22:37:15.383 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2023-07-21T22:37:15.384 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.384 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.384 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.385 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.385 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.385 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.386 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.386 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:37:15.387 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.387 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.388 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.388 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.389 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2023-07-21T22:37:15.389 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2023-07-21T22:37:15.389 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.389 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.389 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.390 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2023-07-21T22:37:15.390 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.390 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.390 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.390 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.391 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.391 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.391 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.391 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.391 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-07-21T22:37:15.392 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-07-21T22:37:15.392 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.392 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.392 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.392 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.393 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.393 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.393 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.393 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.393 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.394 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.394 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.394 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.394 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.394 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.395 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.395 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.395 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.395 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.395 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.395 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.396 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.396 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.396 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.396 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.397 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.397 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.397 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.397 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.397 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.398 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.398 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.398 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.398 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.398 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.399 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.399 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.399 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.399 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.399 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.400 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.400 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.400 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.400 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.400 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.401 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.401 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.401 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.401 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.402 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.402 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.402 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.402 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.402 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.403 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.403 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.403 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.403 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.403 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.404 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.404 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.404 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.404 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.405 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.406 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.406 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.406 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.406 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.406 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.407 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.407 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.407 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.407 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.407 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.408 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.408 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.408 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2023-07-21T22:37:15.409 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2023-07-21T22:37:15.409 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.409 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.409 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.409 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.409 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.410 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.410 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.410 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2023-07-21T22:37:15.410 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.410 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.411 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.411 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:37:15.411 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:37:15.411 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.412 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.412 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.412 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.412 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.413 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.413 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.413 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.413 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.413 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.414 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.414 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:15.414 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:15.414 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.415 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.415 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.415 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-07-21T22:37:15.415 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.415 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.415 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.416 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.416 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.416 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.416 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.417 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:15.417 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:15.417 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.417 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.417 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.418 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2023-07-21T22:37:15.418 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.418 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.418 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.418 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.419 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.419 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.419 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.419 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-07-21T22:37:15.420 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-07-21T22:37:15.420 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.420 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.420 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.420 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.421 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.421 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.421 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.421 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.421 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.422 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.422 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.422 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2023-07-21T22:37:15.422 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2023-07-21T22:37:15.422 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.423 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.423 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2023-07-21T22:37:15.423 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.423 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.424 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.424 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.424 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.424 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.424 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.425 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-07-21T22:37:15.425 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-07-21T22:37:15.425 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.425 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.426 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.426 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2023-07-21T22:37:15.426 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.426 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.426 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.427 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.427 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2023-07-21T22:37:15.427 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.427 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.427 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.427 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.428 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.428 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-07-21T22:37:15.428 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-07-21T22:37:15.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.429 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.429 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.429 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-07-21T22:37:15.429 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-07-21T22:37:15.430 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-07-21T22:37:15.430 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.430 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.430 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.430 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.430 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.431 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2023-07-21T22:37:15.431 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2023-07-21T22:37:15.431 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.431 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.432 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.432 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.432 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.432 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.432 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.433 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.433 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.433 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.433 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.433 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.434 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.434 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.434 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.434 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-07-21T22:37:15.434 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.435 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.435 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.435 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2023-07-21T22:37:15.435 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2023-07-21T22:37:15.435 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.435 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.436 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.436 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.436 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.436 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.437 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.437 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2023-07-21T22:37:15.437 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.437 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.437 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.438 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.438 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.438 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.438 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.438 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.438 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.439 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.439 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.439 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.439 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.439 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.440 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.440 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.440 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.440 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.440 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.441 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.441 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.441 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.441 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.441 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.442 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.442 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.442 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.442 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.442 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.443 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.443 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.443 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.443 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.443 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.444 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.444 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.444 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.444 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.445 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.445 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.445 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.445 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.446 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.446 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.446 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.446 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.446 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.446 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.447 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.448 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.448 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.448 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.448 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.449 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.449 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.449 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.449 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.449 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.450 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.450 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.450 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.450 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.450 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.451 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.451 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.451 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.451 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.452 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2023-07-21T22:37:15.452 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2023-07-21T22:37:15.452 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.452 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.452 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.453 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2023-07-21T22:37:15.453 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.453 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.453 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.453 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:37:15.454 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.454 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.454 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.454 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.454 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2023-07-21T22:37:15.454 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2023-07-21T22:37:15.455 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.455 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.455 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.456 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2023-07-21T22:37:15.456 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.456 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.457 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.457 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:37:15.457 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.457 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.458 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.458 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2023-07-21T22:37:15.458 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2023-07-21T22:37:15.459 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.459 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.459 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.459 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2023-07-21T22:37:15.459 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.460 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.460 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.460 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:37:15.460 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.460 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.461 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.461 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2023-07-21T22:37:15.461 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2023-07-21T22:37:15.462 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.462 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.462 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.462 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-07-21T22:37:15.462 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.462 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.463 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.463 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:37:15.463 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.463 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.463 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.464 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2023-07-21T22:37:15.464 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2023-07-21T22:37:15.464 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.464 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.465 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.465 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.465 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.465 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.466 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:37:15.466 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.466 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.466 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.467 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:15.467 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:15.467 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.467 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.468 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.468 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:15.468 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.468 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.468 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.469 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:37:15.469 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.469 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.469 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.470 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.470 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2023-07-21T22:37:15.470 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2023-07-21T22:37:15.470 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.470 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.471 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.471 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2023-07-21T22:37:15.471 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.471 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.471 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.472 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:37:15.472 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.472 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.472 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.473 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.473 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.473 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.473 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2023-07-21T22:37:15.473 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.474 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.474 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.474 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.474 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.475 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.475 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.475 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.475 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.475 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.476 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.477 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.478 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.480 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.482 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.483 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.484 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.485 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.486 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.487 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.488 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.489 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.490 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.491 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2023-07-21T22:37:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2023-07-21T22:37:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.492 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-07-21T22:37:15.493 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.493 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.493 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.493 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.493 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.494 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.494 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2023-07-21T22:37:15.495 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2023-07-21T22:37:15.495 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.495 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.495 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.496 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2023-07-21T22:37:15.496 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.496 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.496 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.497 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.497 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.497 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.497 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.498 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:15.498 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:15.498 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.498 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.499 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.499 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:15.499 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.499 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.499 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.500 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.500 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.500 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.500 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.500 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.501 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.501 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.501 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2023-07-21T22:37:15.501 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2023-07-21T22:37:15.502 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:37:15.502 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.502 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2023-07-21T22:37:15.502 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2023-07-21T22:37:15.502 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.503 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.503 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.503 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2023-07-21T22:37:15.503 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.503 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.504 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.504 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:37:15.504 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.504 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2023-07-21T22:37:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2023-07-21T22:37:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.505 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.506 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.506 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2023-07-21T22:37:15.506 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.507 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.507 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.507 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:37:15.507 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.508 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.508 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.508 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2023-07-21T22:37:15.509 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2023-07-21T22:37:15.509 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.509 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.510 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.510 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.510 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.510 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.510 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:37:15.511 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.511 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.511 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.511 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.511 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:15.512 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:15.512 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.512 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.512 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.512 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2023-07-21T22:37:15.513 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2023-07-21T22:37:15.513 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.513 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.513 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:37:15.513 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.514 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.514 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.514 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.514 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.515 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.515 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.515 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.515 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.516 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.516 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.516 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.516 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.517 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.517 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.517 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.517 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.518 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.518 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.518 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.518 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.519 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.519 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.519 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.520 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.520 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.520 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.521 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.521 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.522 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.522 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.522 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.523 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.523 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.524 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.524 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.525 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.525 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.526 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.526 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.527 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.527 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.527 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.528 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.528 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.528 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.528 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.529 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.529 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.529 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.529 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.530 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.530 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.530 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.530 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.530 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.531 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.531 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.531 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.531 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.531 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.532 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.532 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.532 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.532 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.532 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.533 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2023-07-21T22:37:15.533 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2023-07-21T22:37:15.533 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.533 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.534 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.534 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.534 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.534 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.534 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.535 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:37:15.535 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.535 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.536 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2023-07-21T22:37:15.536 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2023-07-21T22:37:15.536 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.536 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.536 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.537 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2023-07-21T22:37:15.537 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.537 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.537 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.537 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2023-07-21T22:37:15.538 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.538 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.538 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.538 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-07-21T22:37:15.538 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-07-21T22:37:15.539 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.539 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.539 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.539 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2023-07-21T22:37:15.540 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.540 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.540 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.540 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:37:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2023-07-21T22:37:15.541 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2023-07-21T22:37:15.542 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.542 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.542 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.542 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:15.543 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:15.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2023-07-21T22:37:15.543 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.543 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:37:15.543 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.544 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.544 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.544 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2023-07-21T22:37:15.544 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2023-07-21T22:37:15.545 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.545 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.545 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.545 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.546 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.546 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.546 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.546 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:37:15.546 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.547 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.547 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.547 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2023-07-21T22:37:15.547 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2023-07-21T22:37:15.548 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.548 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.548 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.548 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2023-07-21T22:37:15.548 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.549 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.549 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.549 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:37:15.549 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.550 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.550 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.550 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.550 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.551 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.551 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2023-07-21T22:37:15.551 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.551 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.552 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.552 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.553 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.553 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.553 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.554 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.554 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.555 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.555 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.556 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.556 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.556 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.557 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.557 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.557 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.557 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.558 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.558 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.558 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.558 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.559 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.559 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.559 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.559 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.560 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.560 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.561 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.561 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.561 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.561 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.561 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.562 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.562 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.562 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.563 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.563 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.563 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.563 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.563 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.564 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.564 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.564 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.564 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.564 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.565 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.565 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.565 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.565 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.565 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.566 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.566 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.566 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.566 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.566 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.567 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.567 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.567 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.567 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.567 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.568 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.568 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.568 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.568 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.568 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.569 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.569 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.569 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.569 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2023-07-21T22:37:15.570 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.570 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.570 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.570 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.571 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.571 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.571 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.571 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.571 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.572 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.572 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.572 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.572 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.572 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.573 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.573 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.573 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.573 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.573 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.574 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.574 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.574 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.574 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.575 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.575 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.575 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.575 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.576 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.576 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.576 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.576 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.577 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.577 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.577 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.577 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.577 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.578 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.578 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.578 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.578 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.578 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.579 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.579 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.579 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.579 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.579 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.580 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.580 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.580 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.580 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.580 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.581 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.581 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.581 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.581 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.581 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.582 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.582 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.582 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.582 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.582 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.582 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.583 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.583 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.583 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.583 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.583 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.584 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.584 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.584 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.584 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.584 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.585 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.585 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.585 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2023-07-21T22:37:15.585 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.585 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.586 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.586 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2023-07-21T22:37:15.586 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:37:15.586 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.586 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.587 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.587 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2023-07-21T22:37:15.587 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.587 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.587 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.588 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:37:15.588 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.588 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.588 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2023-07-21T22:37:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2023-07-21T22:37:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.589 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.590 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2023-07-21T22:37:15.590 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.590 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.590 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:37:15.590 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.591 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.591 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.591 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.591 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.592 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.592 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.592 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.592 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.592 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.593 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.593 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.593 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.593 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.593 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.594 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.594 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.594 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.594 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.594 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.595 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.595 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.595 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.595 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.596 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.596 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.596 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.596 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.597 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.597 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.597 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.597 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.598 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.598 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.599 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.599 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.600 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.600 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.601 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.601 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.602 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.602 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.602 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.603 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.603 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.603 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.603 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.604 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.604 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.604 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.604 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.604 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.605 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.605 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.605 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.605 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.606 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.606 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.606 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.607 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.607 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.607 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.607 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.608 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.608 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.608 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.608 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.608 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.609 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.609 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.609 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.610 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.610 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2023-07-21T22:37:15.610 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.610 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.611 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.611 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2023-07-21T22:37:15.611 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2023-07-21T22:37:15.611 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.611 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.612 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.612 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2023-07-21T22:37:15.612 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.612 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.612 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.613 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:37:15.613 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:37:15.613 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.613 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.613 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.614 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.615 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.615 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.615 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.615 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.616 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.616 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.616 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.616 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.616 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.617 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.617 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.617 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.617 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.617 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.618 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.618 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.618 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.618 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.619 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.619 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.619 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.619 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.619 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.620 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.620 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.620 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.620 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.620 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.621 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.621 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.621 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.621 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.622 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.623 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.623 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.623 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.623 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.623 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.624 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.624 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.624 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.624 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.624 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.625 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.625 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.625 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.625 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.625 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.626 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.626 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.626 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.626 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.626 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.627 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.627 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.627 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.627 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.627 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2023-07-21T22:37:15.628 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2023-07-21T22:37:15.628 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.628 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.628 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:37:15.629 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:37:15.630 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.630 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.630 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.630 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2023-07-21T22:37:15.631 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2023-07-21T22:37:15.631 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.631 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.632 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.632 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2023-07-21T22:37:15.633 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.633 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.633 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.634 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:37:15.634 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.635 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.635 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.636 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2023-07-21T22:37:15.636 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2023-07-21T22:37:15.636 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.637 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.637 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.637 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2023-07-21T22:37:15.637 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.637 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.638 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.638 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:37:15.638 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.638 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.638 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.639 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2023-07-21T22:37:15.639 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2023-07-21T22:37:15.639 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.639 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.640 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.640 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.640 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.640 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.640 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.641 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:37:15.641 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.641 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.641 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.641 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.642 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2023-07-21T22:37:15.642 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2023-07-21T22:37:15.643 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.643 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.644 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.644 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2023-07-21T22:37:15.645 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.645 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.645 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.646 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:37:15.646 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:37:15.646 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.647 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.647 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.648 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.648 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.649 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2023-07-21T22:37:15.649 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.649 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.649 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.649 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.650 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.650 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.650 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.650 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.650 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.651 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.652 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.654 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.655 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.655 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.655 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.655 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.655 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.656 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.656 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.656 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.656 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.656 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.657 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.657 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.657 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.657 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.657 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.658 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.658 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.658 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.658 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.658 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.659 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.659 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.659 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.659 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.659 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.660 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.660 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.660 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.661 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.661 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.661 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.661 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.662 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.662 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.662 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.662 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.662 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.663 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.663 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.663 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.663 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.664 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.664 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2023-07-21T22:37:15.664 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.664 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.664 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.665 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.666 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.666 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.666 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.666 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.666 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.667 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.667 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.667 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.667 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.667 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.668 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.668 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.668 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.668 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.668 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.669 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.669 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.669 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.669 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.670 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.670 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.670 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.670 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.670 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.670 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.671 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.671 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.671 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.671 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.672 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.672 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.672 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.672 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.673 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.673 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.673 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.673 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.673 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.674 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.674 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.674 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.674 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.674 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.675 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.675 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.675 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.675 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.675 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.676 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.676 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.676 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.676 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.676 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.677 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.677 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.677 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.677 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.677 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.678 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.678 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.678 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.679 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.679 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.679 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2023-07-21T22:37:15.679 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.680 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.680 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.680 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.680 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.681 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.681 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.681 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.682 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.682 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.683 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.683 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.684 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.684 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.685 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.685 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.685 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.686 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.686 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.686 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.686 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.686 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.687 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.687 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.687 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.687 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.688 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.688 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.688 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.688 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.688 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.689 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.689 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.689 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.689 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.690 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.690 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.690 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.690 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.691 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.691 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.691 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.691 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.691 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.692 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.692 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.692 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.692 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.692 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.693 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.693 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.693 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.693 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.693 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.694 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.694 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.694 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.694 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.694 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.695 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.695 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.695 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.695 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.695 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.696 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.696 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.696 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.696 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.696 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.697 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.697 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.697 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.697 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.697 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-07-21T22:37:15.698 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.698 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.698 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.700 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2023-07-21T22:37:15.700 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2023-07-21T22:37:15.700 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.700 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.701 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.701 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.701 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.701 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.701 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.702 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:37:15.702 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.702 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.702 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.702 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.703 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.703 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.703 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.703 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.703 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.704 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.704 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.704 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.704 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.704 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.705 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.705 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.705 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.706 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.706 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.706 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.706 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.706 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.707 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.707 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.707 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.707 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.708 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.708 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.708 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.708 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.708 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.709 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.709 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.709 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.709 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.709 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.709 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.710 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.710 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.710 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.710 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.711 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.711 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.711 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.711 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.711 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.712 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.712 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.712 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.712 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.712 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.713 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.713 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.713 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.713 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.713 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.714 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.715 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.715 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.715 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.715 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.715 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.715 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.716 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.716 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.716 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.716 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.716 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.717 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2023-07-21T22:37:15.717 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-07-21T22:37:15.717 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.717 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.717 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.718 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.718 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.718 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.718 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.718 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-07-21T22:37:15.719 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2023-07-21T22:37:15.719 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2023-07-21T22:37:15.719 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.719 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2023-07-21T22:37:15.719 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.719 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.720 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.720 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.720 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.720 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2023-07-21T22:37:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.721 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.722 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.722 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.722 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.722 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.722 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.723 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.723 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.723 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.723 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.723 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.724 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.724 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.724 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.724 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.724 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.725 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.725 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.725 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.725 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.725 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.726 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.726 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.726 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.726 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.727 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.727 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.727 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.727 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.727 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.727 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.728 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.728 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.728 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.728 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.729 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.729 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.729 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.729 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.729 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.730 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.730 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.730 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.730 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.730 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.731 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.731 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.731 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.731 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.731 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.732 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.732 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.732 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.732 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.733 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.733 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.733 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.733 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.733 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.734 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.734 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.734 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.734 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.734 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.734 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.735 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.735 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.735 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.735 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.735 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.736 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.736 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2023-07-21T22:37:15.736 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.736 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.736 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.737 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.737 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.737 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.737 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.737 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.738 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.738 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.738 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.738 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.738 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.739 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.739 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.739 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.739 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.739 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.739 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.740 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.740 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.740 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.740 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.741 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.741 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.741 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.741 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.741 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.742 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.742 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.742 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.742 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.742 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.742 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.743 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.743 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.743 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.743 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.744 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.744 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.744 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.744 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.744 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.745 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.745 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.745 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.745 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.745 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.746 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.746 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.746 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.746 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.746 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.746 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.747 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.747 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.747 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.747 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.748 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.748 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.748 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.748 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.748 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.749 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.749 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.749 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.749 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.749 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.750 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.750 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.750 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.750 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.750 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.750 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2023-07-21T22:37:15.751 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.751 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.751 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.751 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.751 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.752 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.752 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.752 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.752 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.752 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.753 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.753 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.753 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.753 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.753 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.754 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.755 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.755 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.755 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.755 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.755 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.756 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.756 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.756 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.756 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.756 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.757 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.757 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.757 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.757 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.757 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.758 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.758 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.758 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.758 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.758 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.759 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.759 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.759 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.759 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.759 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.760 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.760 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.760 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.760 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.760 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.761 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.761 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.761 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.761 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.761 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.762 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.762 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.762 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.762 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.762 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.763 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.763 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.763 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.763 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.763 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.764 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.764 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.764 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.764 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.765 INFO:teuthology.orchestra.run.smithi066.stdout: "noautoscale": { 2023-07-21T22:37:15.765 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "noautoscale", 2023-07-21T22:37:15.765 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.765 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.765 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.766 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.766 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.766 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.766 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.766 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "global autoscale flag", 2023-07-21T22:37:15.766 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:37:15.767 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.767 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.767 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:15.767 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:15.768 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.768 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.768 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.768 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:15.768 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.769 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.769 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.769 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.769 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.769 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.770 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.770 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.770 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2023-07-21T22:37:15.770 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2023-07-21T22:37:15.770 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.771 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.771 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.771 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-07-21T22:37:15.771 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2023-07-21T22:37:15.771 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.772 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.772 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2023-07-21T22:37:15.772 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:37:15.772 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.772 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.773 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.773 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.773 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.773 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2023-07-21T22:37:15.773 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.774 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.774 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.774 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2023-07-21T22:37:15.774 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:37:15.774 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.775 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.775 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.775 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.775 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.775 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.776 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.776 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:37:15.776 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.776 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.776 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.777 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-07-21T22:37:15.777 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-07-21T22:37:15.777 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.777 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.777 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.778 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.778 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.778 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.778 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.778 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.779 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.779 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.779 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.779 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.780 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.780 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.780 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.780 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.780 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.780 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.781 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.781 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.781 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.781 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.781 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.782 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.782 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.782 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.782 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.782 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.783 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.783 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.783 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.783 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.783 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.784 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.784 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.784 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.784 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.784 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.785 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.785 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.785 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.785 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.786 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.786 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.786 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.786 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.786 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.787 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.787 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.787 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.787 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.787 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.787 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.788 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.788 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.788 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.788 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.788 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.789 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.789 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.789 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.789 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.789 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.790 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.790 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.790 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.790 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.790 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.791 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.792 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.792 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.792 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.792 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.793 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2023-07-21T22:37:15.793 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2023-07-21T22:37:15.793 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.793 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.793 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.794 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2023-07-21T22:37:15.794 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.794 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.794 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.794 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:37:15.795 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.795 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.795 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.795 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:15.795 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:15.796 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:15.796 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.796 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.796 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:15.796 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.797 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.797 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.797 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:37:15.797 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.797 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.798 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.798 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.798 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.798 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.798 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.798 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2023-07-21T22:37:15.799 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.799 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.799 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.799 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2023-07-21T22:37:15.799 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2023-07-21T22:37:15.800 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.800 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.800 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.800 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.800 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.801 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.801 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.801 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.801 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.801 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.802 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.802 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2023-07-21T22:37:15.802 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2023-07-21T22:37:15.802 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.802 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.803 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.803 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.803 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.803 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.803 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.804 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:37:15.804 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:37:15.804 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.804 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.805 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.805 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.805 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.805 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.805 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.805 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.806 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.806 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.806 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.806 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.806 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.807 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.807 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.807 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.807 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.807 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.808 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.808 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.808 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.808 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.808 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.809 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.809 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.809 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.809 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.809 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.810 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.810 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.810 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.810 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.810 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.811 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.811 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.811 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.811 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.811 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.812 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.812 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.812 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.812 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.812 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.813 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.813 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.813 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.813 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.813 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.813 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.814 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.814 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.814 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.814 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.814 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.815 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.815 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.815 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.815 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.816 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.816 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.816 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.816 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.816 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.817 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.817 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.817 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.817 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.817 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.817 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.818 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.818 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2023-07-21T22:37:15.818 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2023-07-21T22:37:15.818 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.819 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.819 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.819 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.819 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.819 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.820 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.820 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.820 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.820 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.820 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.821 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:37:15.821 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:37:15.821 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.821 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.821 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.822 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2023-07-21T22:37:15.822 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.822 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.822 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.822 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.823 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.823 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.823 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.823 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.823 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2023-07-21T22:37:15.824 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2023-07-21T22:37:15.824 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.824 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.824 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.824 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2023-07-21T22:37:15.825 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.825 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.825 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.825 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.825 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.826 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.826 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.826 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:15.827 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:15.827 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.827 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.827 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.827 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-07-21T22:37:15.828 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.828 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.828 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.828 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:37:15.828 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.829 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.829 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.829 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.829 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:15.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:15.830 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.830 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.830 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.830 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2023-07-21T22:37:15.831 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.831 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.831 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.831 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:37:15.831 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.832 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.832 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.832 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.832 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2023-07-21T22:37:15.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2023-07-21T22:37:15.833 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.833 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.833 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.833 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2023-07-21T22:37:15.833 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.833 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.834 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.834 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.834 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.834 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.834 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.835 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-07-21T22:37:15.835 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-07-21T22:37:15.835 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.835 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.836 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.836 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2023-07-21T22:37:15.836 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.836 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.836 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.837 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2023-07-21T22:37:15.837 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2023-07-21T22:37:15.837 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.837 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.837 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.837 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.838 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.838 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.838 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-07-21T22:37:15.838 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-07-21T22:37:15.838 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.839 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.839 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.839 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-07-21T22:37:15.839 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-07-21T22:37:15.839 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-07-21T22:37:15.840 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.840 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.840 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.840 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.840 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.841 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.841 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.841 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.841 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.841 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-07-21T22:37:15.842 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.842 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.842 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.842 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.842 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.843 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.843 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.843 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.843 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.843 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.843 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.844 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.844 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.844 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.844 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.844 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.845 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.845 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.845 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.845 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.845 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.846 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.846 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.846 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.846 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.846 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.847 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.847 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.847 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.847 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.847 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.848 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.848 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.848 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.848 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.848 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.849 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.849 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.849 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.850 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.850 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.850 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.850 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.850 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.850 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.851 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.851 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.851 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.851 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.851 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.852 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.852 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.852 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.852 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.852 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.853 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.853 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.853 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.853 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.853 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.854 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.854 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.854 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.854 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.854 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.854 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.855 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.855 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.855 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.855 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2023-07-21T22:37:15.856 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:37:15.856 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.856 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.856 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.856 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:15.857 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.857 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.857 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.857 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.857 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.858 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.858 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.858 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:37:15.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:37:15.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.859 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.859 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.859 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.859 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.859 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.860 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.860 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.860 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.860 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.860 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.861 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.861 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2023-07-21T22:37:15.861 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2023-07-21T22:37:15.861 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.861 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.862 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.862 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.862 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.862 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.862 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.863 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.863 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.863 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.863 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.864 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.864 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2023-07-21T22:37:15.864 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.864 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.865 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.865 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2023-07-21T22:37:15.865 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2023-07-21T22:37:15.865 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.865 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.865 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.866 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.866 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.866 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.866 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.866 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.867 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.867 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.867 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.867 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-07-21T22:37:15.868 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-07-21T22:37:15.868 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.868 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.868 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.868 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.869 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.870 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.870 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.870 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.870 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.870 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.870 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.871 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.871 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.871 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.871 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.871 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.872 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.872 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.872 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.872 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.872 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.873 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.873 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.873 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.873 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.873 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.874 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.874 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.874 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.874 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.874 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.875 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.875 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.875 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.875 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.875 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.876 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.876 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.876 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.876 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.876 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.877 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.877 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.877 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.877 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.878 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.878 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.878 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.878 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.879 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.879 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.879 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.879 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.879 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.880 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.880 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.880 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.880 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.880 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.881 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.881 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.881 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.881 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.882 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.882 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.882 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.882 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.882 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.883 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.883 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.883 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.883 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.883 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.883 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.884 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:15.884 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:15.884 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.884 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.885 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.885 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.885 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.885 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.885 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.886 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.886 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.886 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.886 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.887 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:15.887 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:15.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.887 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.887 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.887 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.888 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.888 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.888 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.888 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.888 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.889 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.889 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.889 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.889 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.890 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2023-07-21T22:37:15.890 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.890 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.890 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.890 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.891 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.891 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.891 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.891 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.891 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.892 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.892 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.892 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.892 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.892 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.893 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.893 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.893 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.893 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.893 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.894 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.894 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.894 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.894 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.894 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.894 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.895 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.895 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.895 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.895 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.895 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.896 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.896 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.896 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.896 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.896 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.897 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.897 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.897 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.897 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.897 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.898 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.898 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.898 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.898 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.898 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.899 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.899 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.899 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.899 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.899 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.900 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.900 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.900 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.900 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.900 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.902 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.902 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.902 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.902 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.903 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.903 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.903 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.903 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.903 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.904 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.904 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2023-07-21T22:37:15.905 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.905 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.905 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.905 INFO:teuthology.orchestra.run.smithi066.stdout: "drive_group_interval": { 2023-07-21T22:37:15.905 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "drive_group_interval", 2023-07-21T22:37:15.905 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.906 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.906 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300.0", 2023-07-21T22:37:15.906 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.906 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.907 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.907 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:37:15.907 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.907 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.907 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.908 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.908 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.908 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.908 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.909 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.909 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.909 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.909 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.909 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.909 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.910 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.910 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.910 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.910 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.910 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.911 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.911 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.911 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.911 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.911 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.912 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.912 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.912 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.912 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.912 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.913 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.913 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.913 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.913 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.913 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.914 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.914 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.914 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.914 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.914 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.915 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.915 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.915 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.915 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.916 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.916 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.916 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.916 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.917 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.917 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.917 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.917 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.917 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.917 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.918 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.918 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.918 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.918 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.918 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.919 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.919 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.919 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.919 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.919 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.920 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.920 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.920 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.920 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.920 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.921 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.921 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.921 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.921 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2023-07-21T22:37:15.921 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2023-07-21T22:37:15.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.922 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2023-07-21T22:37:15.922 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.923 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:37:15.923 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.924 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.924 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.925 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2023-07-21T22:37:15.925 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.925 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.925 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.925 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.925 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.926 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.926 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.926 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.926 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.926 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.927 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.927 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.927 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.927 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.927 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.928 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.928 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.928 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.928 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.928 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.929 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.929 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.929 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.929 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.929 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.930 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.930 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.930 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.930 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.931 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.931 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.931 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.931 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.931 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.932 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.932 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.932 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.932 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.932 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.933 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.933 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.933 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.933 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.934 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.934 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.934 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.934 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.934 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.935 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.935 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.935 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.935 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.935 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.936 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.936 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.936 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.936 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.936 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.937 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.937 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.937 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.937 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.938 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.938 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.938 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.938 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.938 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2023-07-21T22:37:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2023-07-21T22:37:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.939 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.940 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2023-07-21T22:37:15.941 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2023-07-21T22:37:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2023-07-21T22:37:15.942 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.943 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.943 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.943 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.943 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.943 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.944 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.944 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.944 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2023-07-21T22:37:15.944 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2023-07-21T22:37:15.944 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.946 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.946 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.946 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.946 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.947 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2023-07-21T22:37:15.947 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2023-07-21T22:37:15.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.947 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.948 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.948 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.948 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.948 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.948 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.948 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.949 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.949 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.949 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2023-07-21T22:37:15.949 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2023-07-21T22:37:15.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:15.950 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.950 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.950 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.950 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.951 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.951 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.951 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.951 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.951 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.952 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2023-07-21T22:37:15.952 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2023-07-21T22:37:15.952 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.953 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.954 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.954 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.954 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.954 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.954 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2023-07-21T22:37:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2023-07-21T22:37:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.955 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.956 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.956 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.956 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.956 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.956 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.957 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2023-07-21T22:37:15.958 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2023-07-21T22:37:15.958 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.958 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.958 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:15.958 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.959 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.960 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.960 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2023-07-21T22:37:15.960 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2023-07-21T22:37:15.960 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:15.961 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.961 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.961 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:15.961 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2023-07-21T22:37:15.962 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.962 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.962 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.962 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.962 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.962 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.963 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2023-07-21T22:37:15.963 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2023-07-21T22:37:15.963 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.963 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.963 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.964 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.964 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.964 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.964 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.964 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.965 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.965 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2023-07-21T22:37:15.965 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2023-07-21T22:37:15.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.966 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.966 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.966 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.967 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.967 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.967 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.967 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.968 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.968 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.968 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2023-07-21T22:37:15.968 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2023-07-21T22:37:15.968 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.969 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:15.969 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.969 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.969 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.969 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.970 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.970 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.970 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.970 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.970 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.971 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.971 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.971 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.971 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-07-21T22:37:15.971 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.972 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.972 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.972 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2023-07-21T22:37:15.972 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2023-07-21T22:37:15.972 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.972 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.973 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.973 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.973 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.973 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.973 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.974 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:37:15.974 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.974 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.974 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.974 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.975 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2023-07-21T22:37:15.975 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2023-07-21T22:37:15.975 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.975 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.975 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.975 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.976 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.976 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.976 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:37:15.976 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.977 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.977 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.977 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.978 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.978 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.979 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.979 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.979 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.979 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.980 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.980 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.980 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.980 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.980 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.981 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.981 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.981 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.981 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.982 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.982 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.982 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.982 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.983 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.983 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.983 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.983 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.984 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.984 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.984 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.984 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:15.985 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.985 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.985 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.986 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.986 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.986 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.986 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.986 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.986 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.987 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.987 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.987 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.987 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.988 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:15.988 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:15.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.988 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.989 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.989 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.989 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.989 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.989 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.990 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.990 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.990 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.990 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.990 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:15.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.991 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:15.991 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2023-07-21T22:37:15.991 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:15.991 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:15.992 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:15.992 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:15.992 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:15.992 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.992 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:15.993 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.993 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.993 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:15.994 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:15.994 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:15.994 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:15.994 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:15.994 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.995 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.996 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.996 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:15.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:15.996 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:15.996 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.997 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:15.997 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:15.997 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:15.997 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:15.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:15.998 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:15.998 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:15.998 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:15.998 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:15.999 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:15.999 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:15.999 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:15.999 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:15.999 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:15.999 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.000 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.000 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.000 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.000 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.001 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.001 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.001 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.001 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.001 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.002 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.002 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.002 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.002 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.003 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.003 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.003 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.003 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.003 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.004 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.004 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.004 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.004 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.004 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.005 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.005 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.005 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.005 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.006 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.006 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2023-07-21T22:37:16.006 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.006 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.006 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.007 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.007 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.007 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.007 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.007 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.008 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.008 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.008 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.008 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.008 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.009 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.009 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.009 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.009 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.009 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.010 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.010 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.010 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.010 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.011 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.011 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.011 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.011 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.011 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.011 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.012 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.012 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.012 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.012 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.013 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.013 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.013 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.013 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.013 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.014 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.014 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.014 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.014 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.014 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.015 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.015 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.015 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.015 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.015 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.015 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.016 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.017 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.017 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.017 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.018 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.018 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.018 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.019 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.019 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.019 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.019 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.019 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.019 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.020 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.020 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.020 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2023-07-21T22:37:16.021 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.021 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.021 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.021 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2023-07-21T22:37:16.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2023-07-21T22:37:16.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:37:16.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.023 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.023 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.023 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.023 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.024 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:16.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:16.024 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.024 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2023-07-21T22:37:16.025 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.026 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.026 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.026 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.026 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.027 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.028 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.029 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.029 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.029 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.029 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.029 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.030 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.030 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.030 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.030 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.030 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.031 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.031 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.031 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.031 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.031 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.031 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.032 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.032 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.032 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.033 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.033 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.033 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.034 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.035 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.035 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.035 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.035 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.035 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.035 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.036 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.036 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.036 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.036 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.036 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.037 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.037 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.037 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.037 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.038 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.038 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.038 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.038 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.039 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.039 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.039 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.040 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.040 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2023-07-21T22:37:16.041 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.041 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.041 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.041 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2023-07-21T22:37:16.041 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2023-07-21T22:37:16.042 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.042 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.042 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.042 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.042 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.043 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.043 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.043 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:37:16.043 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.043 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.043 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.044 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.044 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2023-07-21T22:37:16.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2023-07-21T22:37:16.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.045 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.045 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.045 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.045 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.046 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:37:16.046 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.046 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.046 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2023-07-21T22:37:16.047 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2023-07-21T22:37:16.047 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.047 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.047 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.048 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.048 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.048 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.048 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:37:16.048 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.049 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.049 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.049 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.049 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2023-07-21T22:37:16.049 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2023-07-21T22:37:16.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.050 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.050 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.050 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.050 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.050 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.051 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.051 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:37:16.051 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.051 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.051 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.052 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.052 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2023-07-21T22:37:16.052 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2023-07-21T22:37:16.052 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.052 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.053 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.053 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.053 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.053 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:37:16.054 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.054 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2023-07-21T22:37:16.055 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2023-07-21T22:37:16.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.055 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.055 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.056 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2023-07-21T22:37:16.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2023-07-21T22:37:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.059 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2023-07-21T22:37:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2023-07-21T22:37:16.060 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:37:16.061 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-07-21T22:37:16.063 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-07-21T22:37:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.064 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.065 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2023-07-21T22:37:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2023-07-21T22:37:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.067 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.068 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2023-07-21T22:37:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2023-07-21T22:37:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.069 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2023-07-21T22:37:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.070 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2023-07-21T22:37:16.071 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2023-07-21T22:37:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.072 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.073 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2023-07-21T22:37:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2023-07-21T22:37:16.074 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.075 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.076 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.077 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.078 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.079 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.081 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.082 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.083 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.084 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.085 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.086 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.087 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.088 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.089 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2023-07-21T22:37:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2023-07-21T22:37:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.090 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.091 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.092 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2023-07-21T22:37:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2023-07-21T22:37:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.093 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.094 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2023-07-21T22:37:16.095 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2023-07-21T22:37:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:37:16.096 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.097 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.098 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2023-07-21T22:37:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.099 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.100 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.101 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.102 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.103 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.103 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.103 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.103 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.103 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.104 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.105 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.106 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.107 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.108 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.109 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.110 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.111 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.112 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-07-21T22:37:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.113 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.114 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.115 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.116 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.117 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.118 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.119 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.120 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.121 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.122 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.123 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.124 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.126 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2023-07-21T22:37:16.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2023-07-21T22:37:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2023-07-21T22:37:16.128 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:37:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.129 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2023-07-21T22:37:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:37:16.130 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.131 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:37:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.132 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2023-07-21T22:37:16.133 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2023-07-21T22:37:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2023-07-21T22:37:16.134 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2023-07-21T22:37:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.135 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.136 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2023-07-21T22:37:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2023-07-21T22:37:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.137 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.138 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.139 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.140 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.141 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.142 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.143 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.144 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.145 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.146 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.147 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.148 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.149 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.150 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.151 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.152 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.152 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.152 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.152 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.152 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.153 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.153 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.153 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.154 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.154 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.154 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.154 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.154 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.155 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.155 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.155 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.155 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.155 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.155 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2023-07-21T22:37:16.156 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2023-07-21T22:37:16.156 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.156 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.156 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.156 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.157 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.157 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.157 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.157 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.157 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.158 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.158 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.158 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.158 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2023-07-21T22:37:16.158 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2023-07-21T22:37:16.158 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.159 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.159 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.159 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2023-07-21T22:37:16.159 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.159 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.160 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.160 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.160 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.160 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.160 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.161 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2023-07-21T22:37:16.161 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2023-07-21T22:37:16.161 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.161 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.161 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.162 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:37:16.162 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.162 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.162 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.162 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.163 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.163 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.163 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.163 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.164 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.164 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:16.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.164 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.164 INFO:teuthology.orchestra.run.smithi066.stdout: "gid": 25535, 2023-07-21T22:37:16.164 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "y", 2023-07-21T22:37:16.165 INFO:teuthology.orchestra.run.smithi066.stdout: "mgr_features": 4540138322906710015, 2023-07-21T22:37:16.165 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2023-07-21T22:37:16.165 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.165 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2023-07-21T22:37:16.165 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.166 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.166 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.166 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:16.166 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:16.166 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.167 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:16.167 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.167 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.167 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.168 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:37:16.168 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.168 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.168 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.168 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.169 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.169 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.169 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.170 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.170 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.170 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.171 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.171 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.171 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.171 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.171 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.171 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.172 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.172 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.172 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.172 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.173 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.173 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.173 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.173 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.173 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.174 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.174 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.174 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.174 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.174 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.175 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.175 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.175 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.175 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.175 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.176 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.176 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.176 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.176 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.176 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.177 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.177 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.177 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.177 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.177 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.177 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.178 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.178 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.178 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.178 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.178 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.179 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.179 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.179 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.180 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.180 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.180 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.181 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.181 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.181 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.181 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.181 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.182 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.182 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2023-07-21T22:37:16.182 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2023-07-21T22:37:16.182 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.183 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.183 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.183 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.183 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.183 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.183 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.184 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:37:16.184 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.184 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.184 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.184 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.185 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2023-07-21T22:37:16.185 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2023-07-21T22:37:16.185 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.185 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.185 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.186 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2023-07-21T22:37:16.186 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.186 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.186 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.186 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2023-07-21T22:37:16.187 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.187 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.187 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.187 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.187 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2023-07-21T22:37:16.187 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2023-07-21T22:37:16.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.188 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.188 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.188 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.188 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.189 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.189 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.189 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2023-07-21T22:37:16.189 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.189 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.190 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.190 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2023-07-21T22:37:16.190 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2023-07-21T22:37:16.190 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.190 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.191 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.191 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.191 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.191 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.191 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.192 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2023-07-21T22:37:16.192 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.192 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.192 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.193 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2023-07-21T22:37:16.193 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2023-07-21T22:37:16.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.193 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.193 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.194 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2023-07-21T22:37:16.194 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.194 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.194 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.194 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2023-07-21T22:37:16.195 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.195 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.195 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.195 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2023-07-21T22:37:16.195 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2023-07-21T22:37:16.196 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.196 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.196 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.196 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.196 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.197 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.197 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.197 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:37:16.197 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.197 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.198 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.198 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2023-07-21T22:37:16.198 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2023-07-21T22:37:16.198 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.199 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.199 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.199 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.199 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.199 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:37:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.200 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.201 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2023-07-21T22:37:16.201 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2023-07-21T22:37:16.201 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.201 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.201 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.202 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.202 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.202 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.202 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.202 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2023-07-21T22:37:16.203 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.203 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.203 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.203 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.203 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.204 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.204 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2023-07-21T22:37:16.204 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.204 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.204 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.205 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2023-07-21T22:37:16.205 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2023-07-21T22:37:16.205 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.205 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.205 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.206 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.206 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.206 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.206 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.206 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:37:16.207 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.207 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.207 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.207 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2023-07-21T22:37:16.208 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2023-07-21T22:37:16.208 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.208 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.208 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.208 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2023-07-21T22:37:16.208 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.209 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.209 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.209 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:37:16.209 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:37:16.209 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.210 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.210 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.210 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2023-07-21T22:37:16.210 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2023-07-21T22:37:16.210 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:16.211 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.211 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.211 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:16.211 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-07-21T22:37:16.211 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-07-21T22:37:16.212 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.212 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:37:16.212 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:37:16.212 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.212 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.213 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.213 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2023-07-21T22:37:16.213 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:37:16.213 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:16.213 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.214 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.214 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2023-07-21T22:37:16.214 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:16.214 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2023-07-21T22:37:16.214 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.214 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:37:16.215 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.215 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.215 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.215 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2023-07-21T22:37:16.216 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2023-07-21T22:37:16.216 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.216 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.216 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.216 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:37:16.217 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.217 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.217 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.217 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:37:16.217 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:37:16.218 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.218 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.218 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2023-07-21T22:37:16.218 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2023-07-21T22:37:16.218 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:16.219 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.219 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.219 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2023-07-21T22:37:16.219 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2023-07-21T22:37:16.219 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2023-07-21T22:37:16.220 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.220 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:37:16.220 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:37:16.220 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.220 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.221 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.221 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2023-07-21T22:37:16.221 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2023-07-21T22:37:16.221 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.221 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.222 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.222 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2023-07-21T22:37:16.222 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.222 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.222 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.222 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:37:16.223 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:37:16.223 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.223 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.223 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.223 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2023-07-21T22:37:16.224 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2023-07-21T22:37:16.224 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:16.224 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.224 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.224 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:16.225 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-07-21T22:37:16.225 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-07-21T22:37:16.225 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.225 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:37:16.225 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:37:16.226 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.226 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.226 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.226 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.226 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.226 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.227 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.227 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.227 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.227 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.227 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.228 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.228 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.228 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.228 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.228 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.229 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.230 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.230 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.230 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.230 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.230 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.231 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.231 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.231 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.231 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.232 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.232 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.232 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.232 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.232 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.233 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.233 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.233 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.233 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.234 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.234 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.234 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.234 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.234 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.234 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.235 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.235 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.235 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.235 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.235 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.236 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.236 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.236 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.236 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.236 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.237 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.237 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.237 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.237 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.237 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.238 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.238 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.238 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.238 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.238 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.239 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.239 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.239 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.239 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.239 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2023-07-21T22:37:16.240 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2023-07-21T22:37:16.240 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:16.240 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.240 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.240 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:37:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.241 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.242 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.242 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.242 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-07-21T22:37:16.242 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-07-21T22:37:16.242 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.243 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.243 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.243 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2023-07-21T22:37:16.243 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.243 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.243 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.244 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2023-07-21T22:37:16.244 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2023-07-21T22:37:16.244 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap" 2023-07-21T22:37:16.244 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.245 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2023-07-21T22:37:16.245 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.245 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.245 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.245 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2023-07-21T22:37:16.246 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2023-07-21T22:37:16.246 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.246 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.246 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.246 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.247 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.247 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.247 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.247 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:37:16.247 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.248 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.248 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.248 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:16.248 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:16.249 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.249 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.249 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.249 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:16.249 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.249 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.250 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.250 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:37:16.250 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.250 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.250 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.251 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.251 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2023-07-21T22:37:16.251 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2023-07-21T22:37:16.251 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.251 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.252 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.252 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:16.252 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:16.252 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.252 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.253 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:37:16.253 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:37:16.253 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.253 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.253 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2023-07-21T22:37:16.254 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:37:16.254 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:16.254 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.254 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.254 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.255 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.255 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.255 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.255 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:37:16.255 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.256 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.256 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.256 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.256 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.257 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.257 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2023-07-21T22:37:16.257 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.257 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.257 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.257 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2023-07-21T22:37:16.258 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2023-07-21T22:37:16.258 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:16.258 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.258 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.258 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-07-21T22:37:16.259 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.259 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.259 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.259 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:37:16.259 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.260 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.260 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.260 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2023-07-21T22:37:16.260 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2023-07-21T22:37:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2023-07-21T22:37:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.261 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.262 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.262 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:37:16.262 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.262 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.262 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.263 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2023-07-21T22:37:16.263 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2023-07-21T22:37:16.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.263 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.264 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.264 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2023-07-21T22:37:16.264 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.264 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.264 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.265 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:37:16.265 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.265 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.265 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.265 INFO:teuthology.orchestra.run.smithi066.stdout: "alertmanager_web_password": { 2023-07-21T22:37:16.266 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alertmanager_web_password", 2023-07-21T22:37:16.266 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.266 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.266 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.266 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.267 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.267 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.267 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.267 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Alertmanager web password", 2023-07-21T22:37:16.267 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.268 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.268 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.268 INFO:teuthology.orchestra.run.smithi066.stdout: "alertmanager_web_user": { 2023-07-21T22:37:16.268 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alertmanager_web_user", 2023-07-21T22:37:16.269 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.269 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.269 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.269 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.269 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.270 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.270 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.270 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Alertmanager web user", 2023-07-21T22:37:16.270 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.270 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.270 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.271 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2023-07-21T22:37:16.271 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2023-07-21T22:37:16.271 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.271 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.272 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.272 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.272 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.272 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.272 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.273 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:37:16.273 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:37:16.273 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.273 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.273 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.274 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2023-07-21T22:37:16.274 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2023-07-21T22:37:16.274 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.274 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.274 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:37:16.275 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.276 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.276 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.276 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:37:16.276 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:37:16.277 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:16.277 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.277 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.277 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2023-07-21T22:37:16.277 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.278 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.278 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.278 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:37:16.278 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.278 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.279 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.279 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2023-07-21T22:37:16.279 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2023-07-21T22:37:16.279 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.280 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.280 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.280 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.280 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.280 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.281 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.281 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:37:16.281 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.281 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.281 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.282 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2023-07-21T22:37:16.282 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2023-07-21T22:37:16.282 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.282 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.282 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.283 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.283 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.283 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.283 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.283 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:37:16.284 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.284 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.284 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.284 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2023-07-21T22:37:16.285 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2023-07-21T22:37:16.285 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.285 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.285 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.285 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.286 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.286 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.286 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.286 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:37:16.286 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.286 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.287 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.287 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.287 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2023-07-21T22:37:16.287 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2023-07-21T22:37:16.287 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.288 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.288 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.288 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:37:16.288 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.288 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.289 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.289 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:16.289 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.289 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.289 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.290 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2023-07-21T22:37:16.290 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2023-07-21T22:37:16.290 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.290 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.290 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.291 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:37:16.291 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.291 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.291 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.291 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:37:16.292 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.292 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.292 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.292 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.292 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2023-07-21T22:37:16.293 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:37:16.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.293 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.293 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.293 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:37:16.294 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.294 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.294 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.294 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2023-07-21T22:37:16.294 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.294 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.295 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.295 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.295 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2023-07-21T22:37:16.295 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2023-07-21T22:37:16.295 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.296 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.296 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.296 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:37:16.296 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.296 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.297 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.297 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:16.297 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.297 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.297 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.298 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2023-07-21T22:37:16.298 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2023-07-21T22:37:16.298 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.298 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.298 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.299 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:37:16.299 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.299 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.299 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.299 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2023-07-21T22:37:16.300 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.300 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.300 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.300 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.300 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2023-07-21T22:37:16.301 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:37:16.301 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.301 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.301 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.301 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:37:16.302 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.302 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.302 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.302 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:37:16.302 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.302 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.303 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.303 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.303 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2023-07-21T22:37:16.303 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:37:16.303 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.304 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.304 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.304 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:37:16.304 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.304 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.305 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.305 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:37:16.305 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.305 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.305 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.306 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.306 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2023-07-21T22:37:16.306 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:37:16.306 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.306 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.307 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.307 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:37:16.307 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.307 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.307 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.307 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2023-07-21T22:37:16.308 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.308 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.308 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.308 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2023-07-21T22:37:16.309 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2023-07-21T22:37:16.309 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.309 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.309 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.309 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:37:16.310 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.310 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.310 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.310 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2023-07-21T22:37:16.310 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.311 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.311 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.311 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2023-07-21T22:37:16.311 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2023-07-21T22:37:16.312 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.312 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.312 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.312 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:37:16.312 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.313 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.313 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.313 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2023-07-21T22:37:16.313 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.313 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.314 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.314 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.314 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2023-07-21T22:37:16.314 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2023-07-21T22:37:16.314 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.314 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.315 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.315 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:37:16.315 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.315 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.315 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.316 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:16.316 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.316 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.316 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.317 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2023-07-21T22:37:16.317 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2023-07-21T22:37:16.317 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.317 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.317 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.318 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:37:16.318 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.318 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.318 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.318 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:16.318 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.319 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.319 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.319 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.319 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2023-07-21T22:37:16.319 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2023-07-21T22:37:16.320 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.320 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.320 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.320 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:37:16.320 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.321 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.321 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.321 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2023-07-21T22:37:16.321 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.321 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.322 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.322 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.322 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2023-07-21T22:37:16.322 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:37:16.322 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.322 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.323 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.323 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:37:16.323 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.323 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.323 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.324 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:37:16.324 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.324 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.324 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2023-07-21T22:37:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2023-07-21T22:37:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.325 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.326 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.326 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.326 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.326 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:37:16.326 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.327 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.327 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.327 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.327 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2023-07-21T22:37:16.327 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:37:16.328 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.328 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.328 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.328 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:16.328 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.329 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.329 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.329 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:37:16.329 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.329 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.330 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.330 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.330 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:37:16.330 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:37:16.330 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.331 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.331 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.331 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2023-07-21T22:37:16.331 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.331 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.331 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.332 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:37:16.332 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.332 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.332 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.333 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2023-07-21T22:37:16.333 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2023-07-21T22:37:16.333 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.333 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.333 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.334 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2023-07-21T22:37:16.334 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.334 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.334 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.334 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:37:16.335 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.335 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.335 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.335 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.335 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2023-07-21T22:37:16.335 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2023-07-21T22:37:16.336 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.336 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.336 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.336 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2023-07-21T22:37:16.336 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.337 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.337 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.337 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:37:16.337 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.337 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.338 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.338 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.338 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2023-07-21T22:37:16.338 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2023-07-21T22:37:16.338 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.338 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.339 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:37:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2023-07-21T22:37:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2023-07-21T22:37:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.341 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.342 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:37:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.343 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2023-07-21T22:37:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2023-07-21T22:37:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.344 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:37:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.345 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.346 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.347 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.348 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.349 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2023-07-21T22:37:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2023-07-21T22:37:16.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.351 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:37:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.352 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.353 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.354 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:37:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.355 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.356 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.357 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.358 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.359 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.360 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.361 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:37:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:37:16.362 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.363 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:37:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.364 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:37:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:37:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.365 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2023-07-21T22:37:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.366 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:37:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.367 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2023-07-21T22:37:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2023-07-21T22:37:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.368 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:37:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.369 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2023-07-21T22:37:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2023-07-21T22:37:16.370 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.371 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:37:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.372 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2023-07-21T22:37:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2023-07-21T22:37:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.373 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.374 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2023-07-21T22:37:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.375 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-07-21T22:37:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-07-21T22:37:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.376 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2023-07-21T22:37:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2023-07-21T22:37:16.377 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2023-07-21T22:37:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:37:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.378 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.379 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2023-07-21T22:37:16.379 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:37:16.379 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.379 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.379 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.380 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:37:16.380 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.380 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.380 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.380 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:37:16.381 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.381 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.381 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.381 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_web_password": { 2023-07-21T22:37:16.382 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_web_password", 2023-07-21T22:37:16.382 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.382 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.382 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.382 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.383 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.383 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.383 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.383 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus web password", 2023-07-21T22:37:16.383 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.383 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.384 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.384 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_web_user": { 2023-07-21T22:37:16.384 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_web_user", 2023-07-21T22:37:16.384 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.385 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.385 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.385 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.385 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.385 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.386 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.386 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus web user", 2023-07-21T22:37:16.386 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.386 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.386 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.387 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2023-07-21T22:37:16.387 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2023-07-21T22:37:16.387 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.387 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.387 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.388 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.388 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.388 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.388 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.388 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:37:16.389 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.389 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.389 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.389 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2023-07-21T22:37:16.390 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2023-07-21T22:37:16.390 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.390 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.390 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.390 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.391 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.391 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.391 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.391 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:37:16.391 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.391 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.392 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.392 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2023-07-21T22:37:16.392 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2023-07-21T22:37:16.392 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.393 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.393 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.393 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.393 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.393 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.394 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.394 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:37:16.394 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.394 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.394 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.395 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2023-07-21T22:37:16.395 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2023-07-21T22:37:16.395 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.395 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.395 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.396 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.396 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.396 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.396 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.396 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:37:16.397 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.397 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.397 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.397 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.397 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2023-07-21T22:37:16.398 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:37:16.398 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.398 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.398 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.398 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.398 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.399 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.399 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.399 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:37:16.399 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.399 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.400 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.400 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2023-07-21T22:37:16.400 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2023-07-21T22:37:16.400 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.401 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.401 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.401 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2023-07-21T22:37:16.401 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.401 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.402 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.402 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:37:16.402 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.402 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.402 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.403 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2023-07-21T22:37:16.403 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2023-07-21T22:37:16.403 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.403 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.403 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.404 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.404 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.404 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.404 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.404 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:37:16.405 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.405 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.405 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.405 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2023-07-21T22:37:16.406 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2023-07-21T22:37:16.406 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.406 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.406 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.406 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.406 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.407 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.407 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.407 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:37:16.407 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.407 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.408 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.408 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2023-07-21T22:37:16.408 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2023-07-21T22:37:16.408 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.409 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.409 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.409 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.409 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.409 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.410 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.410 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:37:16.410 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.410 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.410 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.411 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2023-07-21T22:37:16.411 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:37:16.411 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.411 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.412 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:37:16.413 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.413 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.413 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.413 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.413 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2023-07-21T22:37:16.414 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:37:16.414 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.414 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.414 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.414 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.415 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.415 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.415 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.415 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:37:16.415 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.415 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.416 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.416 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.416 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2023-07-21T22:37:16.416 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:37:16.416 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.417 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.417 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.417 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.417 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.417 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.418 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.418 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:37:16.418 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.418 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.418 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.419 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.419 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.419 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.419 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.419 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2023-07-21T22:37:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.420 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.421 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.422 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.423 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.424 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.424 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.424 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.424 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.424 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.425 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.425 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.425 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.425 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.425 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.426 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.426 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.426 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.426 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.426 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.427 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.427 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.427 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.427 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.427 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.428 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.428 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.428 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.428 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.428 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.429 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.429 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.429 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.429 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.429 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.430 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.430 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.430 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.430 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.430 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.431 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.431 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.431 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.431 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.431 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.432 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.432 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.432 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.432 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.432 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.433 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2023-07-21T22:37:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2023-07-21T22:37:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.434 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2023-07-21T22:37:16.435 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.435 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.435 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.435 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:37:16.435 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.435 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.436 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.436 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2023-07-21T22:37:16.436 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2023-07-21T22:37:16.436 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.437 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.437 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.437 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2023-07-21T22:37:16.437 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.437 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.438 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.438 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:37:16.438 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.438 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.438 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.439 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.439 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.439 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.439 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2023-07-21T22:37:16.439 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.440 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.440 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.440 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:37:16.440 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:37:16.440 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.441 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.441 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.441 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.441 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.441 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.442 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.442 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.442 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.442 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.442 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.443 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.443 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:37:16.443 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:37:16.443 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.443 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.444 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.444 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.444 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.444 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.444 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.445 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.445 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.445 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.445 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.445 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.445 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:37:16.446 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:37:16.446 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.446 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.446 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.446 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.447 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.447 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.447 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.447 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.447 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.448 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.448 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.448 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.448 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:37:16.448 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:37:16.449 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.449 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.449 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.449 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.449 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.449 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.450 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.450 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.450 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.450 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.450 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.451 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.451 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:37:16.451 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:37:16.451 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.451 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.452 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.452 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.452 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.452 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.452 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.453 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.453 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.453 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.453 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.453 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.453 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:37:16.454 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:37:16.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.454 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.454 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.454 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.455 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.455 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.455 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.455 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.455 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.456 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.456 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.456 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:37:16.456 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:37:16.457 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.457 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.457 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.457 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.457 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.457 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.458 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.458 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.458 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.458 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.458 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.459 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.459 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:37:16.459 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:37:16.459 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.459 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.460 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.460 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.460 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.460 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.460 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.461 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:37:16.462 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:37:16.462 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.462 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.462 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.462 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.463 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.463 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.463 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.463 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.463 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.464 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.464 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.464 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.464 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:37:16.464 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:37:16.465 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.465 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.465 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.465 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.465 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.465 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.466 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.466 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.466 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.466 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.466 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.467 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:37:16.467 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:37:16.467 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.467 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.468 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.468 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.468 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.468 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.468 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.469 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.469 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.469 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.469 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.469 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:37:16.470 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:37:16.470 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.470 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.470 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.470 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.471 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.471 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.471 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.471 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.471 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.472 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.472 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.472 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:37:16.472 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:37:16.473 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.473 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.473 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.473 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.473 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.473 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.474 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.474 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.474 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.474 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.475 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:37:16.475 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:37:16.475 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.475 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.476 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.476 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.476 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.476 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.476 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.476 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.477 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.477 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.477 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.477 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:37:16.478 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:37:16.478 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.478 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.478 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.478 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.479 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.479 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.479 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.479 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.479 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.480 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.480 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.480 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:37:16.480 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:37:16.481 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.481 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.481 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.481 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.482 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.482 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.482 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.482 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.482 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.482 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.483 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.483 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2023-07-21T22:37:16.483 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:37:16.483 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.483 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.484 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.484 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.484 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.484 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.484 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.485 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.485 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.485 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.485 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.485 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:37:16.486 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:37:16.486 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.486 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.486 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.486 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.487 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.487 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.487 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.487 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.487 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.488 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.488 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.488 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:37:16.488 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:37:16.489 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.489 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.489 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.489 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.489 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.490 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.490 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.490 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.490 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.490 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.490 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.491 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:37:16.491 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:37:16.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.491 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.492 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.492 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.492 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.492 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.493 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.493 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.493 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.493 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.493 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:37:16.494 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:37:16.494 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.494 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.494 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.494 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.495 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.495 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.495 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.495 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.495 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.496 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.496 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.496 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:37:16.496 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:37:16.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.497 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.497 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.497 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.497 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.497 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.498 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.498 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.498 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.498 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.498 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.499 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.499 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:37:16.499 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:37:16.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.499 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.500 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.500 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.500 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.500 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.500 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.501 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.501 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.501 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.501 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.501 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:37:16.502 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:37:16.502 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.502 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.502 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.502 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.503 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.503 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.503 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.503 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.503 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.504 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.504 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.504 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.504 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:37:16.504 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:37:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.505 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.506 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.507 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.507 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:37:16.507 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:37:16.507 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.507 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.508 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.508 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.508 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.508 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.508 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.509 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.509 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.509 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.509 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.509 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.509 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:37:16.510 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:37:16.510 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.510 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.510 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.510 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.511 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.511 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.511 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.511 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.511 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:37:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:37:16.512 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.513 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.513 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.513 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.513 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.513 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.514 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.514 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.514 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.514 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.514 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.515 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.515 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:37:16.515 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:37:16.515 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.515 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.516 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.516 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.516 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.516 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.516 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.517 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:37:16.518 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:37:16.518 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.518 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.518 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.518 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.519 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.519 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.519 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.519 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.519 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.520 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.520 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.520 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.520 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:37:16.520 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:37:16.521 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.521 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.521 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.521 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.521 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.521 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.522 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.522 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.522 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.522 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.522 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.523 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.523 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:37:16.523 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:37:16.523 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.523 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.524 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.524 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.524 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.524 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.524 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.525 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.525 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.525 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.525 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.525 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.525 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:37:16.526 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:37:16.526 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.526 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.526 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.526 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:37:16.527 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.527 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.527 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.527 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.527 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.528 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.528 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.528 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.528 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:37:16.528 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:37:16.529 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.529 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.529 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.529 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.529 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.529 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.530 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.530 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.530 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.530 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.530 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.531 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.531 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:37:16.531 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:37:16.531 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.531 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.532 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.532 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2023-07-21T22:37:16.532 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.532 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.532 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.532 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.533 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.533 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.533 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.533 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.533 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:37:16.534 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:37:16.534 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.534 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.534 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.534 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2023-07-21T22:37:16.535 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.535 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.535 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.535 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.535 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.536 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.536 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.536 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:37:16.536 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:37:16.536 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.537 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.537 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.537 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.537 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.537 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.538 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.538 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.538 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.538 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.538 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.539 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.539 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:37:16.539 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:37:16.539 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.539 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.539 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.540 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:16.540 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.540 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.540 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.540 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.541 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.541 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.541 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.541 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.541 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:37:16.542 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:37:16.542 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.542 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.542 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.542 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.543 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.543 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.543 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.543 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.543 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.544 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.544 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.544 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.544 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:37:16.544 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:37:16.545 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.545 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.545 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.545 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.545 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.545 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.546 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.546 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.546 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.546 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.546 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.547 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.547 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:37:16.547 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:37:16.547 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.547 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.548 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.548 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:16.548 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.548 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.548 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.549 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.549 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.549 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.549 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.549 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.550 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:37:16.550 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:37:16.550 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.550 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.550 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.551 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.551 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.551 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.551 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.551 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.551 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.552 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.552 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.552 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.552 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:37:16.552 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:37:16.553 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.553 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.553 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.553 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:16.553 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.554 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.554 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.554 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.554 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.554 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2023-07-21T22:37:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2023-07-21T22:37:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.555 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.556 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.556 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.556 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.556 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.556 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.557 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.557 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.557 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.557 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.557 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.558 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2023-07-21T22:37:16.558 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2023-07-21T22:37:16.558 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.558 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.558 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.559 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.559 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.559 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.559 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.559 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.559 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.560 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.560 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.560 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.560 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2023-07-21T22:37:16.560 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2023-07-21T22:37:16.561 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.561 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.561 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.561 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.561 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.562 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.562 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.562 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:37:16.562 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.562 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.563 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.563 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.563 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2023-07-21T22:37:16.563 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2023-07-21T22:37:16.563 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.564 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.564 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.564 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2023-07-21T22:37:16.564 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.564 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.564 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.565 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.565 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.565 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.565 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.565 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.566 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-07-21T22:37:16.566 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-07-21T22:37:16.566 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.566 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.566 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.567 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.567 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.567 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.567 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.567 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.567 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.568 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.568 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.568 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.568 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.569 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.569 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.569 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.569 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.569 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.570 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.570 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.570 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.570 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.570 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.571 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.571 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.571 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.571 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.571 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.571 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.572 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.572 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.572 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.572 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.572 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.573 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.573 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.573 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.573 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.573 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.574 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.574 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.574 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.574 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.574 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.574 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.575 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.575 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.575 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.575 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.575 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.576 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.576 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.576 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.576 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.577 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.577 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.577 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.577 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.577 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.578 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.578 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.578 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.578 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.578 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.578 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.579 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.579 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.579 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.579 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.579 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.580 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.580 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.580 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.580 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.580 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.581 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.581 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.581 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.581 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.581 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.582 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2023-07-21T22:37:16.582 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2023-07-21T22:37:16.582 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.582 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.582 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.582 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2023-07-21T22:37:16.583 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.584 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.584 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.584 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.584 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:37:16.584 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:37:16.585 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.585 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.585 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.585 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.585 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.586 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.586 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.586 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.586 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.586 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.587 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.587 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.587 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:16.587 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:16.587 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.587 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.588 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.588 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-07-21T22:37:16.588 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.588 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.588 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.589 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.589 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.589 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.589 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.589 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.590 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:16.590 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:16.590 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.590 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.590 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.591 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2023-07-21T22:37:16.591 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.591 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.591 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.591 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.591 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.592 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.592 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.592 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.592 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-07-21T22:37:16.592 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-07-21T22:37:16.593 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.593 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.593 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.593 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.593 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.594 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.594 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.594 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.594 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.594 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.595 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.595 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.595 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2023-07-21T22:37:16.595 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2023-07-21T22:37:16.595 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.595 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.596 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.596 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2023-07-21T22:37:16.596 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.596 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.596 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.597 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.597 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.597 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.597 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.597 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.598 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-07-21T22:37:16.598 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-07-21T22:37:16.598 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.598 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.598 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2023-07-21T22:37:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.599 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2023-07-21T22:37:16.600 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.600 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.600 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.600 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.600 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.601 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.601 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-07-21T22:37:16.601 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-07-21T22:37:16.601 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.601 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.602 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.602 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-07-21T22:37:16.602 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-07-21T22:37:16.602 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-07-21T22:37:16.602 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.603 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.603 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.603 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.603 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.603 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2023-07-21T22:37:16.604 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2023-07-21T22:37:16.604 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.604 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.604 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.604 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.605 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.605 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.605 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.605 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.605 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.606 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.606 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.606 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.606 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.606 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.607 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.607 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-07-21T22:37:16.607 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.607 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.607 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.607 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2023-07-21T22:37:16.608 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2023-07-21T22:37:16.608 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.608 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.608 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.608 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.609 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.609 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.609 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.609 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2023-07-21T22:37:16.609 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.610 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.610 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.610 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.610 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.610 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.611 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.611 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.611 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.611 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.612 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.612 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.612 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.612 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.613 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.613 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.613 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.613 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.614 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.614 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.614 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.614 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.614 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.615 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.615 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.615 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.615 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.615 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.616 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.616 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.616 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.616 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.616 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.617 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.617 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.617 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.617 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.618 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.618 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.618 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.618 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.618 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.619 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.619 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.619 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.619 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.619 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.620 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.620 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.620 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.620 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.620 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.621 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.621 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.621 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.621 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.622 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.622 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.622 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.622 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.622 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.623 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.623 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.623 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.623 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.623 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.623 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.624 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.624 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.624 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2023-07-21T22:37:16.624 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2023-07-21T22:37:16.624 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.625 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.625 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.625 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2023-07-21T22:37:16.625 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.625 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.626 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.626 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:37:16.626 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.626 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.626 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2023-07-21T22:37:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2023-07-21T22:37:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.627 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.628 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2023-07-21T22:37:16.628 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.628 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.628 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.628 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:37:16.629 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.629 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.629 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.629 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.629 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2023-07-21T22:37:16.630 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2023-07-21T22:37:16.630 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.630 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.630 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.630 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2023-07-21T22:37:16.631 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.631 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.631 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.631 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:37:16.631 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.631 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.632 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.632 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.632 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2023-07-21T22:37:16.632 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2023-07-21T22:37:16.632 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.633 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.633 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.633 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-07-21T22:37:16.633 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.633 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.634 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.634 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:37:16.634 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.634 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.634 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.635 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.635 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2023-07-21T22:37:16.635 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2023-07-21T22:37:16.635 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.635 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.635 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.636 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.636 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.636 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.636 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.636 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:37:16.637 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.637 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.637 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.637 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.637 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:16.638 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:16.638 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.638 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.638 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.638 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:16.639 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.639 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.639 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.639 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:37:16.639 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.639 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.640 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.640 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.640 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2023-07-21T22:37:16.640 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2023-07-21T22:37:16.640 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.641 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.641 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.641 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2023-07-21T22:37:16.641 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.641 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.642 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.642 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:37:16.642 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.642 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.642 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.643 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.643 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.643 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.643 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2023-07-21T22:37:16.643 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.644 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.644 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.644 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.644 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.644 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.645 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.645 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.645 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.645 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.645 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.646 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.646 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.646 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.646 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.646 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.647 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.647 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.647 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.647 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.647 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.647 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.648 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.648 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.648 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.648 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.649 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.649 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.649 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.649 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.649 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.651 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.651 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.651 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.651 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.651 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.652 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.652 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.652 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.652 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.652 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.653 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.653 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.653 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.653 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.653 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.654 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.655 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.656 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.657 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2023-07-21T22:37:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2023-07-21T22:37:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.658 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-07-21T22:37:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.659 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2023-07-21T22:37:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2023-07-21T22:37:16.660 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2023-07-21T22:37:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.661 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.662 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.663 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.664 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.665 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2023-07-21T22:37:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2023-07-21T22:37:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:37:16.666 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2023-07-21T22:37:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2023-07-21T22:37:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.667 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2023-07-21T22:37:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.668 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:37:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.669 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2023-07-21T22:37:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2023-07-21T22:37:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.670 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2023-07-21T22:37:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:37:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.671 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2023-07-21T22:37:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2023-07-21T22:37:16.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.673 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:37:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.674 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.675 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2023-07-21T22:37:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2023-07-21T22:37:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.676 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:37:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.677 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.678 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.679 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.680 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.681 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.682 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.683 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.684 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.685 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.686 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.687 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.688 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.689 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.690 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2023-07-21T22:37:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2023-07-21T22:37:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.691 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.692 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:37:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.693 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2023-07-21T22:37:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2023-07-21T22:37:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.694 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2023-07-21T22:37:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2023-07-21T22:37:16.695 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-07-21T22:37:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-07-21T22:37:16.696 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2023-07-21T22:37:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.697 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:37:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.698 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2023-07-21T22:37:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2023-07-21T22:37:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.699 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2023-07-21T22:37:16.700 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:37:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.701 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2023-07-21T22:37:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2023-07-21T22:37:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.702 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:37:16.703 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2023-07-21T22:37:16.704 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2023-07-21T22:37:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2023-07-21T22:37:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.705 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:37:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.706 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.707 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2023-07-21T22:37:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.708 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.709 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.710 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.711 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.712 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.713 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.714 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.714 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.714 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.714 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.714 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.715 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.715 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.715 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.715 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.715 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.716 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.717 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.718 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.719 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.720 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2023-07-21T22:37:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.722 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.723 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.724 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.725 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.726 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.727 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.728 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.729 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.730 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.731 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.732 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.733 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.734 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.735 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2023-07-21T22:37:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.737 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2023-07-21T22:37:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:37:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.738 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2023-07-21T22:37:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:37:16.739 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2023-07-21T22:37:16.740 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2023-07-21T22:37:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.741 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2023-07-21T22:37:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:37:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.742 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.743 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.744 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.745 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.746 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.747 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.748 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.749 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.750 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.751 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.752 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.753 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.754 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.755 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.756 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2023-07-21T22:37:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.757 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2023-07-21T22:37:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2023-07-21T22:37:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.758 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2023-07-21T22:37:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:37:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:37:16.759 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.760 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.761 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.762 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.763 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.764 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.765 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.766 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.767 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.768 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.769 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.770 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.771 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.772 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.773 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2023-07-21T22:37:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2023-07-21T22:37:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.774 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:37:16.775 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:37:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2023-07-21T22:37:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2023-07-21T22:37:16.776 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2023-07-21T22:37:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.777 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:37:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.778 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2023-07-21T22:37:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2023-07-21T22:37:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.779 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2023-07-21T22:37:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.780 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:37:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.781 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2023-07-21T22:37:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2023-07-21T22:37:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.782 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:37:16.783 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.784 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.784 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.784 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.784 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2023-07-21T22:37:16.784 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2023-07-21T22:37:16.784 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.785 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.785 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.785 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2023-07-21T22:37:16.785 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.785 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.786 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.786 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:37:16.786 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:37:16.786 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.786 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.787 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.787 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.787 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.787 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.787 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2023-07-21T22:37:16.788 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.788 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.788 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.788 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.788 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.788 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.789 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.789 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.789 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.789 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.789 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.790 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.790 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.790 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.790 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.790 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.791 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.791 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.791 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.791 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.791 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.791 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.792 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.792 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.792 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.792 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.792 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.793 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.793 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.793 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.793 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.793 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.794 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.794 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.794 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.794 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.794 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.795 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.795 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.795 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.795 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.795 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.795 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.796 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.796 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.796 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.796 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.796 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.797 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.797 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.797 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.797 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.797 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.798 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.798 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.798 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.798 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.798 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.799 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.799 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.799 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.799 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.799 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.800 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.800 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.800 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.800 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.800 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.801 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.801 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.801 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.802 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.802 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.802 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.802 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.802 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2023-07-21T22:37:16.803 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.803 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.803 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.803 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.803 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.803 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.804 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.804 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.804 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.804 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.804 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.805 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.805 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.805 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.805 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.805 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.806 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.806 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.806 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.806 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.806 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.807 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.807 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.807 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.807 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.807 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.808 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.808 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.808 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.808 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.808 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.809 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.809 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.809 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.809 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.809 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.809 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.810 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.810 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.810 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.810 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.810 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.811 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.811 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.811 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.811 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.811 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.812 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.812 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.812 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.812 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.812 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.813 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.813 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.813 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.813 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.813 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.813 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.814 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.814 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.814 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.814 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.814 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.815 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.815 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.815 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.815 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.815 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.816 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.816 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.816 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.816 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.816 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.817 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.817 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.817 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.817 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2023-07-21T22:37:16.817 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.817 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.818 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.818 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.818 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.818 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.818 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.819 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.819 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.819 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.819 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.819 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.820 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.820 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.820 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.820 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.820 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.820 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.821 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.822 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.822 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.822 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.822 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.823 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.823 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.823 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.823 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.823 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.824 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.824 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.824 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.824 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.824 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.825 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.825 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.825 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.825 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.825 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.826 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.826 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.826 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.826 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.826 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.827 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.827 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.827 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.827 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.827 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.827 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.828 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.828 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.828 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.828 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.828 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.829 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.829 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.829 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.829 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.829 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.830 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.830 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.830 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.830 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.830 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.831 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.831 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.831 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.831 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.831 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.832 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-07-21T22:37:16.832 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.832 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.832 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.832 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2023-07-21T22:37:16.833 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2023-07-21T22:37:16.833 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.833 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.833 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.833 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.834 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.834 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.834 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.834 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:37:16.834 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.835 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.835 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.835 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.835 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.835 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.835 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.836 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.836 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.836 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.836 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.836 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.837 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.838 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.838 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.838 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.838 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.838 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.839 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.839 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.839 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.839 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.839 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.840 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.840 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.840 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.840 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.840 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.841 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.841 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.841 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.841 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.841 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.841 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.842 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.842 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.842 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.842 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.842 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.843 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.843 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.843 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.843 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.843 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.844 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.844 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.844 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.844 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.844 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.845 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.845 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.845 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.845 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.845 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.845 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.846 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.846 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.846 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.846 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.847 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.847 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.847 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.847 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.847 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.848 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.848 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.848 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.848 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.848 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.849 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2023-07-21T22:37:16.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-07-21T22:37:16.849 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.849 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.849 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.850 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.850 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.850 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.850 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.850 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-07-21T22:37:16.851 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2023-07-21T22:37:16.851 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2023-07-21T22:37:16.851 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.851 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2023-07-21T22:37:16.851 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.851 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.852 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.852 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.852 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.852 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.853 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2023-07-21T22:37:16.853 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.853 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.853 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.853 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.854 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.854 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.854 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.854 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.854 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.855 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.855 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.855 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.855 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.855 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.855 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.856 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.856 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.856 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.856 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.856 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.857 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.857 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.857 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.857 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.857 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.858 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.858 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.858 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.858 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.859 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.859 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.859 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.859 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.859 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.860 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.860 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.860 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.860 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.860 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.861 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.861 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.861 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.861 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.861 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.862 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.862 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.862 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.862 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.862 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.863 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.863 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.863 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.863 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.863 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.864 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.864 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.864 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.864 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.864 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.865 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.865 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.865 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.865 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.865 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.865 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.866 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.866 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.866 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.866 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.866 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.867 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.867 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.867 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.867 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.867 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2023-07-21T22:37:16.868 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.868 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.868 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.868 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.868 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.869 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.869 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.869 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.869 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.869 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.870 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.870 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.870 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.870 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.870 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.871 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.871 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.871 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.871 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.871 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.872 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.872 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.872 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.872 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.872 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.873 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.873 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.873 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.873 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.873 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.874 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.874 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.874 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.874 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.874 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.874 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.875 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.875 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.875 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.875 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.875 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.876 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.876 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.876 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.876 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.876 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.877 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.877 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.877 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.877 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.877 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.878 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.878 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.878 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.878 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.878 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.879 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.879 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.879 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.879 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.879 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.879 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.880 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.880 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.880 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.880 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.880 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.881 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.882 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.882 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.882 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2023-07-21T22:37:16.882 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.883 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.883 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.883 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.883 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.883 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.884 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.884 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.884 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.884 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.884 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.885 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.885 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.885 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.885 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.885 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.885 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.886 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.886 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.886 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.886 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.886 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.887 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.887 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.887 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.887 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.887 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.888 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.888 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.888 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.888 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.888 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.888 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.889 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.889 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.889 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.889 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.890 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.890 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.890 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.890 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.890 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.891 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.891 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.891 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.891 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.891 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.892 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.892 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.892 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.892 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.892 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.893 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.893 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.893 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.893 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.893 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.894 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.894 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.894 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.894 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.894 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.895 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.895 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.895 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.895 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.895 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.895 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.896 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.896 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.896 INFO:teuthology.orchestra.run.smithi066.stdout: "noautoscale": { 2023-07-21T22:37:16.896 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "noautoscale", 2023-07-21T22:37:16.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.897 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.897 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.897 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.898 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.898 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.898 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "global autoscale flag", 2023-07-21T22:37:16.898 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:37:16.898 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.899 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.899 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.899 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:16.899 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:16.899 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.899 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.900 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.900 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:16.900 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.900 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.900 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.901 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.901 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.901 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.901 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.901 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.902 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2023-07-21T22:37:16.902 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2023-07-21T22:37:16.902 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:16.902 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.902 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.903 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-07-21T22:37:16.903 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2023-07-21T22:37:16.903 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.903 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.903 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2023-07-21T22:37:16.903 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:37:16.904 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.904 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.904 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.904 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.905 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.905 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2023-07-21T22:37:16.905 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.905 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.905 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.906 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2023-07-21T22:37:16.906 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:37:16.906 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.906 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.906 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.907 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.907 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.907 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.907 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.907 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:37:16.907 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.908 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.908 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.908 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-07-21T22:37:16.908 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-07-21T22:37:16.909 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.909 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.909 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.909 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.909 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.910 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.910 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.910 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.910 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.910 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.910 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.911 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.911 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.911 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.911 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.911 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.912 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.912 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.912 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.912 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.912 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.913 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.913 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.913 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.913 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.913 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.914 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.914 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.914 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.914 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.914 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.915 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.915 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.915 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.915 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.915 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.916 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.916 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.916 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.916 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.916 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.916 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.917 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.917 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.917 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.917 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.918 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.918 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.918 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.919 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.919 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.919 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.919 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.919 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.920 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.920 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.920 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.920 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.920 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.920 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.921 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2023-07-21T22:37:16.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2023-07-21T22:37:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2023-07-21T22:37:16.925 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.926 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.926 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.926 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:37:16.926 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.926 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.927 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.927 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.927 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:16.927 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:16.927 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:16.928 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.928 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.928 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:16.928 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.928 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.929 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.929 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:37:16.929 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.929 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.929 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.930 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.930 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.930 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.930 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.930 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2023-07-21T22:37:16.930 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.931 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.931 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.931 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2023-07-21T22:37:16.931 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2023-07-21T22:37:16.931 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.932 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.932 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.932 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.932 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.932 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.933 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2023-07-21T22:37:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2023-07-21T22:37:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.934 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.935 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.935 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.935 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.935 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.935 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:37:16.936 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:37:16.936 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.936 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.936 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.936 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.937 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.937 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.937 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.937 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.937 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.938 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.938 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.938 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.938 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.938 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.938 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.939 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.939 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.939 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.939 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.939 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.940 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.940 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.940 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.940 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.941 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.941 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.941 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.941 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.942 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.942 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.942 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.942 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.943 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.943 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.943 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.944 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.944 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.944 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.944 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.945 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.945 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.945 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.945 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.946 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.946 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.946 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.946 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.946 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.947 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.947 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.947 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.947 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.947 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.948 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.948 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.948 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.948 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.948 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.949 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.949 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.949 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.949 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.950 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2023-07-21T22:37:16.950 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2023-07-21T22:37:16.950 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.950 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.951 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.951 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.951 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.951 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.951 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.952 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.952 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.952 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.952 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.952 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.953 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:37:16.953 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:37:16.953 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.953 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.953 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.954 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2023-07-21T22:37:16.954 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.954 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.954 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.955 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.955 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.955 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2023-07-21T22:37:16.955 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2023-07-21T22:37:16.956 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:16.956 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.956 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.956 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2023-07-21T22:37:16.956 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.957 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.957 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.957 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.957 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.957 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.958 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.958 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.958 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:16.958 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:16.958 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.959 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.959 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-07-21T22:37:16.959 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.960 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:37:16.960 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.960 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.960 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.961 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:16.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:16.961 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.961 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.961 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.962 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2023-07-21T22:37:16.962 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.962 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.962 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.962 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:37:16.963 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.963 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.963 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.963 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.963 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2023-07-21T22:37:16.964 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2023-07-21T22:37:16.964 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.964 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.964 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.964 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2023-07-21T22:37:16.965 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.965 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.965 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.965 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.966 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.966 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.966 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-07-21T22:37:16.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-07-21T22:37:16.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.967 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.967 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.967 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2023-07-21T22:37:16.967 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.968 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.968 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2023-07-21T22:37:16.968 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2023-07-21T22:37:16.968 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.968 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.969 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.969 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.969 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.969 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-07-21T22:37:16.970 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-07-21T22:37:16.970 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.970 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.970 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-07-21T22:37:16.970 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-07-21T22:37:16.971 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-07-21T22:37:16.971 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.971 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.971 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.971 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.972 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.972 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.972 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.972 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.972 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.973 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-07-21T22:37:16.973 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.973 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.973 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.973 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:16.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:16.974 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.974 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.974 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.974 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.974 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.975 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.975 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.975 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.976 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.976 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.976 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.976 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.976 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.977 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.977 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.977 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.977 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.977 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:16.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:16.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.978 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.979 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.979 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.979 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.980 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.980 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.980 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.980 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:16.980 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:16.981 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.981 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.981 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.981 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:16.981 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.982 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.982 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:16.982 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:16.982 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:16.982 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:16.983 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:16.983 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:16.983 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:16.983 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:16.983 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.984 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.984 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.984 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:16.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:16.985 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.985 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:16.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:16.986 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.986 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.986 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.986 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.986 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.987 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.987 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.987 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.987 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2023-07-21T22:37:16.987 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:37:16.988 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:16.988 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.988 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.988 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:16.988 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.989 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.990 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.990 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:37:16.990 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:37:16.990 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.990 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.991 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.991 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.991 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.991 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.991 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.992 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.992 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.992 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.992 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.992 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.992 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2023-07-21T22:37:16.993 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2023-07-21T22:37:16.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:16.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.993 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:16.994 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.994 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.994 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.994 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.995 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.995 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.995 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:16.995 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.996 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:16.996 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2023-07-21T22:37:16.996 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:16.996 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:16.996 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:16.996 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2023-07-21T22:37:16.997 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2023-07-21T22:37:16.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:16.997 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:16.997 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:16.997 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:16.998 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:16.998 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:16.998 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:16.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:16.998 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:16.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:16.999 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:16.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:16.999 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-07-21T22:37:16.999 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-07-21T22:37:17.000 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.000 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.000 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.000 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.000 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.000 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.001 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.001 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.001 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.001 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.001 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.002 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.002 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.002 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.002 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.002 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.003 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.003 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.003 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.003 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.003 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.004 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.004 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.004 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.004 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.004 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.004 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.005 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.005 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.005 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.005 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.005 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.006 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.006 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.006 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.006 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.007 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.007 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.007 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.007 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.007 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.008 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.008 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.008 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.008 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.008 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.009 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.009 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.009 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.009 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.009 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.010 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.010 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.010 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.010 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.010 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.010 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.011 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.011 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.011 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.011 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.011 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.012 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.012 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.012 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.012 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.012 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.013 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.013 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.013 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.013 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.013 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.013 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.014 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.014 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.014 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.014 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.014 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.015 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.015 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.015 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.015 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:17.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:17.016 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.016 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.016 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.016 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.016 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.017 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.017 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.017 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.017 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.017 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.018 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.018 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.018 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:17.018 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:17.018 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.019 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.019 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.019 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.019 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.019 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.019 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.020 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.020 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.020 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.020 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.021 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.021 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2023-07-21T22:37:17.021 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.022 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.022 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.022 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.022 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.022 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.023 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.023 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.023 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.023 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.023 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.024 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.024 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.024 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.024 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.024 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.024 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.025 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.025 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.025 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.025 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.025 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.026 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.026 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.026 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.026 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.026 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.027 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.027 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.027 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.027 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.027 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.028 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.028 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.028 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.028 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.028 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.029 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.029 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.029 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.029 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.029 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.029 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.030 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.030 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.030 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.030 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.030 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.031 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.031 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.031 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.031 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.031 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.032 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.032 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.032 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.032 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.033 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.033 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.033 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.034 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.034 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.035 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.035 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.035 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.036 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2023-07-21T22:37:17.036 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.036 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.037 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.037 INFO:teuthology.orchestra.run.smithi066.stdout: "drive_group_interval": { 2023-07-21T22:37:17.037 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "drive_group_interval", 2023-07-21T22:37:17.037 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:17.037 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.038 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300.0", 2023-07-21T22:37:17.038 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.038 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.038 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.038 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:37:17.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.039 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.039 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.040 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.040 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.040 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.041 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.041 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.041 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.041 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.042 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.042 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.042 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.042 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.042 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.043 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.043 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.043 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.043 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.044 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.044 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.045 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.045 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.045 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.045 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.045 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.046 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.046 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.046 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.046 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.046 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.046 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.047 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.047 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.047 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.047 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.047 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.048 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.048 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.048 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.048 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.048 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.049 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.049 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.049 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.049 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.049 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.050 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.050 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.050 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.050 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.050 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.051 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.051 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.051 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.051 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.052 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.052 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.052 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.052 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.052 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.053 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.053 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2023-07-21T22:37:17.053 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2023-07-21T22:37:17.053 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.053 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.053 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.054 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2023-07-21T22:37:17.054 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.054 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.054 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:37:17.055 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.055 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.055 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.055 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.056 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.056 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.056 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2023-07-21T22:37:17.056 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.056 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.057 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.057 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.057 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.057 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.057 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.057 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.058 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.058 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.058 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.058 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.058 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.059 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.059 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.059 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.059 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.059 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.060 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.061 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.061 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.061 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.061 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.061 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.062 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.062 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.062 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.062 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.062 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.063 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.063 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.063 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.063 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.063 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.064 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.064 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.064 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.064 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.065 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.065 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.065 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.065 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.065 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.066 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.066 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.066 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.066 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.066 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.066 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.067 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.067 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.067 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.067 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.067 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.068 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.068 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.068 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.068 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.068 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.069 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.069 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.069 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.069 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.070 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.070 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.070 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.070 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2023-07-21T22:37:17.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2023-07-21T22:37:17.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.071 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.071 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.071 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.071 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.071 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.072 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.072 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.072 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.072 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.072 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.073 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.073 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2023-07-21T22:37:17.073 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2023-07-21T22:37:17.073 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.073 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.074 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.074 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2023-07-21T22:37:17.074 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.074 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.074 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.074 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.075 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.075 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.075 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.075 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2023-07-21T22:37:17.076 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2023-07-21T22:37:17.076 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.076 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.076 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.076 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.077 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.077 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.077 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.077 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.077 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.078 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.078 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.078 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.078 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2023-07-21T22:37:17.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2023-07-21T22:37:17.078 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.079 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.079 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.079 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.079 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.079 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.080 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.080 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.080 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.080 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.081 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.081 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2023-07-21T22:37:17.081 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2023-07-21T22:37:17.081 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:17.081 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.082 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.082 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.082 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.082 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.082 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.083 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.083 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.083 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.083 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.083 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2023-07-21T22:37:17.084 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2023-07-21T22:37:17.084 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.084 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.084 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.084 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.085 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.085 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.085 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.085 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.085 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.086 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.086 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.086 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.086 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2023-07-21T22:37:17.086 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2023-07-21T22:37:17.086 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.087 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.087 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.087 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.087 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.087 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.088 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.088 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.088 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.088 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.088 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.089 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.089 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2023-07-21T22:37:17.089 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2023-07-21T22:37:17.089 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.089 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.089 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.090 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.090 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.090 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.090 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.090 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.091 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.091 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.091 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.091 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2023-07-21T22:37:17.092 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2023-07-21T22:37:17.092 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.092 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.092 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.092 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.093 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:17.093 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2023-07-21T22:37:17.093 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.093 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.093 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.093 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.094 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.094 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2023-07-21T22:37:17.094 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2023-07-21T22:37:17.094 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.095 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.095 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.095 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.095 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.095 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.096 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.096 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.096 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.096 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.096 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.097 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.097 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2023-07-21T22:37:17.097 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2023-07-21T22:37:17.097 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.097 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.098 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.098 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.098 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.098 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.098 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.099 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.099 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.099 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.099 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.099 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.100 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2023-07-21T22:37:17.100 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2023-07-21T22:37:17.100 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.100 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.100 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.101 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.101 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.101 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.101 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.101 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.101 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.102 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.102 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.102 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.102 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.102 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.103 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.103 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-07-21T22:37:17.103 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.103 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.103 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.104 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2023-07-21T22:37:17.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2023-07-21T22:37:17.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.105 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.105 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.105 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.105 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.106 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.106 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:37:17.106 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.106 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.107 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.107 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2023-07-21T22:37:17.107 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2023-07-21T22:37:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.108 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.109 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.109 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.109 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.109 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:37:17.109 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.110 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.110 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.110 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.111 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.111 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.111 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.111 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.112 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.112 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.112 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.112 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.113 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.113 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.113 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.113 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.114 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.114 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.114 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.114 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.115 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.115 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.115 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.115 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.116 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.116 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.116 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.116 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.117 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.117 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.117 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.117 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.117 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.118 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.118 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.118 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.118 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.119 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.119 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.119 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.119 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.120 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.120 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.120 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.120 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.121 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.121 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.121 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.121 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.122 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.122 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.122 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.122 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.123 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.123 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.123 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.123 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.123 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.123 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.124 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.124 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.124 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.124 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.124 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.125 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.125 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.125 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.125 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.126 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.126 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.126 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.126 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2023-07-21T22:37:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.127 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.128 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.128 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.128 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.128 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.128 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.129 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.129 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.129 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.129 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.129 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.130 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.130 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.130 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.130 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.130 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.130 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.131 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.131 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.131 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.131 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.131 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.132 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.132 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.132 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.132 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.132 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.133 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.133 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.133 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.133 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.133 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.134 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.134 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.134 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.134 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.134 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.134 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.135 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.135 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.135 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.135 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.135 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.136 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.136 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.136 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.136 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.136 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.137 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.137 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.137 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.137 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.137 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.138 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.138 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.138 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.138 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.139 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.139 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.139 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.139 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.139 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.140 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.140 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.140 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.140 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.141 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.141 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.141 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.141 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.141 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.142 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2023-07-21T22:37:17.142 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.142 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.142 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.142 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.142 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.143 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.143 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.143 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.143 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.143 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.144 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.144 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.144 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.144 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.144 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.145 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.145 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.145 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.145 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.145 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.146 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.146 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.146 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.147 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.147 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.147 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.148 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.148 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.148 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.148 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.149 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.149 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.149 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.150 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.150 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.150 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.150 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.150 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.151 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.152 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.152 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.152 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.152 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.152 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.153 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.153 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.153 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.153 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.153 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.154 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.154 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.154 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.154 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.154 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.154 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.155 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.155 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.155 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.155 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.155 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.156 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.156 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.156 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.156 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2023-07-21T22:37:17.156 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.157 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.157 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.157 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2023-07-21T22:37:17.157 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2023-07-21T22:37:17.157 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.158 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.158 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:37:17.158 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.158 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.159 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.159 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.159 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.159 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.159 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.159 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.160 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:17.160 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:17.160 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.160 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.160 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.161 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2023-07-21T22:37:17.161 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.161 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.161 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.161 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.162 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.162 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.162 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.162 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.162 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.162 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.163 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.163 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.163 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.163 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.163 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.164 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.164 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.164 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.164 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.165 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.165 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.165 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.165 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.165 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.166 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.166 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.166 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.166 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.166 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.167 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.167 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.167 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.167 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.168 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.169 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.169 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.169 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.170 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.170 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.170 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.170 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.170 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.171 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.171 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.171 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.171 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.171 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.172 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.172 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.172 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.172 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.172 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.173 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.173 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.173 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.173 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.173 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.174 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.174 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.174 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.174 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.175 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.175 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.175 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.175 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.175 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.175 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.176 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.176 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.176 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2023-07-21T22:37:17.176 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.177 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.177 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.177 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2023-07-21T22:37:17.177 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2023-07-21T22:37:17.177 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.178 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.178 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.178 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.178 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.178 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.179 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.179 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:37:17.179 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.179 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.179 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.180 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.180 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2023-07-21T22:37:17.180 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2023-07-21T22:37:17.180 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.180 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.180 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.181 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.181 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.181 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.181 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.181 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:37:17.182 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.182 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.182 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.182 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.182 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2023-07-21T22:37:17.183 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2023-07-21T22:37:17.183 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.183 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.183 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.183 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:37:17.184 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.185 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.185 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.185 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2023-07-21T22:37:17.185 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2023-07-21T22:37:17.185 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.186 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.186 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.186 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.186 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.186 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.187 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.187 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:37:17.187 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.187 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.187 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.188 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.188 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2023-07-21T22:37:17.188 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2023-07-21T22:37:17.188 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.188 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.188 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.189 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.189 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.189 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.189 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.189 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:37:17.190 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.190 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.190 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.190 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.190 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2023-07-21T22:37:17.191 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2023-07-21T22:37:17.191 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.191 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.191 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.191 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.192 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.192 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.192 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.192 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.192 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.192 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.193 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.193 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.193 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2023-07-21T22:37:17.193 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2023-07-21T22:37:17.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.194 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.194 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.194 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.194 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.194 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.195 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.195 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.195 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.195 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.195 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.196 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.196 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2023-07-21T22:37:17.196 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2023-07-21T22:37:17.196 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.196 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.197 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.197 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:37:17.197 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.197 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.197 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.197 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.198 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.198 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.198 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.198 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.198 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-07-21T22:37:17.199 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-07-21T22:37:17.199 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.199 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.199 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.199 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.200 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.200 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.200 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.200 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.200 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.200 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.201 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.201 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.201 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:17.201 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:17.201 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.202 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.202 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.202 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2023-07-21T22:37:17.202 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2023-07-21T22:37:17.202 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.203 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.203 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.203 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.203 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.203 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.204 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.204 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2023-07-21T22:37:17.204 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2023-07-21T22:37:17.204 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.204 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.204 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.205 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2023-07-21T22:37:17.205 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.205 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.205 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.205 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.206 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.206 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.206 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.206 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.206 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2023-07-21T22:37:17.207 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2023-07-21T22:37:17.207 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.207 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.207 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.207 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.207 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.208 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.208 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.208 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.208 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.208 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.209 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.209 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.209 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2023-07-21T22:37:17.209 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2023-07-21T22:37:17.209 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.210 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.210 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.210 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.210 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.210 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.212 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.212 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.212 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.212 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.212 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.213 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.213 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.213 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.213 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.213 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.214 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.214 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.214 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.214 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.214 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.215 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.215 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.215 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.215 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.215 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.215 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.216 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.216 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.216 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.216 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.216 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.217 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.217 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.217 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.217 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.217 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.218 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.218 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.218 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.218 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.218 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.219 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.219 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.219 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.219 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.219 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.219 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.220 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.220 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.220 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.220 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.220 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.221 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.221 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.221 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.221 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.221 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.222 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.222 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.222 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.222 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.222 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.223 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.223 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.223 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.223 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.223 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.223 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.224 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.224 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.224 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.224 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.224 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.225 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.225 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.225 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2023-07-21T22:37:17.225 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2023-07-21T22:37:17.225 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.226 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.226 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.226 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.226 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.226 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.227 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.227 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.227 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.227 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.227 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.227 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.228 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2023-07-21T22:37:17.228 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2023-07-21T22:37:17.228 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.228 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.228 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.229 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.229 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.229 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.229 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.229 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2023-07-21T22:37:17.230 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2023-07-21T22:37:17.231 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.231 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.231 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:37:17.231 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.232 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.232 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.232 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.232 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.232 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.233 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.233 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.233 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.233 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2023-07-21T22:37:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.234 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.235 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.235 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.235 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.235 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.235 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.236 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.236 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.236 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.236 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.236 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.237 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.237 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.237 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.237 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.237 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.238 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.238 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.238 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.238 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.238 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.238 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.239 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.239 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.239 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.239 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.239 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.240 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.240 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.240 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.240 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.240 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.241 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.241 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.241 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.241 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.241 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.242 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.242 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.242 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.242 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.242 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.242 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.243 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.243 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.243 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.243 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.243 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.244 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.245 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.246 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.247 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-07-21T22:37:17.248 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.249 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.250 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.251 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.252 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.253 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.254 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.256 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.257 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.258 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.259 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.260 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.261 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.262 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2023-07-21T22:37:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2023-07-21T22:37:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2023-07-21T22:37:17.263 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:37:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.264 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2023-07-21T22:37:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:37:17.265 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.266 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:37:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.267 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2023-07-21T22:37:17.268 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2023-07-21T22:37:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2023-07-21T22:37:17.269 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2023-07-21T22:37:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.270 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2023-07-21T22:37:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2023-07-21T22:37:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.272 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.273 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.274 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:17.275 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.276 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.277 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.278 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.279 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.280 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.281 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.282 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.283 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.284 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.285 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.286 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.287 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.288 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.289 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.290 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2023-07-21T22:37:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2023-07-21T22:37:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.291 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.292 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2023-07-21T22:37:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2023-07-21T22:37:17.293 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2023-07-21T22:37:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.294 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.295 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2023-07-21T22:37:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2023-07-21T22:37:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.296 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:37:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.297 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.298 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.299 INFO:teuthology.orchestra.run.smithi066.stdout: "modules": [ 2023-07-21T22:37:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-07-21T22:37:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "dashboard", 2023-07-21T22:37:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "iostat", 2023-07-21T22:37:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "nfs", 2023-07-21T22:37:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "restful", 2023-07-21T22:37:17.300 INFO:teuthology.orchestra.run.smithi066.stdout: "snap_schedule" 2023-07-21T22:37:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "available_modules": [ 2023-07-21T22:37:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alerts", 2023-07-21T22:37:17.301 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:17.302 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.303 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T22:37:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.304 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.305 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.306 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.307 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.308 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.309 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.310 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.311 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.312 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.313 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.314 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.315 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.316 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.317 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_destination": { 2023-07-21T22:37:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_destination", 2023-07-21T22:37:17.318 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.319 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email address to send alerts to", 2023-07-21T22:37:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.320 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_from_name": { 2023-07-21T22:37:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_from_name", 2023-07-21T22:37:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.321 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "Ceph", 2023-07-21T22:37:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.322 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Email From: name", 2023-07-21T22:37:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.323 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_host": { 2023-07-21T22:37:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_host", 2023-07-21T22:37:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.324 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP server", 2023-07-21T22:37:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.325 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_password": { 2023-07-21T22:37:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_password", 2023-07-21T22:37:17.326 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.327 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Password to authenticate with", 2023-07-21T22:37:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.328 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_port": { 2023-07-21T22:37:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_port", 2023-07-21T22:37:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.329 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "465", 2023-07-21T22:37:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.330 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP port", 2023-07-21T22:37:17.331 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.331 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.331 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.331 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.331 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_sender": { 2023-07-21T22:37:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_sender", 2023-07-21T22:37:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.333 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.333 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.333 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.333 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SMTP envelope sender", 2023-07-21T22:37:17.333 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_ssl": { 2023-07-21T22:37:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_ssl", 2023-07-21T22:37:17.334 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.335 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.335 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.335 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.335 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.335 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.336 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.336 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T22:37:17.336 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.336 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.336 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.337 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.337 INFO:teuthology.orchestra.run.smithi066.stdout: "smtp_user": { 2023-07-21T22:37:17.337 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "smtp_user", 2023-07-21T22:37:17.337 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.337 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.338 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.338 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.338 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.338 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.338 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.339 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "User to authenticate as", 2023-07-21T22:37:17.339 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.339 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.339 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.340 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.340 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.340 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "balancer", 2023-07-21T22:37:17.340 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.340 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.341 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.341 INFO:teuthology.orchestra.run.smithi066.stdout: "active": { 2023-07-21T22:37:17.341 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "active", 2023-07-21T22:37:17.341 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.341 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.342 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.342 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.342 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.342 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.342 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.343 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T22:37:17.343 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.343 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.343 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.343 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_time": { 2023-07-21T22:37:17.344 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_time", 2023-07-21T22:37:17.344 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.344 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.344 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.344 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0000", 2023-07-21T22:37:17.345 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.345 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.345 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.345 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T22:37:17.345 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:37:17.346 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.346 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.346 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.346 INFO:teuthology.orchestra.run.smithi066.stdout: "begin_weekday": { 2023-07-21T22:37:17.346 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "begin_weekday", 2023-07-21T22:37:17.347 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:17.347 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.347 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.347 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:17.347 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-07-21T22:37:17.347 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-07-21T22:37:17.348 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.348 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T22:37:17.348 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:37:17.348 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.348 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.349 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_max_iterations": { 2023-07-21T22:37:17.349 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_max_iterations", 2023-07-21T22:37:17.349 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:17.349 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.350 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "25", 2023-07-21T22:37:17.350 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:17.350 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "250", 2023-07-21T22:37:17.350 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.351 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T22:37:17.351 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.351 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.351 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.351 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.351 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_metrics": { 2023-07-21T22:37:17.352 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_metrics", 2023-07-21T22:37:17.352 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.352 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.352 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.352 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T22:37:17.353 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.353 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.353 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.353 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T22:37:17.353 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T22:37:17.354 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.354 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.354 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.354 INFO:teuthology.orchestra.run.smithi066.stdout: "crush_compat_step": { 2023-07-21T22:37:17.354 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crush_compat_step", 2023-07-21T22:37:17.354 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:17.355 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.355 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.355 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.5", 2023-07-21T22:37:17.355 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0.001", 2023-07-21T22:37:17.355 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "0.999", 2023-07-21T22:37:17.356 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.356 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "aggressiveness of optimization", 2023-07-21T22:37:17.356 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T22:37:17.356 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.356 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.357 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.357 INFO:teuthology.orchestra.run.smithi066.stdout: "end_time": { 2023-07-21T22:37:17.357 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_time", 2023-07-21T22:37:17.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.357 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.358 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.358 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2359", 2023-07-21T22:37:17.358 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.358 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.358 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.358 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T22:37:17.359 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T22:37:17.359 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.359 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.359 INFO:teuthology.orchestra.run.smithi066.stdout: "end_weekday": { 2023-07-21T22:37:17.360 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "end_weekday", 2023-07-21T22:37:17.360 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:17.360 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.360 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.360 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:17.361 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "0", 2023-07-21T22:37:17.361 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "6", 2023-07-21T22:37:17.361 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.361 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T22:37:17.361 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T22:37:17.362 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.362 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.362 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.362 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.362 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.362 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.363 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.363 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.363 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.363 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.363 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.364 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.364 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.364 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.364 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.364 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.365 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.365 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.365 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.365 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.365 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.365 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.366 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.366 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.366 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.366 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.366 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.367 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.367 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.367 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.367 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.367 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.368 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.368 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.368 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.368 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.368 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.369 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.369 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.369 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.369 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.369 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.370 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.370 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.370 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.370 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.370 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.370 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.371 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.371 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.371 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.371 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.371 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.372 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.372 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.372 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.372 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.372 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.373 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.373 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.373 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.373 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.373 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.374 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.374 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.374 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.374 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.374 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.374 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.375 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.375 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.375 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.375 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.375 INFO:teuthology.orchestra.run.smithi066.stdout: "min_score": { 2023-07-21T22:37:17.376 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_score", 2023-07-21T22:37:17.376 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:17.376 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.376 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.376 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:17.377 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.377 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.377 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.377 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T22:37:17.377 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.378 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.378 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.378 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.378 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-07-21T22:37:17.378 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-07-21T22:37:17.378 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.379 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.379 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.379 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "upmap", 2023-07-21T22:37:17.379 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.379 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.380 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.380 INFO:teuthology.orchestra.run.smithi066.stdout: "crush-compat", 2023-07-21T22:37:17.380 INFO:teuthology.orchestra.run.smithi066.stdout: "none", 2023-07-21T22:37:17.380 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap" 2023-07-21T22:37:17.380 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.381 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Balancer mode", 2023-07-21T22:37:17.381 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.381 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.381 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.381 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.381 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_ids": { 2023-07-21T22:37:17.382 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_ids", 2023-07-21T22:37:17.382 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.382 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.382 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.382 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.383 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.383 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.383 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.383 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T22:37:17.383 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.384 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.384 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.384 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.384 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:17.384 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:17.385 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.385 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.385 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.385 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:17.385 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.385 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.386 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.386 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T22:37:17.386 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.386 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.386 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.387 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_deviation": { 2023-07-21T22:37:17.387 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_deviation", 2023-07-21T22:37:17.387 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.387 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.388 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T22:37:17.389 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T22:37:17.389 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.389 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.389 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.389 INFO:teuthology.orchestra.run.smithi066.stdout: "upmap_max_optimizations": { 2023-07-21T22:37:17.390 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "upmap_max_optimizations", 2023-07-21T22:37:17.390 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:17.390 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.390 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.390 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.391 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.391 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.391 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.391 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T22:37:17.391 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.392 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.392 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.392 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.392 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.392 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cephadm", 2023-07-21T22:37:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.393 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_down_multiplier": { 2023-07-21T22:37:17.394 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_down_multiplier", 2023-07-21T22:37:17.394 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:17.394 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.394 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.394 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-07-21T22:37:17.395 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.395 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.395 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.395 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T22:37:17.395 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.396 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.396 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.396 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.396 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_refresh_rate": { 2023-07-21T22:37:17.396 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_refresh_rate", 2023-07-21T22:37:17.397 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.397 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.397 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.397 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "20", 2023-07-21T22:37:17.397 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.397 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.398 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.398 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T22:37:17.398 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.398 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.398 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.399 INFO:teuthology.orchestra.run.smithi066.stdout: "agent_starting_port": { 2023-07-21T22:37:17.399 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "agent_starting_port", 2023-07-21T22:37:17.399 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.399 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4721", 2023-07-21T22:37:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.400 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T22:37:17.401 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.401 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.401 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.401 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.401 INFO:teuthology.orchestra.run.smithi066.stdout: "alertmanager_web_password": { 2023-07-21T22:37:17.402 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alertmanager_web_password", 2023-07-21T22:37:17.402 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.402 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.402 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.402 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.403 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.403 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.403 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.403 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Alertmanager web password", 2023-07-21T22:37:17.403 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.403 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.404 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.404 INFO:teuthology.orchestra.run.smithi066.stdout: "alertmanager_web_user": { 2023-07-21T22:37:17.404 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "alertmanager_web_user", 2023-07-21T22:37:17.404 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.405 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.405 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.405 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.405 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.405 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.406 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.406 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Alertmanager web user", 2023-07-21T22:37:17.406 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.406 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.406 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.407 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.407 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_ptrace": { 2023-07-21T22:37:17.407 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_ptrace", 2023-07-21T22:37:17.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.407 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.407 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.408 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.408 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.408 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.408 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.408 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T22:37:17.409 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T22:37:17.409 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.409 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.409 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_interval": { 2023-07-21T22:37:17.410 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_interval", 2023-07-21T22:37:17.410 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.410 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.410 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.410 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:17.411 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.411 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.411 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.411 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T22:37:17.411 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.412 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.412 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.412 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.412 INFO:teuthology.orchestra.run.smithi066.stdout: "autotune_memory_target_ratio": { 2023-07-21T22:37:17.412 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T22:37:17.413 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:17.413 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.413 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.413 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0.7", 2023-07-21T22:37:17.413 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.413 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.414 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.414 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T22:37:17.414 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.414 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.414 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.415 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.415 INFO:teuthology.orchestra.run.smithi066.stdout: "cgroups_split": { 2023-07-21T22:37:17.415 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cgroups_split", 2023-07-21T22:37:17.415 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.415 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.416 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.416 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.416 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.416 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.416 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.417 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T22:37:17.417 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.417 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.417 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.417 INFO:teuthology.orchestra.run.smithi066.stdout: "config_checks_enabled": { 2023-07-21T22:37:17.418 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_checks_enabled", 2023-07-21T22:37:17.418 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.418 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.418 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.418 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.419 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.419 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.419 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.419 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T22:37:17.419 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.420 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.420 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.420 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.420 INFO:teuthology.orchestra.run.smithi066.stdout: "config_dashboard": { 2023-07-21T22:37:17.420 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_dashboard", 2023-07-21T22:37:17.421 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.421 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.421 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.421 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.421 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.421 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.422 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.422 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T22:37:17.422 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.422 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.422 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.423 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.423 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_alertmanager": { 2023-07-21T22:37:17.423 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_alertmanager", 2023-07-21T22:37:17.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.423 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.424 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.424 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T22:37:17.424 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.424 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.424 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.425 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:17.425 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.425 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.425 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.425 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_base": { 2023-07-21T22:37:17.426 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_base", 2023-07-21T22:37:17.426 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.426 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.426 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.426 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T22:37:17.427 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.427 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.427 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.427 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Container image name, without the tag", 2023-07-21T22:37:17.427 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.428 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.428 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.428 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_elasticsearch": { 2023-07-21T22:37:17.428 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_elasticsearch", 2023-07-21T22:37:17.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.429 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.429 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.429 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T22:37:17.429 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.429 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.430 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.430 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "elasticsearch container image", 2023-07-21T22:37:17.430 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.430 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.430 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.431 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.431 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_grafana": { 2023-07-21T22:37:17.431 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_grafana", 2023-07-21T22:37:17.431 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.431 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.432 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.432 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T22:37:17.432 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.432 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.432 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.432 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:17.433 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.433 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.433 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.433 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.433 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_haproxy": { 2023-07-21T22:37:17.434 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_haproxy", 2023-07-21T22:37:17.434 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.434 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.434 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T22:37:17.435 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.435 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.435 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.435 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "HAproxy container image", 2023-07-21T22:37:17.435 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.436 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.436 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.436 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.436 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_agent": { 2023-07-21T22:37:17.436 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_agent", 2023-07-21T22:37:17.436 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.437 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.437 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.437 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T22:37:17.437 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.437 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.438 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.438 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger agent container image", 2023-07-21T22:37:17.438 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.438 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.438 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.439 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.439 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_collector": { 2023-07-21T22:37:17.439 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_collector", 2023-07-21T22:37:17.439 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.439 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.439 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.440 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T22:37:17.440 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.440 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.440 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.440 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger collector container image", 2023-07-21T22:37:17.441 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.441 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.441 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.441 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.441 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_jaeger_query": { 2023-07-21T22:37:17.442 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_jaeger_query", 2023-07-21T22:37:17.442 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.442 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.442 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.442 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T22:37:17.443 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.443 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.443 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.443 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Jaeger query container image", 2023-07-21T22:37:17.443 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.443 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.444 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.444 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_keepalived": { 2023-07-21T22:37:17.444 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_keepalived", 2023-07-21T22:37:17.444 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.445 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.445 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.445 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T22:37:17.445 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.445 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.446 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.446 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Keepalived container image", 2023-07-21T22:37:17.446 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.446 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.446 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.447 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.447 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_loki": { 2023-07-21T22:37:17.447 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_loki", 2023-07-21T22:37:17.447 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.447 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.447 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.448 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T22:37:17.448 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.448 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.448 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.448 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Loki container image", 2023-07-21T22:37:17.449 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.449 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.449 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.449 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_node_exporter": { 2023-07-21T22:37:17.450 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_node_exporter", 2023-07-21T22:37:17.450 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.450 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.450 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.450 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T22:37:17.451 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.451 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.451 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.451 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:17.451 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.452 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.452 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.452 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_prometheus": { 2023-07-21T22:37:17.452 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_prometheus", 2023-07-21T22:37:17.453 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.453 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.453 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.453 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T22:37:17.453 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.453 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.454 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.454 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus container image", 2023-07-21T22:37:17.454 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.454 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.454 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.455 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.455 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_promtail": { 2023-07-21T22:37:17.455 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_promtail", 2023-07-21T22:37:17.455 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.455 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.456 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.456 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T22:37:17.456 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.456 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.456 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.457 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Promtail container image", 2023-07-21T22:37:17.457 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.457 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.457 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.457 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.457 INFO:teuthology.orchestra.run.smithi066.stdout: "container_image_snmp_gateway": { 2023-07-21T22:37:17.458 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_image_snmp_gateway", 2023-07-21T22:37:17.458 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.458 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.458 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.458 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T22:37:17.459 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.459 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.459 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.459 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "SNMP Gateway container image", 2023-07-21T22:37:17.459 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.460 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.460 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.460 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.460 INFO:teuthology.orchestra.run.smithi066.stdout: "container_init": { 2023-07-21T22:37:17.460 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "container_init", 2023-07-21T22:37:17.460 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.461 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.461 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.461 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.461 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.461 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.462 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.462 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T22:37:17.462 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.462 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.462 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.463 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.463 INFO:teuthology.orchestra.run.smithi066.stdout: "daemon_cache_timeout": { 2023-07-21T22:37:17.463 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "daemon_cache_timeout", 2023-07-21T22:37:17.463 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.463 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.464 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.464 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:17.464 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.464 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.464 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.464 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T22:37:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.465 INFO:teuthology.orchestra.run.smithi066.stdout: "default_cephadm_command_timeout": { 2023-07-21T22:37:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T22:37:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.466 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "900", 2023-07-21T22:37:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T22:37:17.467 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.468 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.468 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.468 INFO:teuthology.orchestra.run.smithi066.stdout: "default_registry": { 2023-07-21T22:37:17.468 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "default_registry", 2023-07-21T22:37:17.469 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.469 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.469 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.469 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "docker.io", 2023-07-21T22:37:17.469 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.469 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.470 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.470 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T22:37:17.470 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.470 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.470 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.471 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.471 INFO:teuthology.orchestra.run.smithi066.stdout: "device_cache_timeout": { 2023-07-21T22:37:17.471 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_cache_timeout", 2023-07-21T22:37:17.471 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.471 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.472 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.472 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1800", 2023-07-21T22:37:17.472 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.472 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.472 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.472 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache device inventory", 2023-07-21T22:37:17.473 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.473 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.473 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.473 INFO:teuthology.orchestra.run.smithi066.stdout: "device_enhanced_scan": { 2023-07-21T22:37:17.474 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_enhanced_scan", 2023-07-21T22:37:17.474 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.474 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.474 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.474 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.475 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.475 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.475 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.475 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T22:37:17.475 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.475 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.476 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.476 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.476 INFO:teuthology.orchestra.run.smithi066.stdout: "facts_cache_timeout": { 2023-07-21T22:37:17.476 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "facts_cache_timeout", 2023-07-21T22:37:17.476 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.477 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.477 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.477 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:17.477 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.477 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.478 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.478 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "seconds to cache host facts data", 2023-07-21T22:37:17.478 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.478 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.478 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.479 INFO:teuthology.orchestra.run.smithi066.stdout: "host_check_interval": { 2023-07-21T22:37:17.479 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "host_check_interval", 2023-07-21T22:37:17.479 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.479 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.479 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.480 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:17.480 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.480 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.480 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.480 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to perform a host check", 2023-07-21T22:37:17.481 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.481 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.481 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.481 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.481 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.482 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.482 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.482 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.482 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.482 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.483 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.483 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.483 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.483 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.483 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.483 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.484 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.484 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.484 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.484 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.484 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.485 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.485 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.485 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.485 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.485 INFO:teuthology.orchestra.run.smithi066.stdout: "log_refresh_metadata": { 2023-07-21T22:37:17.486 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_refresh_metadata", 2023-07-21T22:37:17.486 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.486 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.486 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.486 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.486 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.487 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.487 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.487 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T22:37:17.487 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.487 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.488 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.488 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.488 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.488 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.488 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.489 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.489 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.489 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.489 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.489 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.490 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.490 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T22:37:17.490 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.490 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.490 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.491 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.491 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.491 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.491 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.491 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.491 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.492 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.492 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.492 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.492 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.492 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.493 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.493 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.493 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.493 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.493 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.494 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.494 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.494 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.494 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.494 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.494 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.495 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.495 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.495 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.495 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.495 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.496 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.496 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.496 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.496 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.496 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.497 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.497 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.497 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.497 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.497 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T22:37:17.498 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T22:37:17.498 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.498 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.498 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.498 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.498 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.499 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.499 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.499 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T22:37:17.499 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.499 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.500 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.500 INFO:teuthology.orchestra.run.smithi066.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T22:37:17.500 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T22:37:17.500 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.501 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.501 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.501 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "*", 2023-07-21T22:37:17.501 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.501 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.502 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.502 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T22:37:17.502 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.502 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.502 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.502 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.503 INFO:teuthology.orchestra.run.smithi066.stdout: "max_count_per_host": { 2023-07-21T22:37:17.503 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_count_per_host", 2023-07-21T22:37:17.503 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.503 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.503 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.504 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.504 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.504 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.504 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.504 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of daemons per service per host", 2023-07-21T22:37:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: "max_osd_draining_count": { 2023-07-21T22:37:17.505 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_osd_draining_count", 2023-07-21T22:37:17.506 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.506 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.506 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.506 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.506 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.507 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.507 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.507 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T22:37:17.507 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.507 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.508 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.508 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.508 INFO:teuthology.orchestra.run.smithi066.stdout: "migration_current": { 2023-07-21T22:37:17.508 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "migration_current", 2023-07-21T22:37:17.508 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.509 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.509 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.509 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.509 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.509 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.510 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.510 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "internal - do not modify", 2023-07-21T22:37:17.510 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.510 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.510 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.510 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.511 INFO:teuthology.orchestra.run.smithi066.stdout: "mode": { 2023-07-21T22:37:17.511 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mode", 2023-07-21T22:37:17.511 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.511 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.511 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.512 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "root", 2023-07-21T22:37:17.512 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.512 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.512 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.512 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm-package", 2023-07-21T22:37:17.513 INFO:teuthology.orchestra.run.smithi066.stdout: "root" 2023-07-21T22:37:17.513 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.513 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T22:37:17.513 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.513 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.514 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.514 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.514 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_alerts_path": { 2023-07-21T22:37:17.514 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_alerts_path", 2023-07-21T22:37:17.514 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.515 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.515 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.515 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T22:37:17.515 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.515 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.515 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.516 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T22:37:17.516 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.516 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.516 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.516 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.517 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_web_password": { 2023-07-21T22:37:17.517 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_web_password", 2023-07-21T22:37:17.517 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.517 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.517 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.518 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.518 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.518 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.518 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.518 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus web password", 2023-07-21T22:37:17.518 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.519 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.519 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.519 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.519 INFO:teuthology.orchestra.run.smithi066.stdout: "prometheus_web_user": { 2023-07-21T22:37:17.519 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus_web_user", 2023-07-21T22:37:17.520 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.520 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.520 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.520 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.520 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.521 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.521 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.521 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Prometheus web user", 2023-07-21T22:37:17.521 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.521 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.521 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.522 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.522 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_insecure": { 2023-07-21T22:37:17.522 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_insecure", 2023-07-21T22:37:17.522 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.522 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.523 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.523 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.523 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.523 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.523 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.524 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T22:37:17.524 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.524 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.524 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.524 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.525 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_password": { 2023-07-21T22:37:17.525 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_password", 2023-07-21T22:37:17.525 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.525 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.525 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.525 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.526 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.526 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.526 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.526 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T22:37:17.526 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.527 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.527 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.527 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.527 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_url": { 2023-07-21T22:37:17.527 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_url", 2023-07-21T22:37:17.528 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.528 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.528 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.528 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.528 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.529 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.529 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.529 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T22:37:17.529 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.529 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.529 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.530 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.530 INFO:teuthology.orchestra.run.smithi066.stdout: "registry_username": { 2023-07-21T22:37:17.530 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "registry_username", 2023-07-21T22:37:17.530 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.530 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.531 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.531 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.531 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.531 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.531 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.532 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T22:37:17.532 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.532 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.532 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.532 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.533 INFO:teuthology.orchestra.run.smithi066.stdout: "secure_monitoring_stack": { 2023-07-21T22:37:17.533 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "secure_monitoring_stack", 2023-07-21T22:37:17.533 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.533 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.533 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.533 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.534 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.534 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.534 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.534 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T22:37:17.534 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.535 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.535 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.535 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.535 INFO:teuthology.orchestra.run.smithi066.stdout: "service_discovery_port": { 2023-07-21T22:37:17.535 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "service_discovery_port", 2023-07-21T22:37:17.536 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.536 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.536 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.536 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8765", 2023-07-21T22:37:17.536 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.537 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.537 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.537 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "cephadm service discovery port", 2023-07-21T22:37:17.537 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.537 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.537 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.538 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.538 INFO:teuthology.orchestra.run.smithi066.stdout: "ssh_config_file": { 2023-07-21T22:37:17.538 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssh_config_file", 2023-07-21T22:37:17.538 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.538 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.539 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.539 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.539 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.539 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.539 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.540 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T22:37:17.540 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.540 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.540 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.540 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.540 INFO:teuthology.orchestra.run.smithi066.stdout: "use_agent": { 2023-07-21T22:37:17.541 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_agent", 2023-07-21T22:37:17.541 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.541 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.541 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.541 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.542 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.542 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.542 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.542 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T22:37:17.542 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: "use_repo_digest": { 2023-07-21T22:37:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "use_repo_digest", 2023-07-21T22:37:17.543 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.544 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.544 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.544 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.544 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.544 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.545 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.545 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T22:37:17.545 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.545 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.545 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.546 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.546 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_failed_host_check": { 2023-07-21T22:37:17.546 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_failed_host_check", 2023-07-21T22:37:17.546 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.546 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.547 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.547 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.547 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.547 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.547 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.548 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T22:37:17.548 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.548 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.548 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.548 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.548 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_daemons": { 2023-07-21T22:37:17.549 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_daemons", 2023-07-21T22:37:17.549 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.549 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.549 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.549 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.550 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.550 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.550 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.550 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T22:37:17.550 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.551 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.551 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.551 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.551 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_on_stray_hosts": { 2023-07-21T22:37:17.551 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_on_stray_hosts", 2023-07-21T22:37:17.552 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.552 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.552 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.552 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.552 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.552 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.553 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.553 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T22:37:17.553 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.553 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.553 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.554 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.554 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.554 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.554 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.554 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crash", 2023-07-21T22:37:17.555 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.555 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.555 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.555 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.555 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.556 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.556 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.556 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.556 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.556 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.556 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.557 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.557 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.557 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.557 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.557 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.558 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.558 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.558 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.558 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.558 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.559 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.559 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.559 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.559 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.559 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.559 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.560 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.560 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.560 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.560 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.560 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.561 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.561 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.561 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.561 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.561 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.562 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.562 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.562 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.562 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.562 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.563 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.563 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.563 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.563 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.563 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.563 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.564 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.564 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.564 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.564 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.564 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.565 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.565 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.565 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.565 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.565 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.566 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.566 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.566 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.566 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.566 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.566 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.567 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.567 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.567 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.567 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.567 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.568 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.568 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.568 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.568 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.568 INFO:teuthology.orchestra.run.smithi066.stdout: "retain_interval": { 2023-07-21T22:37:17.569 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retain_interval", 2023-07-21T22:37:17.569 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.569 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.569 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.569 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "31536000", 2023-07-21T22:37:17.570 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.570 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.570 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.570 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T22:37:17.570 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.571 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.571 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.571 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.571 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_recent_interval": { 2023-07-21T22:37:17.571 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_recent_interval", 2023-07-21T22:37:17.571 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.572 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.572 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.572 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1209600", 2023-07-21T22:37:17.572 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.572 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.573 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.573 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T22:37:17.573 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.573 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.573 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.574 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.574 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.574 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.574 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dashboard", 2023-07-21T22:37:17.575 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.575 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.575 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.575 INFO:teuthology.orchestra.run.smithi066.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T22:37:17.575 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T22:37:17.575 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.576 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.576 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.576 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.576 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.576 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.577 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.577 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.577 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.577 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.577 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.578 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.578 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T22:37:17.578 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T22:37:17.578 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.578 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.579 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.579 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.579 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.579 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.579 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.579 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.580 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.580 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.580 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.580 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.580 INFO:teuthology.orchestra.run.smithi066.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T22:37:17.581 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T22:37:17.581 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.581 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.581 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.581 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.582 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.582 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.582 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.582 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.582 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.583 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.583 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.583 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.583 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_ENABLED": { 2023-07-21T22:37:17.583 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T22:37:17.583 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.584 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.584 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.584 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.584 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.584 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.585 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T22:37:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T22:37:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.586 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.587 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.588 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.588 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.588 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.588 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.588 INFO:teuthology.orchestra.run.smithi066.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T22:37:17.589 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T22:37:17.589 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.589 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.589 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.589 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.590 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.590 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.590 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.590 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.590 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.591 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.591 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.591 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.591 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T22:37:17.591 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T22:37:17.591 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.592 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.592 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.592 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.592 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.592 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.593 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.593 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.593 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.593 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.593 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.594 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.594 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T22:37:17.594 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T22:37:17.594 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.594 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.595 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.595 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.595 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.595 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.595 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.595 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.596 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.596 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.596 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.596 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.596 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T22:37:17.597 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T22:37:17.597 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.597 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.597 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.597 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.598 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.598 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.598 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.598 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.598 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.599 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.599 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.599 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T22:37:17.599 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T22:37:17.599 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.600 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.600 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.600 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.600 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.600 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.601 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.601 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.601 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.601 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.601 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.602 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.602 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T22:37:17.602 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T22:37:17.602 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.602 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.603 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.603 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.603 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.603 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.603 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.603 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.604 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.604 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.604 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.604 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T22:37:17.605 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T22:37:17.605 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.605 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.605 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.605 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.606 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.606 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.606 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.606 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.607 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.607 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.607 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.608 INFO:teuthology.orchestra.run.smithi066.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T22:37:17.608 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T22:37:17.608 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.608 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.609 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.609 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.609 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.609 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.609 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.610 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.610 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.610 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.610 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.611 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.611 INFO:teuthology.orchestra.run.smithi066.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T22:37:17.611 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T22:37:17.611 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.611 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.612 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.612 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.612 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.612 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.613 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.613 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.613 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.613 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.614 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.614 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.614 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T22:37:17.614 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T22:37:17.615 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.615 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.615 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.615 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.615 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.616 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.616 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.616 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.616 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.617 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.617 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.617 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.617 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T22:37:17.618 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T22:37:17.618 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.618 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.618 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.618 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.619 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.619 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.619 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.619 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.620 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.620 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.620 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.620 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.621 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_URL": { 2023-07-21T22:37:17.621 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_URL", 2023-07-21T22:37:17.621 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.621 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.621 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.622 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.622 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.622 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.622 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.622 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T22:37:17.623 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T22:37:17.624 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.624 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.624 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.624 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.624 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.625 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.625 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.625 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.625 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.625 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.626 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.626 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.626 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T22:37:17.626 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T22:37:17.626 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.627 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.627 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.627 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.627 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.627 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.627 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.628 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.628 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.628 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.628 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.628 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.629 INFO:teuthology.orchestra.run.smithi066.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T22:37:17.629 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T22:37:17.629 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.629 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.629 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.630 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.630 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.630 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.630 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.630 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.631 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.631 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.631 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.631 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.631 INFO:teuthology.orchestra.run.smithi066.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T22:37:17.632 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T22:37:17.632 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.632 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.632 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.632 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.632 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.633 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.633 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.633 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.633 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.633 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.634 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.634 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.634 INFO:teuthology.orchestra.run.smithi066.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T22:37:17.634 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T22:37:17.634 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.635 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.635 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.635 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.635 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.635 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.636 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.636 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.636 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.636 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.636 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.636 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.637 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T22:37:17.637 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T22:37:17.637 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.637 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.637 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.638 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.638 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.638 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.638 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.638 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.639 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.639 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.639 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.639 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.639 INFO:teuthology.orchestra.run.smithi066.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T22:37:17.639 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T22:37:17.640 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.640 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.640 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.640 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.640 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.641 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.641 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.641 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.641 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.641 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.642 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.642 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T22:37:17.642 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T22:37:17.642 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.643 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.643 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.643 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.643 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.643 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.643 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.644 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.644 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.644 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.644 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.644 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.645 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T22:37:17.645 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T22:37:17.645 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.645 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.645 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.646 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.646 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.646 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.646 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.646 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.647 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.647 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.647 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.647 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T22:37:17.647 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T22:37:17.648 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.648 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.648 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.648 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.648 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.649 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.649 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.649 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.649 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.649 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.650 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.650 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.650 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T22:37:17.650 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T22:37:17.650 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.650 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.651 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.651 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.651 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.651 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.651 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.652 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.652 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.652 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.652 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.653 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T22:37:17.653 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T22:37:17.653 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.653 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.653 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.654 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.654 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.654 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.654 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.654 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.654 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.655 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.655 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.655 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.655 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T22:37:17.655 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T22:37:17.656 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.656 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.656 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.656 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.656 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.657 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.657 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.657 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.657 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.657 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.658 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.658 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.658 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T22:37:17.658 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T22:37:17.658 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.658 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.659 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.659 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.659 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.659 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.659 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.660 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.660 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.660 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.660 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.660 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.661 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T22:37:17.661 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T22:37:17.661 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.661 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.661 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.662 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.662 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.662 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.662 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.662 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.663 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.663 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.663 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.663 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.663 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T22:37:17.663 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T22:37:17.664 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.664 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.664 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.664 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T22:37:17.664 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.665 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.665 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.665 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.665 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.665 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.666 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.666 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.666 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T22:37:17.666 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T22:37:17.666 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.666 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.667 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.667 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.667 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.667 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.667 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.668 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.668 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.668 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.668 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.668 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.669 INFO:teuthology.orchestra.run.smithi066.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T22:37:17.669 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T22:37:17.669 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.669 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.669 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.670 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8", 2023-07-21T22:37:17.670 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.670 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.670 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.670 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.671 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.671 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.671 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.671 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.671 INFO:teuthology.orchestra.run.smithi066.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T22:37:17.671 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T22:37:17.672 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.672 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.672 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.672 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "45", 2023-07-21T22:37:17.672 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T22:37:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T22:37:17.674 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.675 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.675 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.675 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.675 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.675 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.675 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.676 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.676 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.676 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.676 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.676 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.677 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T22:37:17.677 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T22:37:17.677 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.677 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.677 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.678 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "admin", 2023-07-21T22:37:17.678 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.678 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.678 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.678 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.679 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.679 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.679 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.679 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.679 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T22:37:17.680 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T22:37:17.680 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.680 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.680 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.680 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.680 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.681 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.681 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.681 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.681 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.681 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.682 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.682 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.682 INFO:teuthology.orchestra.run.smithi066.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T22:37:17.682 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T22:37:17.682 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.683 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.683 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.683 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.683 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.683 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.684 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.684 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.684 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.684 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.684 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.684 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.685 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T22:37:17.685 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T22:37:17.685 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.685 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.685 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.686 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:17.686 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.686 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.686 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.686 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.687 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T22:37:17.688 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T22:37:17.688 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.688 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.688 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.688 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.689 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.689 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.689 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.689 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.689 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.689 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.690 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.690 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.690 INFO:teuthology.orchestra.run.smithi066.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T22:37:17.690 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T22:37:17.690 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.691 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.691 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.691 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:17.691 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.691 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.692 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.692 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.692 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.692 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.692 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: "cross_origin_url": { 2023-07-21T22:37:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cross_origin_url", 2023-07-21T22:37:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.693 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.694 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.694 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.694 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.694 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.694 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.695 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.695 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.695 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.695 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.695 INFO:teuthology.orchestra.run.smithi066.stdout: "crt_file": { 2023-07-21T22:37:17.695 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "crt_file", 2023-07-21T22:37:17.696 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.696 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.696 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.696 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.696 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.697 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.697 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.697 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.697 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.697 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.698 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.698 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.698 INFO:teuthology.orchestra.run.smithi066.stdout: "debug": { 2023-07-21T22:37:17.698 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "debug", 2023-07-21T22:37:17.698 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.699 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.699 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.699 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.699 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.699 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.699 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.700 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Enable/disable debug options", 2023-07-21T22:37:17.700 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.700 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.700 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.700 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.701 INFO:teuthology.orchestra.run.smithi066.stdout: "jwt_token_ttl": { 2023-07-21T22:37:17.701 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "jwt_token_ttl", 2023-07-21T22:37:17.701 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.701 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.701 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.702 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "28800", 2023-07-21T22:37:17.702 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.702 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.702 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.702 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.703 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.703 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.703 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.703 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.703 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-07-21T22:37:17.704 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-07-21T22:37:17.704 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.704 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.704 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.704 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.704 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.705 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.705 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.705 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.705 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.705 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.706 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.706 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.706 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.706 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.706 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.707 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.707 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.707 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.707 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.707 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.708 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.708 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.708 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.708 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.708 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.708 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.709 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.709 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.709 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.709 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.709 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.710 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.710 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.710 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.710 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.710 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.711 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.711 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.711 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.711 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.711 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.712 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.712 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.712 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.712 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.712 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.713 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.713 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.713 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.713 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.713 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.713 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.714 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.714 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.714 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.717 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.717 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.717 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.717 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.718 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.718 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.718 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.718 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.718 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.719 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.719 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.719 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.719 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.719 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.720 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.720 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.720 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.720 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.720 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.721 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.721 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.721 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.721 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.721 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.722 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.722 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.722 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.722 INFO:teuthology.orchestra.run.smithi066.stdout: "motd": { 2023-07-21T22:37:17.722 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "motd", 2023-07-21T22:37:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.723 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.724 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.724 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "The message of the day", 2023-07-21T22:37:17.724 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.724 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.725 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.725 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.725 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect_resolve_ip_addr": { 2023-07-21T22:37:17.725 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T22:37:17.725 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.726 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.726 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.726 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.726 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.726 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.726 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.727 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.727 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.727 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.727 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.728 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.728 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:17.728 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:17.728 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.729 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.729 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.729 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-07-21T22:37:17.729 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.729 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.730 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.730 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.730 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.730 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.730 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.730 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.731 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:17.731 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:17.731 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.731 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.731 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.732 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8080", 2023-07-21T22:37:17.732 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.732 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.732 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.733 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.733 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.733 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.733 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.733 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.733 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-07-21T22:37:17.734 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-07-21T22:37:17.734 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.734 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.734 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.734 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.735 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.735 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.735 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.735 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.735 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.736 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.736 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.736 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.736 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl_server_port": { 2023-07-21T22:37:17.736 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl_server_port", 2023-07-21T22:37:17.737 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.737 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.737 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.737 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8443", 2023-07-21T22:37:17.737 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.738 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.738 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.738 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.738 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.739 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.739 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.739 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.739 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-07-21T22:37:17.739 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-07-21T22:37:17.740 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.740 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.740 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.740 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "redirect", 2023-07-21T22:37:17.740 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.741 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.741 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.741 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.741 INFO:teuthology.orchestra.run.smithi066.stdout: "redirect" 2023-07-21T22:37:17.741 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.742 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.742 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.743 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.743 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.744 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.744 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-07-21T22:37:17.745 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-07-21T22:37:17.745 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.745 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.746 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.746 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-07-21T22:37:17.747 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-07-21T22:37:17.747 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-07-21T22:37:17.748 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.748 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.748 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.749 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.749 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.749 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.750 INFO:teuthology.orchestra.run.smithi066.stdout: "url_prefix": { 2023-07-21T22:37:17.750 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url_prefix", 2023-07-21T22:37:17.750 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.751 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.751 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.752 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.752 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.752 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.752 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.753 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.753 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.753 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.753 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.753 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.754 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.754 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.754 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.754 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-07-21T22:37:17.754 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.754 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.755 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.755 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_monitoring": { 2023-07-21T22:37:17.755 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_monitoring", 2023-07-21T22:37:17.755 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.756 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.756 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.756 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.756 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.756 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.757 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.757 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "monitor device health metrics", 2023-07-21T22:37:17.757 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.757 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.758 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.758 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.758 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.758 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.758 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.759 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.759 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.759 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.759 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.759 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.759 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.760 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.760 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.760 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.760 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.761 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.761 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.761 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.761 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.761 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.761 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.762 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.762 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.762 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.762 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.762 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.763 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.763 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.763 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.763 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.763 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.764 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.764 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.764 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.764 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.764 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.765 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.765 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.765 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.765 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.765 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.766 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.767 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.767 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.767 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.767 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.768 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.768 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.768 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.768 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.768 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.769 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.769 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.769 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.769 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.769 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.770 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.770 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.770 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.770 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.770 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.771 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.771 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.771 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.771 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.771 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.772 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.772 INFO:teuthology.orchestra.run.smithi066.stdout: "mark_out_threshold": { 2023-07-21T22:37:17.772 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mark_out_threshold", 2023-07-21T22:37:17.772 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.772 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.773 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.773 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "2419200", 2023-07-21T22:37:17.773 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.773 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.773 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.774 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T22:37:17.774 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.774 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.774 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.774 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.774 INFO:teuthology.orchestra.run.smithi066.stdout: "pool_name": { 2023-07-21T22:37:17.775 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pool_name", 2023-07-21T22:37:17.775 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.775 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.775 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.775 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "device_health_metrics", 2023-07-21T22:37:17.776 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.776 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.776 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.776 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T22:37:17.776 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.777 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.777 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.777 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.777 INFO:teuthology.orchestra.run.smithi066.stdout: "retention_period": { 2023-07-21T22:37:17.777 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "retention_period", 2023-07-21T22:37:17.778 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.778 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.778 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.778 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15552000", 2023-07-21T22:37:17.778 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.779 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.779 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.779 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long to retain device health metrics", 2023-07-21T22:37:17.779 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.779 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.780 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.780 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.780 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_frequency": { 2023-07-21T22:37:17.780 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_frequency", 2023-07-21T22:37:17.780 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.781 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.781 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.781 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-07-21T22:37:17.781 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.781 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.781 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.782 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T22:37:17.782 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.782 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.782 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.783 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.783 INFO:teuthology.orchestra.run.smithi066.stdout: "self_heal": { 2023-07-21T22:37:17.783 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "self_heal", 2023-07-21T22:37:17.783 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.783 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.784 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.784 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:17.784 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.784 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.784 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.785 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T22:37:17.785 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.785 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.785 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.785 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.786 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:17.786 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:17.786 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.786 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.786 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.787 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:17.787 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.787 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.787 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.787 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T22:37:17.787 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.788 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.788 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.788 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.788 INFO:teuthology.orchestra.run.smithi066.stdout: "warn_threshold": { 2023-07-21T22:37:17.788 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "warn_threshold", 2023-07-21T22:37:17.789 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.789 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.789 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.789 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7257600", 2023-07-21T22:37:17.790 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.790 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.790 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.790 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T22:37:17.790 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.791 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.791 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.791 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.791 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.791 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.792 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.792 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "diskprediction_local", 2023-07-21T22:37:17.792 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.792 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.792 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.793 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.793 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.793 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.793 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.793 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.794 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.794 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.794 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.794 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.794 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.794 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.795 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.795 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.795 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.795 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.795 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.796 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.796 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.796 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.796 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.796 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.797 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.797 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.797 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.797 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.797 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.798 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.798 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.798 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.798 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.798 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.799 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.799 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.799 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.799 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.799 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.799 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.800 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.800 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.800 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.800 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.801 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.801 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.801 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.801 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.801 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.802 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.802 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.802 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.802 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.802 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.803 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.803 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.803 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.803 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.803 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.804 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.804 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.804 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.804 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.804 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.805 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.805 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.805 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.805 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.805 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.806 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.806 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.806 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.806 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.806 INFO:teuthology.orchestra.run.smithi066.stdout: "predict_interval": { 2023-07-21T22:37:17.806 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predict_interval", 2023-07-21T22:37:17.807 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.807 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.807 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.807 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "86400", 2023-07-21T22:37:17.807 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.808 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.808 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.808 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.808 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.808 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.809 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.809 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.809 INFO:teuthology.orchestra.run.smithi066.stdout: "predictor_model": { 2023-07-21T22:37:17.809 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "predictor_model", 2023-07-21T22:37:17.809 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.810 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.810 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.810 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "prophetstor", 2023-07-21T22:37:17.810 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.810 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.811 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.811 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.811 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.811 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.811 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.812 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.812 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:17.812 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:17.812 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.812 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.812 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.813 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "600", 2023-07-21T22:37:17.813 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.813 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.813 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.814 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.814 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.814 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.814 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.814 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.814 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.815 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.815 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.815 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "influx", 2023-07-21T22:37:17.815 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": false, 2023-07-21T22:37:17.815 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "influxdb python module not found", 2023-07-21T22:37:17.816 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.816 INFO:teuthology.orchestra.run.smithi066.stdout: "batch_size": { 2023-07-21T22:37:17.816 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "batch_size", 2023-07-21T22:37:17.816 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.816 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.817 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.817 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5000", 2023-07-21T22:37:17.817 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.817 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.817 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.818 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T22:37:17.818 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.818 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.818 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.818 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.819 INFO:teuthology.orchestra.run.smithi066.stdout: "database": { 2023-07-21T22:37:17.819 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "database", 2023-07-21T22:37:17.819 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.819 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.819 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.819 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "ceph", 2023-07-21T22:37:17.820 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.820 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.820 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.820 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T22:37:17.821 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.821 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.821 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.821 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.821 INFO:teuthology.orchestra.run.smithi066.stdout: "hostname": { 2023-07-21T22:37:17.822 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "hostname", 2023-07-21T22:37:17.822 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.822 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.822 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.822 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.822 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.823 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.823 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.823 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server hostname", 2023-07-21T22:37:17.823 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.823 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.824 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.824 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.824 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:17.824 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:17.824 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:17.825 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.825 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.825 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "30", 2023-07-21T22:37:17.825 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "5", 2023-07-21T22:37:17.825 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.826 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.826 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T22:37:17.826 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.826 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.826 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.826 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.827 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.827 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.827 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.827 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.827 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.828 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.828 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.828 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.828 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.828 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.829 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.829 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.829 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.829 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.829 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.830 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.830 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.830 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.830 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.830 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.831 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.831 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.831 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.831 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.831 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.832 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.832 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.832 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.832 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.832 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.833 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.833 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.833 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.833 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.833 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.834 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.834 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.834 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.834 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.834 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.834 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.835 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.835 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.835 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.835 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.835 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.836 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.836 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.836 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.836 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.836 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.837 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.837 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.837 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.837 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.837 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.838 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.838 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.838 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.838 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.838 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.838 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.839 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.839 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.839 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.839 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.839 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.840 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.840 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.840 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.840 INFO:teuthology.orchestra.run.smithi066.stdout: "password": { 2023-07-21T22:37:17.840 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "password", 2023-07-21T22:37:17.841 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.841 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.841 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.841 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.842 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.842 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.842 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.842 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "password of InfluxDB server user", 2023-07-21T22:37:17.843 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.843 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.843 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.843 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.843 INFO:teuthology.orchestra.run.smithi066.stdout: "port": { 2023-07-21T22:37:17.844 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "port", 2023-07-21T22:37:17.844 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.844 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.844 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.844 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "8086", 2023-07-21T22:37:17.845 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.845 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.845 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.845 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "InfluxDB server port", 2023-07-21T22:37:17.846 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.846 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.846 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.846 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.846 INFO:teuthology.orchestra.run.smithi066.stdout: "ssl": { 2023-07-21T22:37:17.847 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ssl", 2023-07-21T22:37:17.847 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.847 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.847 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.847 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "false", 2023-07-21T22:37:17.848 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.848 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.848 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.848 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:37:17.849 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.849 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.849 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.849 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.849 INFO:teuthology.orchestra.run.smithi066.stdout: "threads": { 2023-07-21T22:37:17.849 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threads", 2023-07-21T22:37:17.850 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.850 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.850 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.850 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:17.850 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:17.851 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "32", 2023-07-21T22:37:17.851 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.851 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T22:37:17.851 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.851 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.852 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.852 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.852 INFO:teuthology.orchestra.run.smithi066.stdout: "username": { 2023-07-21T22:37:17.852 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "username", 2023-07-21T22:37:17.852 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.853 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.853 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.853 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.853 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.853 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.854 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.854 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "username of InfluxDB server user", 2023-07-21T22:37:17.854 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.854 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.854 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.855 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.855 INFO:teuthology.orchestra.run.smithi066.stdout: "verify_ssl": { 2023-07-21T22:37:17.855 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "verify_ssl", 2023-07-21T22:37:17.855 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.855 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.856 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.856 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "true", 2023-07-21T22:37:17.856 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.856 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.856 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.857 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T22:37:17.857 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.857 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.857 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.857 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.858 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.858 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.858 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.858 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "insights", 2023-07-21T22:37:17.858 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.859 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.859 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.859 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.859 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.859 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.860 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.860 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.860 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.860 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.861 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.861 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.862 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.862 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.863 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.863 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.864 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.864 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.864 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.865 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.865 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.865 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.865 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.866 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.866 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.866 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.866 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.866 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.867 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.867 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.867 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.867 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.868 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.868 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.868 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.868 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.869 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.869 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.869 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.869 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.869 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.870 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.870 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.870 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.870 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.870 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.871 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.871 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.871 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.871 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.872 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.872 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.872 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.872 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.872 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.873 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.873 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.873 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.873 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.874 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.874 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.874 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.874 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.874 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.875 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.875 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.875 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.875 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.875 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.876 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.876 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.876 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.876 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.876 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.877 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.877 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.877 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "iostat", 2023-07-21T22:37:17.877 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.877 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.878 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.878 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.878 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.878 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.878 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.879 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.879 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.879 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.879 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.879 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.879 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.880 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.880 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.880 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.880 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.881 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.881 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.881 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.881 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.881 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.881 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.882 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.882 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.882 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.882 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.882 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.883 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.883 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.883 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.883 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.883 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.884 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.884 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.884 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.884 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.884 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.885 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.885 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.885 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.885 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.885 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.886 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.886 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.886 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.886 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.886 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.886 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.887 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.887 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.887 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.887 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.887 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.888 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.888 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.888 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.888 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.889 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.889 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.889 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.890 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.890 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.890 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.890 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.891 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.891 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.891 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.891 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.891 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.891 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.892 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.892 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.892 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.892 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.893 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.893 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "k8sevents", 2023-07-21T22:37:17.893 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.893 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.893 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.893 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_event_retention_days": { 2023-07-21T22:37:17.894 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "ceph_event_retention_days", 2023-07-21T22:37:17.894 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.894 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.894 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.894 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "7", 2023-07-21T22:37:17.895 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.895 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.895 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.895 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T22:37:17.895 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.896 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.896 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.896 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.896 INFO:teuthology.orchestra.run.smithi066.stdout: "config_check_secs": { 2023-07-21T22:37:17.896 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "config_check_secs", 2023-07-21T22:37:17.897 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.897 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.897 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "10", 2023-07-21T22:37:17.898 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.899 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.899 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T22:37:17.899 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.900 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.900 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.900 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.900 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.901 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.901 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.901 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.901 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.902 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.902 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.902 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.902 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.902 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.903 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.903 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.903 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.903 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.904 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.904 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.904 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.904 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.905 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.905 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.905 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.906 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.906 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.907 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.907 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.907 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.907 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.908 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.908 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.908 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.908 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.909 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.909 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.909 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.909 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.909 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.910 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.910 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.911 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.911 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.911 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.912 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.912 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.913 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.913 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.914 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.914 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.914 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.914 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.915 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.915 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.916 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.916 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.916 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.917 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.917 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.917 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.917 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.918 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.918 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.918 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.918 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.918 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.919 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.919 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.919 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.919 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.919 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.920 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.920 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.920 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.920 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.920 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.921 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "localpool", 2023-07-21T22:37:17.921 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.921 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.921 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.921 INFO:teuthology.orchestra.run.smithi066.stdout: "failure_domain": { 2023-07-21T22:37:17.922 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "failure_domain", 2023-07-21T22:37:17.922 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.922 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.922 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.922 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "host", 2023-07-21T22:37:17.923 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.923 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.923 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.923 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "failure domain for any created local pool", 2023-07-21T22:37:17.923 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T22:37:17.923 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.924 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.924 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.924 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.924 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.925 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.925 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.925 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.925 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.925 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.926 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.926 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.926 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.926 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.927 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.927 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.927 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.928 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.928 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.928 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.928 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.928 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.929 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.929 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.929 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.929 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.929 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.930 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.930 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.931 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.931 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.931 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.931 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.931 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.932 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.932 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.932 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.932 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.933 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.933 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.933 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.933 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.933 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.934 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.934 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.934 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.934 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.934 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.935 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.935 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.935 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.935 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.935 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.936 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.936 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.936 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.937 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.937 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.937 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.937 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.938 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.938 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.938 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.938 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.938 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.938 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.939 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.939 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.939 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.939 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.939 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.940 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.940 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.940 INFO:teuthology.orchestra.run.smithi066.stdout: "min_size": { 2023-07-21T22:37:17.940 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "min_size", 2023-07-21T22:37:17.940 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.941 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.941 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.941 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.941 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.941 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.942 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.942 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default min_size for any created local pool", 2023-07-21T22:37:17.942 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T22:37:17.942 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.943 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.943 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.943 INFO:teuthology.orchestra.run.smithi066.stdout: "num_rep": { 2023-07-21T22:37:17.943 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "num_rep", 2023-07-21T22:37:17.943 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.943 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.944 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.944 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3", 2023-07-21T22:37:17.945 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.945 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.945 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.945 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default replica count for any created local pool", 2023-07-21T22:37:17.945 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.946 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.946 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.946 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.946 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_num": { 2023-07-21T22:37:17.946 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_num", 2023-07-21T22:37:17.947 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:17.947 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.947 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.947 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "128", 2023-07-21T22:37:17.947 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.948 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.948 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.948 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T22:37:17.949 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.949 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.950 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.950 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.950 INFO:teuthology.orchestra.run.smithi066.stdout: "prefix": { 2023-07-21T22:37:17.951 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prefix", 2023-07-21T22:37:17.951 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.952 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.952 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.953 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.953 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.954 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.954 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.955 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "name prefix for any created local pool", 2023-07-21T22:37:17.955 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.955 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.956 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.956 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.956 INFO:teuthology.orchestra.run.smithi066.stdout: "subtree": { 2023-07-21T22:37:17.957 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "subtree", 2023-07-21T22:37:17.957 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.958 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.958 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.958 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "rack", 2023-07-21T22:37:17.958 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.959 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.959 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.959 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T22:37:17.959 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T22:37:17.959 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.960 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.960 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.960 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.960 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.961 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.961 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mds_autoscaler", 2023-07-21T22:37:17.961 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.961 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.961 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.961 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.962 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.962 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.962 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.962 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.962 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.963 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.963 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.963 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.963 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.963 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.964 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.964 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.964 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.964 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.964 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.965 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.965 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.965 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.965 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.965 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.966 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.966 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.966 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.966 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.966 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.967 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.967 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.967 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.967 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.968 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.968 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.968 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.968 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.969 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.969 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.969 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.969 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.969 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.970 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.970 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.970 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.970 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.970 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.971 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.971 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.971 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.971 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.971 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.972 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.972 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.972 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.972 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.972 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.973 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.973 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.973 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.974 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.974 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.974 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.974 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.974 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.975 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.975 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.975 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.975 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.975 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.976 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.976 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.976 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.976 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.976 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.977 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.977 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirroring", 2023-07-21T22:37:17.977 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.977 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.977 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.978 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.978 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.978 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.978 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.978 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.979 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.979 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.979 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.979 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.979 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.980 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.980 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.980 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.980 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.980 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.981 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.981 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.981 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.981 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.981 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.982 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.982 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.982 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.982 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.982 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.983 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.983 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.983 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.983 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.983 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.984 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.984 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.984 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.984 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.984 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.984 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:17.985 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:17.985 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.985 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.985 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.985 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:17.986 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.986 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.986 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.986 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.986 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.987 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.987 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.987 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.987 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.987 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.988 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.988 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.988 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.988 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.988 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.989 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:17.989 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:17.989 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.989 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.989 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.990 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.990 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.990 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.990 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.990 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.990 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.991 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.991 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.991 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:17.991 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.992 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:17.992 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "nfs", 2023-07-21T22:37:17.992 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:17.992 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:17.992 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:17.993 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:17.993 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:17.993 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:17.993 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.993 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.994 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:17.994 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.994 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.994 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:17.994 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:17.995 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:17.995 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:17.995 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:17.995 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:17.995 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:17.996 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:17.996 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.996 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.996 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.996 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.997 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.997 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:17.997 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:17.997 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:17.997 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:17.998 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:17.998 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:17.998 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:17.998 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:17.998 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:17.998 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:17.999 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:17.999 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:17.999 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:17.999 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:17.999 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.000 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.000 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.000 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.000 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.000 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.001 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.001 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.001 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.001 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.001 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.002 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.002 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.002 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.002 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.002 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.003 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.003 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.003 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.003 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.003 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.003 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.004 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.004 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.004 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.004 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.004 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.005 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.005 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.005 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.005 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.005 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.006 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.006 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.006 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.006 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.007 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.007 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-07-21T22:37:18.007 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.007 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.007 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stdout: "fail_fs": { 2023-07-21T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "fail_fs", 2023-07-21T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.008 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.009 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.009 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.009 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.009 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.009 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T22:37:18.010 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.010 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.010 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.010 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.010 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.011 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.011 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.011 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.011 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.011 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.011 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.012 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.012 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.012 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.012 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.012 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.013 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.013 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.013 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.013 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.013 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.014 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.014 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.014 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.014 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.014 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.015 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.015 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.015 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.015 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.015 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.016 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.016 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.016 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.016 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.016 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.016 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.017 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.017 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.017 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.017 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.017 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.018 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.019 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.019 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.019 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.019 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.019 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.020 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.020 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.020 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.020 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.020 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.021 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.021 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.021 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.021 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.021 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.021 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.022 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.022 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.022 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.022 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.022 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.023 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.023 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.023 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.023 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.023 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.024 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.024 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator": { 2023-07-21T22:37:18.024 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "orchestrator", 2023-07-21T22:37:18.024 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.024 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.025 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.025 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.025 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.025 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.025 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.026 INFO:teuthology.orchestra.run.smithi066.stdout: "cephadm", 2023-07-21T22:37:18.026 INFO:teuthology.orchestra.run.smithi066.stdout: "rook", 2023-07-21T22:37:18.026 INFO:teuthology.orchestra.run.smithi066.stdout: "test_orchestrator" 2023-07-21T22:37:18.026 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.026 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Orchestrator backend", 2023-07-21T22:37:18.027 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.027 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.027 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.027 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.028 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.028 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.028 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_perf_query", 2023-07-21T22:37:18.028 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.028 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.029 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.029 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.029 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.029 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.029 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.030 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.030 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.030 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.030 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.030 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.031 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.031 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.031 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.031 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.031 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.032 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.032 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.032 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.032 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.032 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.033 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.033 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.033 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.033 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.033 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.034 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.034 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.034 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.034 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.034 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.035 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.035 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.035 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.035 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.035 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.036 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.036 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.036 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.036 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.036 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.037 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.037 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.037 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.037 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.037 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.038 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.038 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.038 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.038 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.038 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.038 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.039 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.039 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.039 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.039 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.039 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.040 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.040 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.040 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.040 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.040 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.041 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.041 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.041 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.041 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.041 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.042 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.042 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.042 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.042 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.042 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.042 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.043 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.043 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.043 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "osd_support", 2023-07-21T22:37:18.043 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.043 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.044 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.044 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.044 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.044 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.044 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.045 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.045 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.045 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.045 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.045 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.046 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.046 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.046 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.046 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.046 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.046 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.047 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.047 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.047 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.047 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.047 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.048 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.048 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.048 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.048 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.048 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.049 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.049 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.049 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.049 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.049 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.050 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.050 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.050 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.050 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.050 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.051 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.051 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.051 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.051 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.051 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.051 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.052 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.052 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.052 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.052 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.052 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.053 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.053 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.053 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.053 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.053 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.054 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.054 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.054 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.054 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.054 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.055 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.055 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.055 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.055 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.055 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.056 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.056 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.056 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.056 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.056 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.057 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.057 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.057 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.057 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.057 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.058 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.058 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "pg_autoscaler", 2023-07-21T22:37:18.058 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.058 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.058 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.059 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.059 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.059 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.059 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.059 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.060 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.060 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.060 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.060 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.060 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.061 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.061 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.061 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.061 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.061 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.062 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.062 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.062 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.062 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.062 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.062 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.063 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.063 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.063 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.063 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.063 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.064 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.064 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.064 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.064 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.064 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.065 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.065 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.065 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.065 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.065 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.066 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.066 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.066 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.066 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.066 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.067 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.067 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.067 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.067 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.067 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.067 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.068 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.068 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.068 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.068 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.068 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.069 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.069 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.069 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.069 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.069 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.070 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.070 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.070 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.070 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.070 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.071 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.071 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.071 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.071 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.071 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.072 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.072 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.072 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.072 INFO:teuthology.orchestra.run.smithi066.stdout: "noautoscale": { 2023-07-21T22:37:18.072 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "noautoscale", 2023-07-21T22:37:18.073 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.073 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.073 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.073 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.073 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.074 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.074 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.074 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "global autoscale flag", 2023-07-21T22:37:18.074 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T22:37:18.074 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.074 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.075 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.075 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:18.075 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:18.075 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:18.075 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.076 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.076 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:18.076 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.076 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.076 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.077 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "threshold": { 2023-07-21T22:37:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "threshold", 2023-07-21T22:37:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.078 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.079 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "3.0", 2023-07-21T22:37:18.079 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1.0", 2023-07-21T22:37:18.079 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.079 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.079 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "scaling threshold", 2023-07-21T22:37:18.079 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T22:37:18.080 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.080 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.080 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.080 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.081 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.081 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "progress", 2023-07-21T22:37:18.081 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.081 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.081 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.082 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_pg_recovery_event": { 2023-07-21T22:37:18.082 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_pg_recovery_event", 2023-07-21T22:37:18.082 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.082 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.082 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.083 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.083 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.083 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.083 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.083 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T22:37:18.084 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.084 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.084 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.084 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.084 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-07-21T22:37:18.085 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-07-21T22:37:18.085 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.085 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.085 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.085 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.086 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.086 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.086 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.086 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.086 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.087 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.087 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.087 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.087 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.087 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.087 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.088 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.088 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.088 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.088 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.088 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.089 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.089 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.089 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.089 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.089 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.090 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.090 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.090 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.090 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.090 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.091 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.091 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.091 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.091 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.091 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.092 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.092 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.092 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.092 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.093 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.093 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.093 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.093 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.093 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.094 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.094 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.094 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.094 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.094 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.095 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.095 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.095 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.095 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.095 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.096 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.096 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.096 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.096 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.096 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.097 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.097 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.097 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.097 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.097 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.098 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.098 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.098 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.098 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.098 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.098 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.099 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.099 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.099 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.099 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.099 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.100 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.100 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.100 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.100 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.100 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.101 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.101 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.101 INFO:teuthology.orchestra.run.smithi066.stdout: "max_completed_events": { 2023-07-21T22:37:18.101 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_completed_events", 2023-07-21T22:37:18.101 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.102 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.102 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.102 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "50", 2023-07-21T22:37:18.102 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.102 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.103 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.103 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "number of past completed events to remember", 2023-07-21T22:37:18.103 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.103 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.103 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.104 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.104 INFO:teuthology.orchestra.run.smithi066.stdout: "sleep_interval": { 2023-07-21T22:37:18.104 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "sleep_interval", 2023-07-21T22:37:18.104 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:18.104 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.105 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.105 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "5", 2023-07-21T22:37:18.105 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.105 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.105 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.105 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "how long the module is going to sleep", 2023-07-21T22:37:18.106 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.106 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.106 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.106 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.107 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.107 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.107 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "prometheus", 2023-07-21T22:37:18.107 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.107 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.108 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.108 INFO:teuthology.orchestra.run.smithi066.stdout: "cache": { 2023-07-21T22:37:18.108 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "cache", 2023-07-21T22:37:18.108 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.108 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.109 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.109 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.109 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.109 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.109 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.110 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.110 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.110 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.110 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.110 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.110 INFO:teuthology.orchestra.run.smithi066.stdout: "exclude_perf_counters": { 2023-07-21T22:37:18.111 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "exclude_perf_counters", 2023-07-21T22:37:18.111 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.111 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.111 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.111 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.112 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.112 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.112 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.112 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T22:37:18.112 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T22:37:18.113 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.113 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.113 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.113 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.113 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.114 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.114 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.114 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.114 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.114 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.115 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.115 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.115 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.115 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.115 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.116 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.116 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.116 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.116 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.116 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.117 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.118 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.119 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.119 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.119 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.119 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.119 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.120 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.120 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.120 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.120 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.120 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.121 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.121 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.121 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.121 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.121 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.121 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.122 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.122 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.122 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.122 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.122 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.123 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.123 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.123 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.123 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.123 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.124 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.124 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.124 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.124 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.124 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.125 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.125 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.125 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.125 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.125 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.126 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.126 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.126 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.126 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.126 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.126 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.127 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.127 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools": { 2023-07-21T22:37:18.127 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools", 2023-07-21T22:37:18.127 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.127 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.128 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.128 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.128 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.128 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.128 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.129 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.129 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.129 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.129 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.129 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.130 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T22:37:18.130 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T22:37:18.130 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.130 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.130 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.131 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300", 2023-07-21T22:37:18.131 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.131 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.131 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.131 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.132 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.132 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.132 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.132 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.132 INFO:teuthology.orchestra.run.smithi066.stdout: "scrape_interval": { 2023-07-21T22:37:18.133 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "scrape_interval", 2023-07-21T22:37:18.133 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:18.133 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.133 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.133 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15.0", 2023-07-21T22:37:18.134 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.134 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.134 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.134 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.134 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.135 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.135 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.135 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.135 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:18.135 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:18.136 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.136 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.136 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.136 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "::", 2023-07-21T22:37:18.136 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.137 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.137 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.137 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T22:37:18.137 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.137 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.138 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.138 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.138 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:18.138 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:18.138 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.138 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.139 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.139 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "9283", 2023-07-21T22:37:18.139 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.139 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.139 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.140 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T22:37:18.140 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.140 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.140 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.140 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.141 INFO:teuthology.orchestra.run.smithi066.stdout: "stale_cache_strategy": { 2023-07-21T22:37:18.141 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stale_cache_strategy", 2023-07-21T22:37:18.141 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.141 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.141 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.142 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "log", 2023-07-21T22:37:18.142 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.142 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.142 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.142 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.143 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.143 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.143 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.143 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.143 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_behaviour": { 2023-07-21T22:37:18.144 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_behaviour", 2023-07-21T22:37:18.144 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.144 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.144 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.144 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "default", 2023-07-21T22:37:18.145 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.145 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.145 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.145 INFO:teuthology.orchestra.run.smithi066.stdout: "default", 2023-07-21T22:37:18.145 INFO:teuthology.orchestra.run.smithi066.stdout: "error" 2023-07-21T22:37:18.145 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.146 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.146 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.146 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.146 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.146 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.147 INFO:teuthology.orchestra.run.smithi066.stdout: "standby_error_status_code": { 2023-07-21T22:37:18.147 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "standby_error_status_code", 2023-07-21T22:37:18.147 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.147 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.147 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.148 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "500", 2023-07-21T22:37:18.148 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "400", 2023-07-21T22:37:18.148 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "599", 2023-07-21T22:37:18.148 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.148 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.149 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.149 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.149 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.149 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.149 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.149 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.150 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.150 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-07-21T22:37:18.150 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.150 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.150 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.151 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.151 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.151 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.151 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.151 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.152 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.152 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.152 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.152 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.152 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.153 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.153 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.153 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.153 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.153 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.154 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.154 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.154 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.154 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.154 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.154 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.155 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.155 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.155 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.155 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.155 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.156 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.156 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.156 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.156 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.156 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.157 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.157 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.157 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.157 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.157 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.157 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.158 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.158 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.158 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.158 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.158 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.159 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.159 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.159 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.159 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.159 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.160 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.160 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.160 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.160 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.160 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.161 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.161 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.161 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.161 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.161 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.161 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.162 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.162 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.162 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.162 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.162 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.163 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.163 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.163 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.163 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.163 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.164 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.164 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.164 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_snap_create": { 2023-07-21T22:37:18.164 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_snap_create", 2023-07-21T22:37:18.164 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.165 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.165 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.165 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10", 2023-07-21T22:37:18.165 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.165 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.165 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.166 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.166 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.166 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.166 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.166 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.167 INFO:teuthology.orchestra.run.smithi066.stdout: "mirror_snapshot_schedule": { 2023-07-21T22:37:18.167 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T22:37:18.167 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.167 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.167 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.168 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.168 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.168 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.168 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.168 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.169 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.169 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.169 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.169 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.169 INFO:teuthology.orchestra.run.smithi066.stdout: "trash_purge_schedule": { 2023-07-21T22:37:18.169 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "trash_purge_schedule", 2023-07-21T22:37:18.170 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.170 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.170 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.170 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.170 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.171 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.171 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.171 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.171 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.171 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.172 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.172 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.172 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.172 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.173 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "restful", 2023-07-21T22:37:18.173 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.173 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.173 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.173 INFO:teuthology.orchestra.run.smithi066.stdout: "enable_auth": { 2023-07-21T22:37:18.173 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enable_auth", 2023-07-21T22:37:18.174 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.174 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.174 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.174 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.174 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.175 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.175 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.175 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.175 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.175 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.176 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.176 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.176 INFO:teuthology.orchestra.run.smithi066.stdout: "key_file": { 2023-07-21T22:37:18.176 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "key_file", 2023-07-21T22:37:18.176 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.177 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.177 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.177 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.177 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.177 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.178 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.178 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.178 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.178 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.178 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.179 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.179 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.179 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.179 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.179 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.179 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.180 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.180 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.180 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.180 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.180 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.181 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.182 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.183 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.184 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.184 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.184 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.184 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.184 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.184 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.185 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.185 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.185 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.185 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.185 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.186 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.186 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.186 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.186 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.186 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.187 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.187 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.187 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.187 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.187 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.188 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.188 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.188 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.188 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.188 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.189 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.189 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.189 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.189 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.189 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.190 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.190 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.190 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.190 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.190 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.191 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.191 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.191 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.191 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.191 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.192 INFO:teuthology.orchestra.run.smithi066.stdout: "server_addr": { 2023-07-21T22:37:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_addr", 2023-07-21T22:37:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.193 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.194 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: "server_port": { 2023-07-21T22:37:18.195 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "server_port", 2023-07-21T22:37:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.196 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.197 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.198 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.199 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rgw", 2023-07-21T22:37:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.200 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.201 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.202 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.202 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.202 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.202 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.203 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.203 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.203 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.204 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.204 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.204 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.205 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.205 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.205 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.205 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.206 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.206 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.206 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.207 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.207 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.207 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.208 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.208 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.208 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.209 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.209 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.209 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.209 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.210 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.210 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.210 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.211 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.211 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.211 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.211 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.212 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.212 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.212 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.213 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.213 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.213 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.214 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.214 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.214 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.214 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.215 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.215 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.215 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.216 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.216 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.216 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.216 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.216 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.217 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.217 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.217 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.217 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.217 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.218 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.218 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.218 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.218 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.218 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.219 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.219 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.219 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.219 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.219 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.220 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.220 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.220 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.220 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.220 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rook", 2023-07-21T22:37:18.220 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.221 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.221 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.221 INFO:teuthology.orchestra.run.smithi066.stdout: "drive_group_interval": { 2023-07-21T22:37:18.221 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "drive_group_interval", 2023-07-21T22:37:18.221 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:18.222 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.222 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.222 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "300.0", 2023-07-21T22:37:18.222 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.222 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.223 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.223 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T22:37:18.223 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.223 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.223 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.224 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.224 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.224 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.224 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.224 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.225 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.225 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.225 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.225 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.225 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.225 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.226 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.226 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.226 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.226 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.226 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.227 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.227 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.227 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.227 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.227 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.228 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.228 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.228 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.228 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.228 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.229 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.229 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.229 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.229 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.229 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.230 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.230 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.230 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.230 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.230 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.230 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.231 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.231 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.231 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.231 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.231 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.232 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.232 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.232 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.232 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.232 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.233 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.233 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.233 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.233 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.233 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.234 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.234 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.234 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.234 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.234 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.234 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.235 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.235 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.235 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.235 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.235 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.236 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.236 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.236 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.236 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.236 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.237 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.237 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.237 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.237 INFO:teuthology.orchestra.run.smithi066.stdout: "storage_class": { 2023-07-21T22:37:18.237 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "storage_class", 2023-07-21T22:37:18.238 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.238 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.238 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.238 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "local", 2023-07-21T22:37:18.238 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.238 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.239 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.239 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T22:37:18.239 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.239 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.239 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.240 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.240 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.240 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.240 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.240 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "selftest", 2023-07-21T22:37:18.241 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.241 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.241 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.241 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.241 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.242 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.242 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.242 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.242 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.242 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.243 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.243 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.243 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.243 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.243 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.244 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.244 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.244 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.244 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.244 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.245 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.245 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.245 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.245 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.245 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.246 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.246 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.246 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.246 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.246 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.246 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.247 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.247 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.247 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.247 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.247 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.248 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.248 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.248 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.248 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.248 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.249 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.249 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.249 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.249 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.249 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.250 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.250 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.250 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.250 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.250 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.251 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.251 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.251 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.251 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.251 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.251 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.252 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.252 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.252 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.252 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.253 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.253 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.253 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.253 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.253 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.254 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.254 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.254 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.254 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.254 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.255 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.255 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.255 INFO:teuthology.orchestra.run.smithi066.stdout: "roption1": { 2023-07-21T22:37:18.255 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption1", 2023-07-21T22:37:18.255 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.256 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.256 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.256 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.256 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.256 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.257 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.257 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.257 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.257 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.257 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.258 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.258 INFO:teuthology.orchestra.run.smithi066.stdout: "roption2": { 2023-07-21T22:37:18.258 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "roption2", 2023-07-21T22:37:18.258 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.258 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.259 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.259 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "xyz", 2023-07-21T22:37:18.259 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.259 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.259 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.260 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.260 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.260 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.260 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.260 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.261 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption1": { 2023-07-21T22:37:18.261 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption1", 2023-07-21T22:37:18.261 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.261 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.261 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.261 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.262 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.262 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.262 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.262 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.262 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.263 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.263 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.263 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.263 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption2": { 2023-07-21T22:37:18.263 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption2", 2023-07-21T22:37:18.264 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.264 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.264 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.264 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.264 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.265 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.265 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.265 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.265 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.265 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.265 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.266 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.266 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption3": { 2023-07-21T22:37:18.266 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption3", 2023-07-21T22:37:18.266 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "float", 2023-07-21T22:37:18.266 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.267 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.267 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.267 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.267 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.267 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.268 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.268 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.268 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.268 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.268 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.269 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption4": { 2023-07-21T22:37:18.269 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption4", 2023-07-21T22:37:18.269 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.269 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.269 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.269 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.270 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.270 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.270 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.270 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.270 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.271 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.271 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.271 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.271 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption5": { 2023-07-21T22:37:18.271 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption5", 2023-07-21T22:37:18.272 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.272 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.272 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.272 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.272 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.273 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.273 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.273 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.273 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.273 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.274 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.274 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.274 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption6": { 2023-07-21T22:37:18.274 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption6", 2023-07-21T22:37:18.274 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.274 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.275 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.275 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.275 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.275 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.275 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.276 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.276 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.276 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.276 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.276 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.277 INFO:teuthology.orchestra.run.smithi066.stdout: "rwoption7": { 2023-07-21T22:37:18.277 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rwoption7", 2023-07-21T22:37:18.277 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.277 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.277 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.278 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.278 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "1", 2023-07-21T22:37:18.278 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "42", 2023-07-21T22:37:18.278 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.278 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.278 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.279 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.279 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.279 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.279 INFO:teuthology.orchestra.run.smithi066.stdout: "testkey": { 2023-07-21T22:37:18.279 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testkey", 2023-07-21T22:37:18.280 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.280 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.280 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.280 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.280 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.281 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.281 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.281 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.281 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.281 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.282 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.282 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.282 INFO:teuthology.orchestra.run.smithi066.stdout: "testlkey": { 2023-07-21T22:37:18.282 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testlkey", 2023-07-21T22:37:18.282 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.282 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.283 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.283 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.283 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.283 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.283 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.284 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.284 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.284 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.284 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.284 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.285 INFO:teuthology.orchestra.run.smithi066.stdout: "testnewline": { 2023-07-21T22:37:18.285 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "testnewline", 2023-07-21T22:37:18.285 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.285 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.285 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.286 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.286 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.286 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.286 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.286 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.287 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.287 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.287 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.287 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.288 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.288 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.288 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-07-21T22:37:18.288 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.288 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.289 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.289 INFO:teuthology.orchestra.run.smithi066.stdout: "allow_m_granularity": { 2023-07-21T22:37:18.289 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "allow_m_granularity", 2023-07-21T22:37:18.289 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.289 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.289 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.290 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.290 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.290 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.290 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.290 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T22:37:18.291 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.291 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.291 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.291 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.291 INFO:teuthology.orchestra.run.smithi066.stdout: "dump_on_update": { 2023-07-21T22:37:18.292 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "dump_on_update", 2023-07-21T22:37:18.292 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.292 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.292 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.292 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.293 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.293 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.293 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.293 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "dump database to debug log on update", 2023-07-21T22:37:18.293 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.294 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.294 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.294 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.294 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.294 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.294 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.295 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.295 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.295 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.295 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.295 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.296 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.296 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.296 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.296 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.296 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.297 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.297 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.297 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.297 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.297 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.298 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.298 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.298 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.298 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.298 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.299 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.299 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.299 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.299 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.299 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.300 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.300 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.300 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.300 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.300 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.301 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.301 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.301 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.301 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.301 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.302 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.302 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.302 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.302 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.302 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.302 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.303 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.303 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.303 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.303 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.303 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.304 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.304 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.304 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.304 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.304 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.305 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.305 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.305 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.305 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.305 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.306 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.306 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.306 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.306 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.306 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.307 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.307 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.307 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.307 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.307 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.308 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.308 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.308 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.308 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.309 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "stats", 2023-07-21T22:37:18.309 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.309 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.309 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.309 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.309 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.310 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.310 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.310 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.310 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.310 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.311 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.311 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.311 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.311 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.311 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.312 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.312 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.312 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.312 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.312 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.313 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.313 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.313 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.313 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.314 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.314 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.314 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.314 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.315 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.315 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.315 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.315 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.315 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.316 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.316 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.316 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.316 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.316 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.317 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.317 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.317 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.317 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.317 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.318 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.318 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.318 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.318 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.318 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.319 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.319 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.319 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.319 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.320 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.320 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.320 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.320 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.320 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.321 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.321 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.321 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.321 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.321 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.322 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.322 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.322 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.322 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.322 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.323 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.323 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.323 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.323 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.323 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.324 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.324 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.324 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.324 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.324 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "status", 2023-07-21T22:37:18.324 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.325 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.325 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.325 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.325 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.325 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.326 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.326 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.326 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.326 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.326 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.327 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.327 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.327 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.327 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.327 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.328 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.328 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.328 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.328 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.328 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.329 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.329 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.329 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.329 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.329 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.330 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.330 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.330 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.330 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.330 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.330 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.331 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.331 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.331 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.331 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.331 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.332 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.332 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.332 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.332 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.332 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.333 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.333 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.333 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.333 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.333 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.334 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.334 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.334 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.334 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.334 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.335 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.335 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.335 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.335 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.335 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.336 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.337 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.337 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.337 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.337 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.337 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.338 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.338 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.338 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.338 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.338 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.339 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.339 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.339 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.339 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telegraf", 2023-07-21T22:37:18.340 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.340 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.340 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.340 INFO:teuthology.orchestra.run.smithi066.stdout: "address": { 2023-07-21T22:37:18.340 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "address", 2023-07-21T22:37:18.341 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.341 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.341 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.341 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T22:37:18.341 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.341 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.342 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.342 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.342 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.342 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.342 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.343 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.343 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:18.343 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:18.343 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:18.343 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.344 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.344 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "15", 2023-07-21T22:37:18.344 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.344 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.344 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.345 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.345 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.345 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.345 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.345 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.346 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.346 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.346 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.346 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.346 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.346 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.347 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.347 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.347 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.347 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.347 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.348 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.348 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.348 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.348 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.348 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.349 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.349 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.349 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.349 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.349 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.350 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.350 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.350 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.350 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.350 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.351 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.351 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.351 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.351 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.351 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.351 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.352 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.352 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.352 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.352 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.352 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.353 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.353 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.353 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.353 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.353 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.354 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.354 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.354 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.354 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.354 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.355 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.355 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.355 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.355 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.355 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.356 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.356 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.356 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.356 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.356 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.357 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.357 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.357 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.357 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.357 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.357 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.358 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.358 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.358 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.358 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.358 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.359 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.360 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "telemetry", 2023-07-21T22:37:18.360 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.360 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.360 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.360 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_basic": { 2023-07-21T22:37:18.361 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_basic", 2023-07-21T22:37:18.361 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.361 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.361 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.361 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.361 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.362 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.362 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.362 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T22:37:18.362 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.362 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.363 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.363 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.363 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_crash": { 2023-07-21T22:37:18.363 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_crash", 2023-07-21T22:37:18.363 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.364 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.364 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.364 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.364 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.364 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.365 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.365 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T22:37:18.365 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.365 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.365 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.365 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.366 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_device": { 2023-07-21T22:37:18.366 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_device", 2023-07-21T22:37:18.366 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.366 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.366 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.367 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "True", 2023-07-21T22:37:18.367 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.367 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.367 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.367 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T22:37:18.368 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.368 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.368 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.368 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.368 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_ident": { 2023-07-21T22:37:18.369 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_ident", 2023-07-21T22:37:18.369 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.369 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.369 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.369 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.370 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.370 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.370 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.370 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T22:37:18.370 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.371 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.371 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.371 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.371 INFO:teuthology.orchestra.run.smithi066.stdout: "channel_perf": { 2023-07-21T22:37:18.371 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "channel_perf", 2023-07-21T22:37:18.372 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.372 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.372 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.372 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.372 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.373 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.373 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.373 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T22:37:18.373 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.373 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.373 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.374 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.374 INFO:teuthology.orchestra.run.smithi066.stdout: "contact": { 2023-07-21T22:37:18.374 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "contact", 2023-07-21T22:37:18.374 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.374 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.375 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.375 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.375 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.375 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.375 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.376 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.376 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.376 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.376 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.376 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.377 INFO:teuthology.orchestra.run.smithi066.stdout: "description": { 2023-07-21T22:37:18.377 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "description", 2023-07-21T22:37:18.377 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.377 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.377 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.378 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.378 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.378 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.378 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.378 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.379 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.379 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.379 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.379 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.379 INFO:teuthology.orchestra.run.smithi066.stdout: "device_url": { 2023-07-21T22:37:18.379 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "device_url", 2023-07-21T22:37:18.380 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.380 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.380 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.380 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T22:37:18.380 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.381 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.381 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.381 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.381 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.381 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.382 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.382 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.382 INFO:teuthology.orchestra.run.smithi066.stdout: "enabled": { 2023-07-21T22:37:18.382 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "enabled", 2023-07-21T22:37:18.382 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.383 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.383 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.383 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.383 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.383 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.384 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.384 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.384 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.384 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.384 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.385 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.385 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:18.385 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:18.385 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.385 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.386 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.386 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "24", 2023-07-21T22:37:18.386 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "8", 2023-07-21T22:37:18.386 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.386 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.386 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.387 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.387 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.387 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.387 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.387 INFO:teuthology.orchestra.run.smithi066.stdout: "last_opt_revision": { 2023-07-21T22:37:18.388 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "last_opt_revision", 2023-07-21T22:37:18.388 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.388 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.388 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.388 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "1", 2023-07-21T22:37:18.389 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.389 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.389 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.389 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.389 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.390 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.390 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.390 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.390 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard": { 2023-07-21T22:37:18.390 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard", 2023-07-21T22:37:18.391 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.391 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.391 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.391 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.391 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.391 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.392 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.392 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.392 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.392 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.392 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.393 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.393 INFO:teuthology.orchestra.run.smithi066.stdout: "leaderboard_description": { 2023-07-21T22:37:18.393 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "leaderboard_description", 2023-07-21T22:37:18.393 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.393 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.394 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.394 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.394 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.394 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.394 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.395 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.395 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.395 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.395 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.395 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.396 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.396 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.396 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.396 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.396 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.397 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.397 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.397 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.397 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.397 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.397 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.398 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.398 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.398 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.398 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.398 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.399 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.399 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.399 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.399 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.400 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.400 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.400 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.400 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.400 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.401 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.401 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.401 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.401 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.401 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.401 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.402 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.402 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.402 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.402 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.402 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.403 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.403 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.403 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.403 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.403 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.404 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.404 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.404 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.404 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.404 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.405 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.405 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.405 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.405 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.405 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.405 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.406 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.406 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.406 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.406 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.406 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.407 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.407 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.407 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.407 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.407 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.408 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.408 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.408 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.408 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.408 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.409 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.409 INFO:teuthology.orchestra.run.smithi066.stdout: "organization": { 2023-07-21T22:37:18.409 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "organization", 2023-07-21T22:37:18.409 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.410 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.410 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.410 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.410 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.410 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.410 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.411 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.411 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.411 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.411 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.411 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.412 INFO:teuthology.orchestra.run.smithi066.stdout: "proxy": { 2023-07-21T22:37:18.412 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "proxy", 2023-07-21T22:37:18.412 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.412 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.412 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.413 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.413 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.413 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.413 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.413 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.414 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.414 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.414 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.414 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.414 INFO:teuthology.orchestra.run.smithi066.stdout: "url": { 2023-07-21T22:37:18.414 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "url", 2023-07-21T22:37:18.415 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.415 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.415 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.415 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T22:37:18.415 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.416 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.416 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.416 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.416 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.416 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.417 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.417 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.417 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.417 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.418 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "test_orchestrator", 2023-07-21T22:37:18.418 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.418 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.418 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.418 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.419 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.419 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.419 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.419 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.419 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.419 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.420 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.420 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.420 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.420 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.420 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.421 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.421 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.421 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.421 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.421 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.422 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.422 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.422 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.422 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.423 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.423 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.423 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.423 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.423 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.423 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.424 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.424 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.424 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.424 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.424 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.425 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.425 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.425 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.425 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.426 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.426 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.426 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.426 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.426 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.427 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.427 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.427 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.427 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.427 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.427 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.428 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.428 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.428 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.428 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.428 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.429 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.429 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.429 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.429 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.429 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.430 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.430 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.430 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.430 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.430 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.431 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.431 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.431 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.431 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.431 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.432 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.432 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.432 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.432 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.432 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-07-21T22:37:18.432 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.433 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.433 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.433 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.433 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.433 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.434 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.434 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.434 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.434 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.434 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.435 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.435 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.435 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.435 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.435 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.436 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.436 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.436 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.436 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.436 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.437 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.437 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.437 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.437 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.437 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.437 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.438 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.438 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.438 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.438 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.438 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.439 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.439 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.439 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.439 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.439 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.440 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.440 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.440 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.440 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.440 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.441 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.441 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.441 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.441 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.441 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.441 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.442 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.442 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.442 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.442 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.442 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.443 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.443 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.443 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.443 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.443 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.444 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.444 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.444 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.444 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.444 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.445 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.445 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.445 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.445 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.445 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.446 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.446 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.446 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.446 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.446 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.446 INFO:teuthology.orchestra.run.smithi066.stdout: "max_concurrent_clones": { 2023-07-21T22:37:18.447 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "max_concurrent_clones", 2023-07-21T22:37:18.447 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.447 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.447 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.447 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "4", 2023-07-21T22:37:18.448 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.448 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.448 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.448 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T22:37:18.448 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.449 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.449 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.449 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.449 INFO:teuthology.orchestra.run.smithi066.stdout: "snapshot_clone_delay": { 2023-07-21T22:37:18.449 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snapshot_clone_delay", 2023-07-21T22:37:18.450 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.450 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.450 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.450 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "0", 2023-07-21T22:37:18.450 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.451 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.451 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.451 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T22:37:18.451 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.451 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.451 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.452 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.452 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.452 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.452 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.452 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix", 2023-07-21T22:37:18.453 INFO:teuthology.orchestra.run.smithi066.stdout: "can_run": true, 2023-07-21T22:37:18.453 INFO:teuthology.orchestra.run.smithi066.stdout: "error_string": "", 2023-07-21T22:37:18.453 INFO:teuthology.orchestra.run.smithi066.stdout: "module_options": { 2023-07-21T22:37:18.453 INFO:teuthology.orchestra.run.smithi066.stdout: "discovery_interval": { 2023-07-21T22:37:18.453 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "discovery_interval", 2023-07-21T22:37:18.454 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "uint", 2023-07-21T22:37:18.454 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.454 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.454 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "100", 2023-07-21T22:37:18.454 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.455 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.455 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.455 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.455 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.455 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.456 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.456 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.456 INFO:teuthology.orchestra.run.smithi066.stdout: "identifier": { 2023-07-21T22:37:18.456 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "identifier", 2023-07-21T22:37:18.456 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.456 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.457 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.457 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.457 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.457 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.457 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.458 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.458 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.458 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.458 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.458 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.459 INFO:teuthology.orchestra.run.smithi066.stdout: "interval": { 2023-07-21T22:37:18.459 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "interval", 2023-07-21T22:37:18.459 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "secs", 2023-07-21T22:37:18.459 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.459 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.460 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "60", 2023-07-21T22:37:18.460 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.460 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.460 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.460 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: "log_level": { 2023-07-21T22:37:18.461 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_level", 2023-07-21T22:37:18.462 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.462 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.462 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.462 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.462 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.463 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.463 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.463 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.463 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.463 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.464 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.464 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.464 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.464 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.464 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.465 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.465 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.465 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.465 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.465 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster": { 2023-07-21T22:37:18.465 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster", 2023-07-21T22:37:18.466 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.466 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.466 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.466 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.466 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.467 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.467 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.467 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.467 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.467 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.468 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.468 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_cluster_level": { 2023-07-21T22:37:18.468 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_cluster_level", 2023-07-21T22:37:18.468 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.469 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.469 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.469 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "info", 2023-07-21T22:37:18.469 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.469 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.469 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [ 2023-07-21T22:37:18.470 INFO:teuthology.orchestra.run.smithi066.stdout: "", 2023-07-21T22:37:18.470 INFO:teuthology.orchestra.run.smithi066.stdout: "critical", 2023-07-21T22:37:18.470 INFO:teuthology.orchestra.run.smithi066.stdout: "debug", 2023-07-21T22:37:18.470 INFO:teuthology.orchestra.run.smithi066.stdout: "error", 2023-07-21T22:37:18.470 INFO:teuthology.orchestra.run.smithi066.stdout: "info", 2023-07-21T22:37:18.471 INFO:teuthology.orchestra.run.smithi066.stdout: "warning" 2023-07-21T22:37:18.471 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.471 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.471 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.471 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.472 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.472 INFO:teuthology.orchestra.run.smithi066.stdout: "log_to_file": { 2023-07-21T22:37:18.472 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "log_to_file", 2023-07-21T22:37:18.472 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "bool", 2023-07-21T22:37:18.473 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.473 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 1, 2023-07-21T22:37:18.473 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "False", 2023-07-21T22:37:18.473 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.473 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.474 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.474 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.474 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.474 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.474 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.475 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.475 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_host": { 2023-07-21T22:37:18.475 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_host", 2023-07-21T22:37:18.475 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.475 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.475 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.476 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "", 2023-07-21T22:37:18.476 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.476 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.476 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.476 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.477 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.477 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.477 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.477 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_port": { 2023-07-21T22:37:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_port", 2023-07-21T22:37:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "int", 2023-07-21T22:37:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.478 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "10051", 2023-07-21T22:37:18.479 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.479 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.479 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.479 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.479 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.480 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.480 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.480 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.480 INFO:teuthology.orchestra.run.smithi066.stdout: "zabbix_sender": { 2023-07-21T22:37:18.480 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "zabbix_sender", 2023-07-21T22:37:18.480 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "str", 2023-07-21T22:37:18.481 INFO:teuthology.orchestra.run.smithi066.stdout: "level": "advanced", 2023-07-21T22:37:18.481 INFO:teuthology.orchestra.run.smithi066.stdout: "flags": 0, 2023-07-21T22:37:18.481 INFO:teuthology.orchestra.run.smithi066.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T22:37:18.481 INFO:teuthology.orchestra.run.smithi066.stdout: "min": "", 2023-07-21T22:37:18.481 INFO:teuthology.orchestra.run.smithi066.stdout: "max": "", 2023-07-21T22:37:18.482 INFO:teuthology.orchestra.run.smithi066.stdout: "enum_allowed": [], 2023-07-21T22:37:18.482 INFO:teuthology.orchestra.run.smithi066.stdout: "desc": "", 2023-07-21T22:37:18.482 INFO:teuthology.orchestra.run.smithi066.stdout: "long_desc": "", 2023-07-21T22:37:18.482 INFO:teuthology.orchestra.run.smithi066.stdout: "tags": [], 2023-07-21T22:37:18.482 INFO:teuthology.orchestra.run.smithi066.stdout: "see_also": [] 2023-07-21T22:37:18.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.483 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.483 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.483 INFO:teuthology.orchestra.run.smithi066.stdout: "services": { 2023-07-21T22:37:18.484 INFO:teuthology.orchestra.run.smithi066.stdout: "dashboard": "https://172.21.15.66:8443/" 2023-07-21T22:37:18.484 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.484 INFO:teuthology.orchestra.run.smithi066.stdout: "always_on_modules": { 2023-07-21T22:37:18.484 INFO:teuthology.orchestra.run.smithi066.stdout: "octopus": [ 2023-07-21T22:37:18.484 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-07-21T22:37:18.485 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-07-21T22:37:18.485 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-07-21T22:37:18.485 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-07-21T22:37:18.485 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-07-21T22:37:18.485 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-07-21T22:37:18.485 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-07-21T22:37:18.486 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-07-21T22:37:18.486 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-07-21T22:37:18.486 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-07-21T22:37:18.486 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.486 INFO:teuthology.orchestra.run.smithi066.stdout: "pacific": [ 2023-07-21T22:37:18.487 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-07-21T22:37:18.487 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-07-21T22:37:18.487 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-07-21T22:37:18.487 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-07-21T22:37:18.487 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-07-21T22:37:18.488 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-07-21T22:37:18.488 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-07-21T22:37:18.488 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-07-21T22:37:18.488 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-07-21T22:37:18.488 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-07-21T22:37:18.489 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.489 INFO:teuthology.orchestra.run.smithi066.stdout: "quincy": [ 2023-07-21T22:37:18.489 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-07-21T22:37:18.489 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-07-21T22:37:18.489 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-07-21T22:37:18.490 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-07-21T22:37:18.490 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-07-21T22:37:18.490 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-07-21T22:37:18.490 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-07-21T22:37:18.490 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-07-21T22:37:18.490 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-07-21T22:37:18.491 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-07-21T22:37:18.491 INFO:teuthology.orchestra.run.smithi066.stdout: ], 2023-07-21T22:37:18.491 INFO:teuthology.orchestra.run.smithi066.stdout: "reef": [ 2023-07-21T22:37:18.491 INFO:teuthology.orchestra.run.smithi066.stdout: "balancer", 2023-07-21T22:37:18.491 INFO:teuthology.orchestra.run.smithi066.stdout: "crash", 2023-07-21T22:37:18.492 INFO:teuthology.orchestra.run.smithi066.stdout: "devicehealth", 2023-07-21T22:37:18.492 INFO:teuthology.orchestra.run.smithi066.stdout: "orchestrator", 2023-07-21T22:37:18.492 INFO:teuthology.orchestra.run.smithi066.stdout: "pg_autoscaler", 2023-07-21T22:37:18.492 INFO:teuthology.orchestra.run.smithi066.stdout: "progress", 2023-07-21T22:37:18.492 INFO:teuthology.orchestra.run.smithi066.stdout: "rbd_support", 2023-07-21T22:37:18.493 INFO:teuthology.orchestra.run.smithi066.stdout: "status", 2023-07-21T22:37:18.493 INFO:teuthology.orchestra.run.smithi066.stdout: "telemetry", 2023-07-21T22:37:18.493 INFO:teuthology.orchestra.run.smithi066.stdout: "volumes" 2023-07-21T22:37:18.493 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.493 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.494 INFO:teuthology.orchestra.run.smithi066.stdout: "last_failure_osd_epoch": 94, 2023-07-21T22:37:18.494 INFO:teuthology.orchestra.run.smithi066.stdout: "active_clients": [ 2023-07-21T22:37:18.494 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.494 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "devicehealth", 2023-07-21T22:37:18.494 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:18.495 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.495 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:18.495 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:37:18.495 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 3361783254 2023-07-21T22:37:18.495 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.496 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.496 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.496 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "libcephsqlite", 2023-07-21T22:37:18.496 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:18.496 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.497 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:18.497 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:37:18.497 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2721744988 2023-07-21T22:37:18.497 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.497 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.498 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.498 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "rbd_support", 2023-07-21T22:37:18.498 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:18.498 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.499 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:18.499 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:37:18.499 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 4079380174 2023-07-21T22:37:18.499 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.499 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.500 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.500 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-07-21T22:37:18.500 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:18.500 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.501 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:18.501 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:37:18.501 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2339188063 2023-07-21T22:37:18.501 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.501 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.501 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.502 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "snap_schedule", 2023-07-21T22:37:18.502 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:18.502 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.502 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:18.503 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:37:18.503 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 2176337567 2023-07-21T22:37:18.503 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.503 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.503 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:18.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.504 INFO:teuthology.orchestra.run.smithi066.stdout: "name": "volumes", 2023-07-21T22:37:18.504 INFO:teuthology.orchestra.run.smithi066.stdout: "addrvec": [ 2023-07-21T22:37:18.504 INFO:teuthology.orchestra.run.smithi066.stdout: { 2023-07-21T22:37:18.504 INFO:teuthology.orchestra.run.smithi066.stdout: "type": "v2", 2023-07-21T22:37:18.505 INFO:teuthology.orchestra.run.smithi066.stdout: "addr": "172.21.15.66:0", 2023-07-21T22:37:18.505 INFO:teuthology.orchestra.run.smithi066.stdout: "nonce": 36632066 2023-07-21T22:37:18.505 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.505 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.505 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:18.506 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2023-07-21T22:37:18.506 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:37:18.507 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1892199429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:37:18.508 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:15 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3022337275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:37:18.508 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:16 smithi104 ceph-mon[129021]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 294 op/s 2023-07-21T22:37:18.508 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1892199429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:37:18.509 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:15 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3022337275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:37:18.509 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:16 smithi149 ceph-mon[129268]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 294 op/s 2023-07-21T22:37:18.509 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:18 smithi149 ceph-mon[129268]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-07-21T22:37:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18000a1300 msgr2=0x7f18000a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18000a1300 0x7f18000a37b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1804009fd0 tx=0x7f1804009260 comp rx=0 tx=0).stop 2023-07-21T22:37:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 msgr2=0x7f18101982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:18.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f18101982d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f17fc005910 tx=0x7f17fc005720 comp rx=0 tx=0).stop 2023-07-21T22:37:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 shutdown_connections 2023-07-21T22:37:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18100ffad0 0x7f1810197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f18000a1300 0x7f18000a37b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1810105ae0 0x7f181019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:18.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 --2- 172.21.15.66:0/3022337275 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1810100480 0x7f18101982d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.881+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 >> 172.21.15.66:0/3022337275 conn(0x7f18100fb590 msgr2=0x7f181010b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.882+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 shutdown_connections 2023-07-21T22:37:18.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:14.882+0000 7f1816fff700 1 -- 172.21.15.66:0/3022337275 wait complete. 2023-07-21T22:37:18.518 DEBUG:tasks.check_counter:Getting stats from a 2023-07-21T22:37:18.519 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-21T22:37:18.520 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1892199429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:37:18.520 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:15 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3022337275' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T22:37:18.521 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:16 smithi066 ceph-mon[126379]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.7 MiB/s wr, 294 op/s 2023-07-21T22:37:18.521 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:18 smithi066 ceph-mon[126379]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-07-21T22:37:18.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:18 smithi104 ceph-mon[129021]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-07-21T22:37:19.283 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:37:20.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:20 smithi066 ceph-mon[126379]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-07-21T22:37:20.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:20 smithi149 ceph-mon[129268]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-07-21T22:37:20.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:20 smithi104 ceph-mon[129021]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 261 op/s 2023-07-21T22:37:21.463 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:37:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 306, 2023-07-21T22:37:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 295, 2023-07-21T22:37:21.464 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1581584, 2023-07-21T22:37:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 168416, 2023-07-21T22:37:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-07-21T22:37:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2023-07-21T22:37:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.068980442, 2023-07-21T22:37:21.465 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.025267643, 2023-07-21T22:37:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.042807704, 2023-07-21T22:37:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.006739411, 2023-07-21T22:37:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:21.466 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 294, 2023-07-21T22:37:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.026603849, 2023-07-21T22:37:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000090489 2023-07-21T22:37:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:21.467 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1581584, 2023-07-21T22:37:21.468 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 168416 2023-07-21T22:37:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 3, 2023-07-21T22:37:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 226, 2023-07-21T22:37:21.469 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 480, 2023-07-21T22:37:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 267008, 2023-07-21T22:37:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:21.470 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.026311939, 2023-07-21T22:37:21.471 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.017679498, 2023-07-21T22:37:21.471 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.004040028, 2023-07-21T22:37:21.471 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:21.471 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:21.471 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 223, 2023-07-21T22:37:21.472 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.034545878, 2023-07-21T22:37:21.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000154914 2023-07-21T22:37:21.472 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.472 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:21.472 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:21.473 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:21.473 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:21.473 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.473 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T22:37:21.473 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 267008 2023-07-21T22:37:21.474 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.474 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:21.474 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 4, 2023-07-21T22:37:21.474 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 4, 2023-07-21T22:37:21.474 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2944, 2023-07-21T22:37:21.475 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 448, 2023-07-21T22:37:21.475 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2023-07-21T22:37:21.475 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-07-21T22:37:21.475 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001507815, 2023-07-21T22:37:21.475 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000073759, 2023-07-21T22:37:21.476 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001369613, 2023-07-21T22:37:21.476 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:21.476 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:21.476 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2023-07-21T22:37:21.476 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001250810, 2023-07-21T22:37:21.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000416936 2023-07-21T22:37:21.477 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.477 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:21.477 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:21.477 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:21.478 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:21.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.478 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-21T22:37:21.478 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:37:21.478 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.478 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-07-21T22:37:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2023-07-21T22:37:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-07-21T22:37:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-07-21T22:37:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:21.480 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:21.480 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:21.480 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:21.480 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:21.481 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:21.481 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:21.481 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:21.482 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:21.482 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:21.483 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:21.483 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:21.483 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:21.484 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:21.484 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:21.485 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:21.485 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:21.485 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:21.486 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:21.486 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:21.487 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:21.487 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:21.487 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:21.488 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:21.488 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:21.488 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:21.488 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:21.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:21.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:21.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:21.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:21.489 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-07-21T22:37:21.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:21.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:21.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:21.490 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:21.490 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 150776, 2023-07-21T22:37:21.491 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 32, 2023-07-21T22:37:21.491 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:21.491 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:21.491 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-07-21T22:37:21.491 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-07-21T22:37:21.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:21.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:21.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:21.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:21.492 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2023-07-21T22:37:21.493 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2023-07-21T22:37:21.493 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:21.493 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:21.493 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:21.493 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-07-21T22:37:21.494 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2023-07-21T22:37:21.494 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2023-07-21T22:37:21.494 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:21.495 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-07-21T22:37:21.495 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:21.495 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-07-21T22:37:21.495 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:21.495 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:21.496 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-07-21T22:37:21.496 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-07-21T22:37:21.496 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-07-21T22:37:21.496 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 309, 2023-07-21T22:37:21.496 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1538388, 2023-07-21T22:37:21.497 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:21.497 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 309, 2023-07-21T22:37:21.497 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-07-21T22:37:21.497 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-07-21T22:37:21.497 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 309, 2023-07-21T22:37:21.498 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1538388, 2023-07-21T22:37:21.498 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-07-21T22:37:21.498 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:21.498 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:21.498 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:21.499 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:21.499 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:21.499 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:37:22.000 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-21T22:37:22.000 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-21T22:37:22.001 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-07-21T22:37:22.001 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-07-21T22:37:22.001 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-07-21T22:37:22.001 DEBUG:tasks.check_counter:Getting stats from d 2023-07-21T22:37:22.002 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-21T22:37:22.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:22 smithi066 ceph-mon[126379]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.8 MiB/s wr, 111 op/s 2023-07-21T22:37:22.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:22 smithi149 ceph-mon[129268]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.8 MiB/s wr, 111 op/s 2023-07-21T22:37:22.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:22 smithi104 ceph-mon[129021]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.8 MiB/s wr, 111 op/s 2023-07-21T22:37:22.803 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:37:24.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:24 smithi066 ceph-mon[126379]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.8 MiB/s wr, 111 op/s 2023-07-21T22:37:24.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:24 smithi149 ceph-mon[129268]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.8 MiB/s wr, 111 op/s 2023-07-21T22:37:24.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:24 smithi104 ceph-mon[129021]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.8 MiB/s wr, 111 op/s 2023-07-21T22:37:24.638 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:37:24.639 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:24.639 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2023-07-21T22:37:24.639 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2023-07-21T22:37:24.639 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:37:24.640 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2023-07-21T22:37:24.640 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-07-21T22:37:24.640 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2023-07-21T22:37:24.640 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.003802025, 2023-07-21T22:37:24.640 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:37:24.641 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000848573, 2023-07-21T22:37:24.641 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:24.641 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:24.641 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:24.641 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:24.642 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:24.642 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.642 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:24.642 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:24.642 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:24.643 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:24.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.643 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:37:24.643 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:37:24.643 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.643 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:24.644 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 7, 2023-07-21T22:37:24.644 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 231, 2023-07-21T22:37:24.644 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3424, 2023-07-21T22:37:24.644 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 267984, 2023-07-21T22:37:24.644 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:24.645 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:24.645 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.025722780, 2023-07-21T22:37:24.645 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.017228772, 2023-07-21T22:37:24.645 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.004148653, 2023-07-21T22:37:24.645 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:24.646 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:24.646 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 229, 2023-07-21T22:37:24.646 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.042902752, 2023-07-21T22:37:24.646 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000187348 2023-07-21T22:37:24.646 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.647 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:24.647 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:24.647 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:24.647 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:24.647 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.647 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-07-21T22:37:24.648 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 267984 2023-07-21T22:37:24.648 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.648 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:24.648 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 311, 2023-07-21T22:37:24.648 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 300, 2023-07-21T22:37:24.649 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1582944, 2023-07-21T22:37:24.649 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 171376, 2023-07-21T22:37:24.649 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2023-07-21T22:37:24.649 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-07-21T22:37:24.649 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.058864824, 2023-07-21T22:37:24.650 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.017901524, 2023-07-21T22:37:24.650 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.042591937, 2023-07-21T22:37:24.650 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.006901002, 2023-07-21T22:37:24.650 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:24.650 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 299, 2023-07-21T22:37:24.651 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.025560231, 2023-07-21T22:37:24.651 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000085485 2023-07-21T22:37:24.651 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.651 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:24.651 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:24.651 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:24.652 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:24.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.652 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1582944, 2023-07-21T22:37:24.652 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 171376 2023-07-21T22:37:24.652 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.653 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-07-21T22:37:24.653 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2023-07-21T22:37:24.653 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-07-21T22:37:24.653 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.653 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-07-21T22:37:24.654 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:24.654 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:24.654 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:24.654 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:24.654 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:24.655 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:24.655 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:24.655 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:24.655 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:24.655 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:24.655 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:24.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:24.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:24.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:24.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:24.656 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:24.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:24.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:24.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:24.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:24.657 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:24.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:24.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:24.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:24.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:24.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:24.658 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:24.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:24.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:24.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:24.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:24.659 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-07-21T22:37:24.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:24.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:24.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:24.660 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:24.660 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 149785, 2023-07-21T22:37:24.661 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 32, 2023-07-21T22:37:24.661 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:24.661 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:24.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-07-21T22:37:24.661 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-07-21T22:37:24.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:24.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:24.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:24.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:24.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2023-07-21T22:37:24.662 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2023-07-21T22:37:24.663 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:24.663 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:24.663 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:24.663 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-07-21T22:37:24.663 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2023-07-21T22:37:24.664 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2023-07-21T22:37:24.664 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:24.664 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-07-21T22:37:24.664 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:24.664 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-07-21T22:37:24.665 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:24.665 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:24.665 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-07-21T22:37:24.665 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-07-21T22:37:24.665 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-07-21T22:37:24.666 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 314, 2023-07-21T22:37:24.666 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1539038, 2023-07-21T22:37:24.666 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:24.666 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 314, 2023-07-21T22:37:24.666 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-07-21T22:37:24.666 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-07-21T22:37:24.667 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 314, 2023-07-21T22:37:24.667 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1539038, 2023-07-21T22:37:24.667 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-07-21T22:37:24.667 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:24.667 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:24.668 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:24.668 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:24.668 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:24.668 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:37:25.163 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-21T22:37:25.164 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-21T22:37:25.164 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-21T22:37:25.165 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-21T22:37:25.165 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-21T22:37:25.165 DEBUG:tasks.check_counter:Getting stats from g 2023-07-21T22:37:25.166 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-21T22:37:25.981 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:37:26.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:26 smithi066 ceph-mon[126379]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 163 op/s 2023-07-21T22:37:26.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:26 smithi149 ceph-mon[129268]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 163 op/s 2023-07-21T22:37:26.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:26 smithi104 ceph-mon[129021]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.1 MiB/s wr, 163 op/s 2023-07-21T22:37:27.595 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:37:27.595 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:27.595 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 0, 2023-07-21T22:37:27.595 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 2, 2023-07-21T22:37:27.596 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:37:27.596 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 0, 2023-07-21T22:37:27.596 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-07-21T22:37:27.596 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2023-07-21T22:37:27.597 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.004102358, 2023-07-21T22:37:27.597 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:37:27.597 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.000941011, 2023-07-21T22:37:27.597 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:27.597 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:27.597 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:27.598 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:27.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:27.598 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.598 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:27.598 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:27.599 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:27.599 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:27.599 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.599 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:37:27.599 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:37:27.600 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.600 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:27.600 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 308, 2023-07-21T22:37:27.600 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 297, 2023-07-21T22:37:27.600 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1582128, 2023-07-21T22:37:27.601 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 169600, 2023-07-21T22:37:27.601 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2023-07-21T22:37:27.601 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-07-21T22:37:27.601 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.063602910, 2023-07-21T22:37:27.601 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.024215006, 2023-07-21T22:37:27.602 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.040401772, 2023-07-21T22:37:27.602 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.006715994, 2023-07-21T22:37:27.602 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:27.602 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 296, 2023-07-21T22:37:27.602 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.025018555, 2023-07-21T22:37:27.602 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000084522 2023-07-21T22:37:27.603 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.603 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:27.603 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:27.603 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:27.603 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:27.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.604 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1582128, 2023-07-21T22:37:27.604 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 169600 2023-07-21T22:37:27.604 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.604 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:27.605 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 7, 2023-07-21T22:37:27.605 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 229, 2023-07-21T22:37:27.605 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 3424, 2023-07-21T22:37:27.605 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 267632, 2023-07-21T22:37:27.605 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:27.606 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:27.606 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.025189839, 2023-07-21T22:37:27.606 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.016541948, 2023-07-21T22:37:27.606 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.004336448, 2023-07-21T22:37:27.606 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:27.606 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:27.607 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 227, 2023-07-21T22:37:27.607 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.034167112, 2023-07-21T22:37:27.607 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000150515 2023-07-21T22:37:27.607 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.607 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:27.608 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:27.608 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:27.608 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:27.608 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.608 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 3424, 2023-07-21T22:37:27.609 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 267632 2023-07-21T22:37:27.609 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.609 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-07-21T22:37:27.609 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2023-07-21T22:37:27.609 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-07-21T22:37:27.610 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.610 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-07-21T22:37:27.610 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:27.610 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:27.610 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:27.610 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:27.611 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:27.611 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:27.611 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:27.611 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:27.611 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:27.612 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:27.612 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:27.612 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:27.612 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:27.612 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:27.613 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:27.613 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:27.613 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:27.613 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:27.613 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:27.614 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:27.614 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:27.614 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:27.614 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:27.614 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:27.615 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:27.615 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:27.615 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:27.615 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:27.615 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:27.616 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:27.616 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:27.616 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-07-21T22:37:27.616 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:27.616 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:27.617 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:27.617 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:27.617 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 148714, 2023-07-21T22:37:27.617 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 30, 2023-07-21T22:37:27.617 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:27.617 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:27.618 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-07-21T22:37:27.618 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-07-21T22:37:27.618 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:27.618 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:27.618 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2023-07-21T22:37:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2023-07-21T22:37:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:27.619 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-07-21T22:37:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2023-07-21T22:37:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2023-07-21T22:37:27.620 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-07-21T22:37:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-07-21T22:37:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:27.621 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:27.622 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-07-21T22:37:27.622 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-07-21T22:37:27.622 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-07-21T22:37:27.622 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 311, 2023-07-21T22:37:27.622 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1538648, 2023-07-21T22:37:27.623 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:27.623 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 311, 2023-07-21T22:37:27.623 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-07-21T22:37:27.623 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-07-21T22:37:27.623 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 311, 2023-07-21T22:37:27.623 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1538648, 2023-07-21T22:37:27.624 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-07-21T22:37:27.624 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:27.624 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:27.624 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:27.624 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:27.625 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:27.625 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:37:28.086 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-21T22:37:28.086 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-21T22:37:28.086 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-07-21T22:37:28.087 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-07-21T22:37:28.087 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-07-21T22:37:28.087 DEBUG:tasks.check_counter:Getting stats from j 2023-07-21T22:37:28.087 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-21T22:37:28.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:28 smithi066 ceph-mon[126379]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.7 MiB/s wr, 159 op/s 2023-07-21T22:37:28.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:28 smithi149 ceph-mon[129268]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.7 MiB/s wr, 159 op/s 2023-07-21T22:37:28.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:28 smithi104 ceph-mon[129021]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.7 MiB/s wr, 159 op/s 2023-07-21T22:37:28.829 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T22:37:30.395 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2023-07-21T22:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 314, 2023-07-21T22:37:30.396 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 303, 2023-07-21T22:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 1583760, 2023-07-21T22:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 173152, 2023-07-21T22:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 4, 2023-07-21T22:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 0, 2023-07-21T22:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.067786394, 2023-07-21T22:37:30.397 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.024353128, 2023-07-21T22:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.042784239, 2023-07-21T22:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.006685735, 2023-07-21T22:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 302, 2023-07-21T22:37:30.398 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.022450192, 2023-07-21T22:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000074338 2023-07-21T22:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:30.399 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 1583760, 2023-07-21T22:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 173152 2023-07-21T22:37:30.400 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 3, 2023-07-21T22:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 232, 2023-07-21T22:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 480, 2023-07-21T22:37:30.401 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 268064, 2023-07-21T22:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.025633524, 2023-07-21T22:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.017399803, 2023-07-21T22:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.003555441, 2023-07-21T22:37:30.402 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 229, 2023-07-21T22:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.040174096, 2023-07-21T22:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000175432 2023-07-21T22:37:30.403 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:30.404 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T22:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 268064 2023-07-21T22:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:30.405 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_messages": 4, 2023-07-21T22:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages": 4, 2023-07-21T22:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_bytes": 2944, 2023-07-21T22:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_bytes": 448, 2023-07-21T22:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_created_connections": 2, 2023-07-21T22:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_active_connections": 1, 2023-07-21T22:37:30.406 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_total_time": 0.001458484, 2023-07-21T22:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_send_time": 0.000064771, 2023-07-21T22:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_recv_time": 0.001347973, 2023-07-21T22:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:30.407 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 3, 2023-07-21T22:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.001235014, 2023-07-21T22:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000411671 2023-07-21T22:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:30.408 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-21T22:37:30.409 INFO:teuthology.orchestra.run.smithi066.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "cct": { 2023-07-21T22:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "total_workers": 0, 2023-07-21T22:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "unhealthy_workers": 0 2023-07-21T22:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.410 INFO:teuthology.orchestra.run.smithi066.stdout: "mempool": { 2023-07-21T22:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:30.411 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:30.412 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:30.413 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:30.414 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:30.415 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:30.416 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_items": 0, 2023-07-21T22:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:30.417 INFO:teuthology.orchestra.run.smithi066.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_bytes": 150776, 2023-07-21T22:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_anon_items": 32, 2023-07-21T22:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:30.418 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_bytes": 0, 2023-07-21T22:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_items": 0, 2023-07-21T22:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:30.419 INFO:teuthology.orchestra.run.smithi066.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_bytes": 0, 2023-07-21T22:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_items": 0, 2023-07-21T22:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:30.420 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "pgmap_items": 0, 2023-07-21T22:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_bytes": 928, 2023-07-21T22:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "mds_co_items": 2, 2023-07-21T22:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:30.421 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_1_items": 0, 2023-07-21T22:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "unittest_2_items": 0 2023-07-21T22:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2023-07-21T22:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "val": 0, 2023-07-21T22:37:30.422 INFO:teuthology.orchestra.run.smithi066.stdout: "max": 104857600, 2023-07-21T22:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "get_started": 0, 2023-07-21T22:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "get": 317, 2023-07-21T22:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "get_sum": 1539428, 2023-07-21T22:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:30.423 INFO:teuthology.orchestra.run.smithi066.stdout: "get_or_fail_success": 317, 2023-07-21T22:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "take": 0, 2023-07-21T22:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "take_sum": 0, 2023-07-21T22:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "put": 317, 2023-07-21T22:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "put_sum": 1539428, 2023-07-21T22:37:30.424 INFO:teuthology.orchestra.run.smithi066.stdout: "wait": { 2023-07-21T22:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "avgcount": 0, 2023-07-21T22:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "sum": 0.000000000, 2023-07-21T22:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: "avgtime": 0.000000000 2023-07-21T22:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:30.425 INFO:teuthology.orchestra.run.smithi066.stdout: } 2023-07-21T22:37:30.426 INFO:teuthology.orchestra.run.smithi066.stdout:} 2023-07-21T22:37:30.426 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:30 smithi066 ceph-mon[126379]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 4.6 MiB/s wr, 152 op/s 2023-07-21T22:37:30.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:30 smithi149 ceph-mon[129268]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 4.6 MiB/s wr, 152 op/s 2023-07-21T22:37:30.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:30 smithi104 ceph-mon[129021]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 4.6 MiB/s wr, 152 op/s 2023-07-21T22:37:30.930 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-21T22:37:30.930 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-21T22:37:30.931 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-21T22:37:30.931 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-21T22:37:30.931 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-21T22:37:30.931 DEBUG:tasks.check_counter:Getting stats from b 2023-07-21T22:37:30.932 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-21T22:37:31.523 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:37:32.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:32 smithi149 ceph-mon[129268]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.3 MiB/s wr, 184 op/s 2023-07-21T22:37:32.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:32 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:32.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:32 smithi104 ceph-mon[129021]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.3 MiB/s wr, 184 op/s 2023-07-21T22:37:32.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:32 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:32.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:32 smithi066 ceph-mon[126379]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.3 MiB/s wr, 184 op/s 2023-07-21T22:37:32.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:32 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:34.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:34 smithi104 ceph-mon[129021]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2023-07-21T22:37:34.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:34 smithi149 ceph-mon[129268]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2023-07-21T22:37:34.510 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-21T22:37:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:34.510 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 145865, 2023-07-21T22:37:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 154877, 2023-07-21T22:37:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 36807086, 2023-07-21T22:37:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 410948515, 2023-07-21T22:37:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 159, 2023-07-21T22:37:34.511 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 8.375354729, 2023-07-21T22:37:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 3.157655586, 2023-07-21T22:37:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 4.451693788, 2023-07-21T22:37:34.512 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 1.290452154, 2023-07-21T22:37:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 154875, 2023-07-21T22:37:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 8.215774937, 2023-07-21T22:37:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000053047 2023-07-21T22:37:34.513 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 222081, 2023-07-21T22:37:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.032626808, 2023-07-21T22:37:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000146 2023-07-21T22:37:34.514 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 43104, 2023-07-21T22:37:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 2848 2023-07-21T22:37:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:34.515 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 256895, 2023-07-21T22:37:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 365474, 2023-07-21T22:37:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 92446592, 2023-07-21T22:37:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 527787582, 2023-07-21T22:37:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2023-07-21T22:37:34.516 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2023-07-21T22:37:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 16.202928533, 2023-07-21T22:37:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 7.219083553, 2023-07-21T22:37:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 10.267715232, 2023-07-21T22:37:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 1.196760539, 2023-07-21T22:37:34.517 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 365474, 2023-07-21T22:37:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 36.645792229, 2023-07-21T22:37:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000100269 2023-07-21T22:37:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:34.518 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 456194, 2023-07-21T22:37:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.074428476, 2023-07-21T22:37:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000163 2023-07-21T22:37:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 82052160, 2023-07-21T22:37:34.519 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 266462640 2023-07-21T22:37:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1921237, 2023-07-21T22:37:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1951488, 2023-07-21T22:37:34.520 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 313872733, 2023-07-21T22:37:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 472592732, 2023-07-21T22:37:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 26, 2023-07-21T22:37:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2023-07-21T22:37:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 66.637859551, 2023-07-21T22:37:34.521 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 29.179858560, 2023-07-21T22:37:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 44.400194449, 2023-07-21T22:37:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.774235142, 2023-07-21T22:37:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1951485, 2023-07-21T22:37:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 23.736758326, 2023-07-21T22:37:34.522 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000012163 2023-07-21T22:37:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3823972, 2023-07-21T22:37:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.368655573, 2023-07-21T22:37:34.523 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000096 2023-07-21T22:37:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 537152, 2023-07-21T22:37:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 291200 2023-07-21T22:37:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.524 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2023-07-21T22:37:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2023-07-21T22:37:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2023-07-21T22:37:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2023-07-21T22:37:34.525 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2023-07-21T22:37:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2023-07-21T22:37:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 94587, 2023-07-21T22:37:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 90.269785657, 2023-07-21T22:37:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000954357 2023-07-21T22:37:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.526 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2023-07-21T22:37:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2023-07-21T22:37:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2023-07-21T22:37:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 32338, 2023-07-21T22:37:34.527 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 7.954969162, 2023-07-21T22:37:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000245994 2023-07-21T22:37:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2023-07-21T22:37:34.528 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 161131, 2023-07-21T22:37:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 149114, 2023-07-21T22:37:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2023-07-21T22:37:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 149114, 2023-07-21T22:37:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 880.362269553, 2023-07-21T22:37:34.529 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005903954 2023-07-21T22:37:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2023-07-21T22:37:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 12017, 2023-07-21T22:37:34.530 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 19, 2023-07-21T22:37:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 2, 2023-07-21T22:37:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 108, 2023-07-21T22:37:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 9, 2023-07-21T22:37:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 12, 2023-07-21T22:37:34.531 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 2130, 2023-07-21T22:37:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 548, 2023-07-21T22:37:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 1175, 2023-07-21T22:37:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 407, 2023-07-21T22:37:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 1330, 2023-07-21T22:37:34.532 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 2545, 2023-07-21T22:37:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 251, 2023-07-21T22:37:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 251, 2023-07-21T22:37:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 11, 2023-07-21T22:37:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 231357, 2023-07-21T22:37:34.533 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 163942, 2023-07-21T22:37:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 11997, 2023-07-21T22:37:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 81, 2023-07-21T22:37:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 2, 2023-07-21T22:37:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2023-07-21T22:37:34.534 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 1300, 2023-07-21T22:37:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 58861, 2023-07-21T22:37:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 17, 2023-07-21T22:37:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 18, 2023-07-21T22:37:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 1715, 2023-07-21T22:37:34.535 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 15, 2023-07-21T22:37:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 514, 2023-07-21T22:37:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2023-07-21T22:37:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 1511, 2023-07-21T22:37:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2023-07-21T22:37:34.536 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 725, 2023-07-21T22:37:34.537 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 213972440, 2023-07-21T22:37:34.537 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2023-07-21T22:37:34.537 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 1511, 2023-07-21T22:37:34.537 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2023-07-21T22:37:34.537 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:37:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:37:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 445, 2023-07-21T22:37:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 17, 2023-07-21T22:37:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 428, 2023-07-21T22:37:34.538 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 1066, 2023-07-21T22:37:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 16, 2023-07-21T22:37:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 24748, 2023-07-21T22:37:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 79725, 2023-07-21T22:37:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:37:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 799, 2023-07-21T22:37:34.539 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:37:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 50877, 2023-07-21T22:37:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 18041, 2023-07-21T22:37:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 4855, 2023-07-21T22:37:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 90565 2023-07-21T22:37:34.540 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2023-07-21T22:37:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 10, 2023-07-21T22:37:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 34, 2023-07-21T22:37:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 21, 2023-07-21T22:37:34.541 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 98, 2023-07-21T22:37:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 149, 2023-07-21T22:37:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2023-07-21T22:37:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2023-07-21T22:37:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:37:34.542 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 19571, 2023-07-21T22:37:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 19571, 2023-07-21T22:37:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2023-07-21T22:37:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2023-07-21T22:37:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2023-07-21T22:37:34.543 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:37:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:37:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2023-07-21T22:37:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2023-07-21T22:37:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 17, 2023-07-21T22:37:34.544 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 19, 2023-07-21T22:37:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2023-07-21T22:37:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 21, 2023-07-21T22:37:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2023-07-21T22:37:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2023-07-21T22:37:34.545 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2023-07-21T22:37:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 113053, 2023-07-21T22:37:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 39566, 2023-07-21T22:37:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 39566, 2023-07-21T22:37:34.546 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 73487, 2023-07-21T22:37:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2023-07-21T22:37:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 430, 2023-07-21T22:37:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 190, 2023-07-21T22:37:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 62, 2023-07-21T22:37:34.547 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 62, 2023-07-21T22:37:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 129, 2023-07-21T22:37:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2023-07-21T22:37:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 1, 2023-07-21T22:37:34.548 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 243279850, 2023-07-21T22:37:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 746574968, 2023-07-21T22:37:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2023-07-21T22:37:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2023-07-21T22:37:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 50469, 2023-07-21T22:37:34.549 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 230.668830453, 2023-07-21T22:37:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004570505 2023-07-21T22:37:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2023-07-21T22:37:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.550 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2023-07-21T22:37:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 741, 2023-07-21T22:37:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 21681, 2023-07-21T22:37:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 20940, 2023-07-21T22:37:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 128, 2023-07-21T22:37:34.551 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 314, 2023-07-21T22:37:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 186, 2023-07-21T22:37:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 2154, 2023-07-21T22:37:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 38884, 2023-07-21T22:37:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 36730, 2023-07-21T22:37:34.552 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 251, 2023-07-21T22:37:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 21476, 2023-07-21T22:37:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 21225, 2023-07-21T22:37:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 81556, 2023-07-21T22:37:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 207156 2023-07-21T22:37:34.553 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2023-07-21T22:37:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 223144, 2023-07-21T22:37:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 21330, 2023-07-21T22:37:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 161131, 2023-07-21T22:37:34.554 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 3007, 2023-07-21T22:37:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 42961, 2023-07-21T22:37:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2023-07-21T22:37:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 20291, 2023-07-21T22:37:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 86.220952622, 2023-07-21T22:37:34.555 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004249221 2023-07-21T22:37:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2023-07-21T22:37:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 26290, 2023-07-21T22:37:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 157.792458303, 2023-07-21T22:37:34.556 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006001995 2023-07-21T22:37:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2023-07-21T22:37:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.557 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2023-07-21T22:37:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.558 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2023-07-21T22:37:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 27093, 2023-07-21T22:37:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 235.765192072, 2023-07-21T22:37:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.008702070 2023-07-21T22:37:34.559 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2023-07-21T22:37:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.560 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2023-07-21T22:37:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.561 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2023-07-21T22:37:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.562 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2023-07-21T22:37:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.563 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2023-07-21T22:37:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.564 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2023-07-21T22:37:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 16, 2023-07-21T22:37:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.006697590, 2023-07-21T22:37:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000418599 2023-07-21T22:37:34.565 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2023-07-21T22:37:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 119, 2023-07-21T22:37:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.631825497, 2023-07-21T22:37:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005309457 2023-07-21T22:37:34.566 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2023-07-21T22:37:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.567 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2023-07-21T22:37:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 16, 2023-07-21T22:37:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.849494114, 2023-07-21T22:37:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.115593382 2023-07-21T22:37:34.568 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2023-07-21T22:37:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 9, 2023-07-21T22:37:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.017033974, 2023-07-21T22:37:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001892663 2023-07-21T22:37:34.569 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2023-07-21T22:37:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 49846, 2023-07-21T22:37:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 239.466987592, 2023-07-21T22:37:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004804136 2023-07-21T22:37:34.570 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2023-07-21T22:37:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4869, 2023-07-21T22:37:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 63.939754609, 2023-07-21T22:37:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.013132009 2023-07-21T22:37:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.571 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2023-07-21T22:37:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.572 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2023-07-21T22:37:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.573 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2023-07-21T22:37:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2023-07-21T22:37:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000918847, 2023-07-21T22:37:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000918847 2023-07-21T22:37:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.574 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2023-07-21T22:37:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10, 2023-07-21T22:37:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.230343684, 2023-07-21T22:37:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.023034368 2023-07-21T22:37:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.575 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2023-07-21T22:37:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.576 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2023-07-21T22:37:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 24, 2023-07-21T22:37:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.558970528, 2023-07-21T22:37:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.023290438 2023-07-21T22:37:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.577 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2023-07-21T22:37:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.578 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2023-07-21T22:37:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 952, 2023-07-21T22:37:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.760329825, 2023-07-21T22:37:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001849085 2023-07-21T22:37:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.579 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2023-07-21T22:37:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.580 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2023-07-21T22:37:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7, 2023-07-21T22:37:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018696612, 2023-07-21T22:37:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002670944 2023-07-21T22:37:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2023-07-21T22:37:34.581 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2023-07-21T22:37:34.582 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 19571, 2023-07-21T22:37:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 92.102613684, 2023-07-21T22:37:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004706076 2023-07-21T22:37:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:37:34.583 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:37:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2023-07-21T22:37:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.584 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2023-07-21T22:37:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2023-07-21T22:37:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 6, 2023-07-21T22:37:34.585 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 4, 2023-07-21T22:37:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2023-07-21T22:37:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2023-07-21T22:37:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 29902, 2023-07-21T22:37:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 14951, 2023-07-21T22:37:34.586 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 956 2023-07-21T22:37:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2023-07-21T22:37:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:34.587 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:34.588 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:34.589 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:34.590 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:34.591 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:34.592 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:34.593 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2023-07-21T22:37:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:34.594 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 330277, 2023-07-21T22:37:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 537, 2023-07-21T22:37:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2023-07-21T22:37:34.595 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2023-07-21T22:37:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:34.596 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 9608, 2023-07-21T22:37:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 184, 2023-07-21T22:37:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:34.597 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2023-07-21T22:37:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 4030767, 2023-07-21T22:37:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 42539, 2023-07-21T22:37:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2023-07-21T22:37:34.598 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2023-07-21T22:37:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2023-07-21T22:37:34.599 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2023-07-21T22:37:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2023-07-21T22:37:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 93977, 2023-07-21T22:37:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 747219552, 2023-07-21T22:37:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 31, 2023-07-21T22:37:34.600 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 93977, 2023-07-21T22:37:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2023-07-21T22:37:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 93977, 2023-07-21T22:37:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 585.934395899, 2023-07-21T22:37:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006234870 2023-07-21T22:37:34.601 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2023-07-21T22:37:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2023-07-21T22:37:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 93977, 2023-07-21T22:37:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 94540 2023-07-21T22:37:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.602 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 93977, 2023-07-21T22:37:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 2197, 2023-07-21T22:37:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 91780, 2023-07-21T22:37:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2023-07-21T22:37:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2023-07-21T22:37:34.603 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2023-07-21T22:37:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 51, 2023-07-21T22:37:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 648, 2023-07-21T22:37:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 51100, 2023-07-21T22:37:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 379, 2023-07-21T22:37:34.604 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2023-07-21T22:37:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2023-07-21T22:37:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2023-07-21T22:37:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 9, 2023-07-21T22:37:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 39943, 2023-07-21T22:37:34.605 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2023-07-21T22:37:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2023-07-21T22:37:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2023-07-21T22:37:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1536, 2023-07-21T22:37:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 98, 2023-07-21T22:37:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:37:34.606 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2023-07-21T22:37:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:37:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2023-07-21T22:37:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2023-07-21T22:37:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2023-07-21T22:37:34.607 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:37:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2023-07-21T22:37:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:37:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2023-07-21T22:37:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2023-07-21T22:37:34.608 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2023-07-21T22:37:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2023-07-21T22:37:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2023-07-21T22:37:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2023-07-21T22:37:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2023-07-21T22:37:34.609 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2023-07-21T22:37:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2023-07-21T22:37:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2023-07-21T22:37:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2023-07-21T22:37:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2023-07-21T22:37:34.610 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2023-07-21T22:37:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2023-07-21T22:37:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2023-07-21T22:37:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2023-07-21T22:37:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2023-07-21T22:37:34.611 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 114, 2023-07-21T22:37:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2023-07-21T22:37:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 30, 2023-07-21T22:37:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2023-07-21T22:37:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2023-07-21T22:37:34.612 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2023-07-21T22:37:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2023-07-21T22:37:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 559, 2023-07-21T22:37:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 42, 2023-07-21T22:37:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 175 2023-07-21T22:37:34.613 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2023-07-21T22:37:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2023-07-21T22:37:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:37:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2023-07-21T22:37:34.614 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2023-07-21T22:37:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2023-07-21T22:37:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 182511 2023-07-21T22:37:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2023-07-21T22:37:34.615 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2023-07-21T22:37:34.616 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 16, 2023-07-21T22:37:34.616 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2023-07-21T22:37:34.616 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 8, 2023-07-21T22:37:34.616 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 19571, 2023-07-21T22:37:34.616 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2023-07-21T22:37:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:34.617 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 2323993, 2023-07-21T22:37:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 224294541, 2023-07-21T22:37:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 2323993, 2023-07-21T22:37:34.618 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2323993, 2023-07-21T22:37:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 224294541, 2023-07-21T22:37:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:34.619 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2023-07-21T22:37:34.620 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:34.621 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:34.621 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:34.621 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:34.621 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:34.621 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 93977, 2023-07-21T22:37:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 749355343, 2023-07-21T22:37:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 93977, 2023-07-21T22:37:34.622 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 749355343, 2023-07-21T22:37:34.623 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:34.623 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.623 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.623 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.623 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2023-07-21T22:37:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2023-07-21T22:37:34.624 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:34.625 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 93977, 2023-07-21T22:37:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 93977, 2023-07-21T22:37:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 93977, 2023-07-21T22:37:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 93977, 2023-07-21T22:37:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.626 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:37:34.627 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2023-07-21T22:37:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 19571, 2023-07-21T22:37:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 2136103, 2023-07-21T22:37:34.628 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 19571, 2023-07-21T22:37:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 631, 2023-07-21T22:37:34.629 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2136103, 2023-07-21T22:37:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x555ada58e0a0": { 2023-07-21T22:37:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2023-07-21T22:37:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:34.631 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 113053, 2023-07-21T22:37:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 742380664, 2023-07-21T22:37:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 113053, 2023-07-21T22:37:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:34.632 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 50469, 2023-07-21T22:37:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 742380664, 2023-07-21T22:37:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:34.633 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.634 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:34.634 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-21T22:37:34.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:34 smithi066 ceph-mon[126379]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 3.1 MiB/s wr, 118 op/s 2023-07-21T22:37:34.978 INFO:tasks.check_counter:Daemon mds.b mds.exported=18 2023-07-21T22:37:34.978 INFO:tasks.check_counter:Daemon mds.b mds.imported=15 2023-07-21T22:37:34.979 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-07-21T22:37:34.979 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2023-07-21T22:37:34.980 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2023-07-21T22:37:34.980 DEBUG:tasks.check_counter:Getting stats from e 2023-07-21T22:37:34.981 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-21T22:37:35.531 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:37:36.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:36 smithi104 ceph-mon[129021]: pgmap v506: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.1 MiB/s wr, 123 op/s 2023-07-21T22:37:36.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:36 smithi149 ceph-mon[129268]: pgmap v506: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.1 MiB/s wr, 123 op/s 2023-07-21T22:37:36.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:36 smithi066 ceph-mon[126379]: pgmap v506: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 3.1 MiB/s wr, 123 op/s 2023-07-21T22:37:36.724 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-21T22:37:36.724 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:36.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2546, 2023-07-21T22:37:36.724 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2546, 2023-07-21T22:37:36.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 101185212, 2023-07-21T22:37:36.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 791017, 2023-07-21T22:37:36.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2023-07-21T22:37:36.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2023-07-21T22:37:36.725 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.247425855, 2023-07-21T22:37:36.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.056878813, 2023-07-21T22:37:36.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.105067411, 2023-07-21T22:37:36.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.068411331, 2023-07-21T22:37:36.726 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:36.726 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2545, 2023-07-21T22:37:36.727 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.100272379, 2023-07-21T22:37:36.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000039399 2023-07-21T22:37:36.727 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.727 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:36.727 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.728 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.728 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.728 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-07-21T22:37:36.728 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T22:37:36.728 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.729 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:36.729 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 732, 2023-07-21T22:37:36.729 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 961, 2023-07-21T22:37:36.729 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 84071612, 2023-07-21T22:37:36.729 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 770340, 2023-07-21T22:37:36.730 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2023-07-21T22:37:36.730 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 7, 2023-07-21T22:37:36.730 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.102490983, 2023-07-21T22:37:36.730 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.021413261, 2023-07-21T22:37:36.730 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.056210361, 2023-07-21T22:37:36.731 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.018481430, 2023-07-21T22:37:36.731 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:36.731 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 958, 2023-07-21T22:37:36.731 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.052406019, 2023-07-21T22:37:36.731 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000054703 2023-07-21T22:37:36.731 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.732 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:36.732 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.732 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.732 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.733 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2023-07-21T22:37:36.733 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 543248 2023-07-21T22:37:36.733 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.733 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:36.734 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1120, 2023-07-21T22:37:36.734 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1081, 2023-07-21T22:37:36.734 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 110987132, 2023-07-21T22:37:36.734 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 410996, 2023-07-21T22:37:36.734 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:36.735 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 4, 2023-07-21T22:37:36.735 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.158220956, 2023-07-21T22:37:36.735 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.029340406, 2023-07-21T22:37:36.735 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.107547394, 2023-07-21T22:37:36.735 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.025651127, 2023-07-21T22:37:36.736 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:36.736 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1080, 2023-07-21T22:37:36.736 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.056490080, 2023-07-21T22:37:36.736 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000052305 2023-07-21T22:37:36.736 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.737 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:36.737 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.737 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.737 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.737 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.737 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1736496, 2023-07-21T22:37:36.738 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 178368 2023-07-21T22:37:36.738 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.738 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2023-07-21T22:37:36.738 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2023-07-21T22:37:36.739 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2023-07-21T22:37:36.739 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.739 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2023-07-21T22:37:36.739 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2023-07-21T22:37:36.739 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2023-07-21T22:37:36.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6757, 2023-07-21T22:37:36.740 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.152547221, 2023-07-21T22:37:36.740 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000022576 2023-07-21T22:37:36.740 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.740 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.741 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2023-07-21T22:37:36.741 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2023-07-21T22:37:36.741 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2023-07-21T22:37:36.741 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.741 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.742 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.742 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.742 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.742 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2023-07-21T22:37:36.742 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2023-07-21T22:37:36.742 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2023-07-21T22:37:36.743 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2023-07-21T22:37:36.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.743 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.743 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.743 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.744 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2023-07-21T22:37:36.744 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2023-07-21T22:37:36.744 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2023-07-21T22:37:36.744 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2023-07-21T22:37:36.744 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2023-07-21T22:37:36.745 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2023-07-21T22:37:36.745 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2023-07-21T22:37:36.745 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 15832, 2023-07-21T22:37:36.745 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 0, 2023-07-21T22:37:36.745 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 0, 2023-07-21T22:37:36.746 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 15832, 2023-07-21T22:37:36.746 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 15832, 2023-07-21T22:37:36.746 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 50, 2023-07-21T22:37:36.746 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2023-07-21T22:37:36.746 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2023-07-21T22:37:36.746 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 6, 2023-07-21T22:37:36.747 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2023-07-21T22:37:36.747 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2023-07-21T22:37:36.747 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2023-07-21T22:37:36.747 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2023-07-21T22:37:36.747 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:37:36.748 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2023-07-21T22:37:36.748 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2023-07-21T22:37:36.748 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2023-07-21T22:37:36.748 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2023-07-21T22:37:36.748 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2023-07-21T22:37:36.749 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2023-07-21T22:37:36.749 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2023-07-21T22:37:36.749 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2023-07-21T22:37:36.749 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2023-07-21T22:37:36.749 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:37:36.750 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2023-07-21T22:37:36.750 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2023-07-21T22:37:36.750 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2023-07-21T22:37:36.750 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2023-07-21T22:37:36.750 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:37:36.750 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2023-07-21T22:37:36.751 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:37:36.751 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:37:36.751 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:37:36.751 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:37:36.751 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:37:36.752 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2023-07-21T22:37:36.752 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:37:36.752 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:37:36.752 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:37:36.752 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:37:36.753 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:37:36.753 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:37:36.753 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2023-07-21T22:37:36.753 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:37:36.753 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2023-07-21T22:37:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2023-07-21T22:37:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2023-07-21T22:37:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2023-07-21T22:37:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2023-07-21T22:37:36.754 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2023-07-21T22:37:36.755 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2023-07-21T22:37:36.755 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2023-07-21T22:37:36.755 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2023-07-21T22:37:36.755 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2023-07-21T22:37:36.755 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:37:36.756 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2023-07-21T22:37:36.756 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2023-07-21T22:37:36.756 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2023-07-21T22:37:36.756 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2023-07-21T22:37:36.756 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2023-07-21T22:37:36.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:37:36.757 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:37:36.757 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2023-07-21T22:37:36.757 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2023-07-21T22:37:36.757 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:37:36.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2023-07-21T22:37:36.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2023-07-21T22:37:36.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:37:36.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2023-07-21T22:37:36.758 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2023-07-21T22:37:36.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.759 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2023-07-21T22:37:36.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2023-07-21T22:37:36.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2023-07-21T22:37:36.759 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2023-07-21T22:37:36.759 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 0, 2023-07-21T22:37:36.760 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2023-07-21T22:37:36.760 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2023-07-21T22:37:36.760 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2023-07-21T22:37:36.760 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2023-07-21T22:37:36.760 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2023-07-21T22:37:36.761 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 112, 2023-07-21T22:37:36.761 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2023-07-21T22:37:36.761 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2023-07-21T22:37:36.761 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2023-07-21T22:37:36.761 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 0, 2023-07-21T22:37:36.762 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 297552363, 2023-07-21T22:37:36.762 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2023-07-21T22:37:36.762 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.762 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.762 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.762 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.763 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 88261 2023-07-21T22:37:36.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.763 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2023-07-21T22:37:36.763 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 793, 2023-07-21T22:37:36.763 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 16135, 2023-07-21T22:37:36.764 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 15342, 2023-07-21T22:37:36.764 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 62, 2023-07-21T22:37:36.764 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 146, 2023-07-21T22:37:36.764 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 84, 2023-07-21T22:37:36.764 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 15832, 2023-07-21T22:37:36.765 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 16487, 2023-07-21T22:37:36.765 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 655, 2023-07-21T22:37:36.765 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2023-07-21T22:37:36.765 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2023-07-21T22:37:36.765 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2023-07-21T22:37:36.765 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 66944, 2023-07-21T22:37:36.766 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 207156 2023-07-21T22:37:36.766 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.766 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2023-07-21T22:37:36.766 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2023-07-21T22:37:36.766 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2023-07-21T22:37:36.767 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2023-07-21T22:37:36.767 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2023-07-21T22:37:36.767 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2023-07-21T22:37:36.767 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2023-07-21T22:37:36.767 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.768 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.768 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.768 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2023-07-21T22:37:36.768 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.769 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.769 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.769 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2023-07-21T22:37:36.769 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.769 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.770 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.770 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2023-07-21T22:37:36.770 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.770 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.771 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.771 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2023-07-21T22:37:36.771 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.771 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.772 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.772 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2023-07-21T22:37:36.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.772 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.772 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.773 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.773 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2023-07-21T22:37:36.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.773 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.773 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.774 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.774 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2023-07-21T22:37:36.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.774 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.774 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.775 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.775 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2023-07-21T22:37:36.775 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.775 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.775 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.776 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2023-07-21T22:37:36.776 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.776 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.776 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.776 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.777 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2023-07-21T22:37:36.777 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.777 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.777 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.777 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.778 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2023-07-21T22:37:36.778 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.778 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.778 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.778 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.779 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2023-07-21T22:37:36.779 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.779 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.779 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.779 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.780 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2023-07-21T22:37:36.780 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.780 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.780 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.781 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2023-07-21T22:37:36.781 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.781 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.781 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.781 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.782 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2023-07-21T22:37:36.782 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.782 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.782 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.782 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.783 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2023-07-21T22:37:36.783 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.783 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.783 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.783 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.784 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2023-07-21T22:37:36.784 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.784 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.784 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.784 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.785 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2023-07-21T22:37:36.785 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.785 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.785 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.785 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.785 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2023-07-21T22:37:36.786 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.786 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.786 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.786 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.786 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2023-07-21T22:37:36.787 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.787 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.787 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.787 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.787 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2023-07-21T22:37:36.788 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.788 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.788 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.788 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.788 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2023-07-21T22:37:36.789 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.789 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.789 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.789 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.789 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2023-07-21T22:37:36.789 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.790 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.790 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.790 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.791 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2023-07-21T22:37:36.791 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.792 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.792 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.792 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.792 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2023-07-21T22:37:36.792 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.793 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.793 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.793 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.793 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2023-07-21T22:37:36.793 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.794 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.794 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.794 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.794 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2023-07-21T22:37:36.794 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.794 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.795 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.795 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.795 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2023-07-21T22:37:36.795 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.795 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.796 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.796 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.796 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:37:36.796 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:37:36.796 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2023-07-21T22:37:36.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.797 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.797 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.797 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.797 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.798 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2023-07-21T22:37:36.798 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2023-07-21T22:37:36.798 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 4, 2023-07-21T22:37:36.798 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 3, 2023-07-21T22:37:36.798 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 3, 2023-07-21T22:37:36.799 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2023-07-21T22:37:36.799 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2023-07-21T22:37:36.799 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2023-07-21T22:37:36.799 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 8204 2023-07-21T22:37:36.799 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.799 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2023-07-21T22:37:36.800 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:36.800 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:36.800 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:36.800 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:36.800 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:36.801 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:36.801 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:36.801 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:36.801 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:36.801 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:36.802 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:36.802 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:36.802 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:36.802 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:36.802 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:36.803 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:36.803 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:36.803 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:36.803 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:36.803 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:36.803 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:36.804 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:36.804 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:36.804 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:36.804 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:36.804 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:36.805 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:36.805 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:36.805 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:36.805 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:36.805 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:36.806 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2023-07-21T22:37:36.806 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:36.806 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:36.806 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:36.806 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:36.807 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 271235, 2023-07-21T22:37:36.807 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 972, 2023-07-21T22:37:36.807 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:36.807 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:36.807 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2023-07-21T22:37:36.807 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2023-07-21T22:37:36.808 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:36.808 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:36.808 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:36.808 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:36.808 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 9608, 2023-07-21T22:37:36.809 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 184, 2023-07-21T22:37:36.809 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:36.809 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:36.809 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:36.809 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2023-07-21T22:37:36.810 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 13257928, 2023-07-21T22:37:36.810 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 84889, 2023-07-21T22:37:36.810 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:36.810 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2023-07-21T22:37:36.810 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:36.811 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2023-07-21T22:37:36.811 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.811 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2023-07-21T22:37:36.811 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2023-07-21T22:37:36.811 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2023-07-21T22:37:36.812 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 4020, 2023-07-21T22:37:36.812 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2023-07-21T22:37:36.812 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2023-07-21T22:37:36.812 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 4020, 2023-07-21T22:37:36.812 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2023-07-21T22:37:36.813 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4020, 2023-07-21T22:37:36.813 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 5.450938472, 2023-07-21T22:37:36.813 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001355954 2023-07-21T22:37:36.813 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.813 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2023-07-21T22:37:36.813 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2023-07-21T22:37:36.814 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4020, 2023-07-21T22:37:36.814 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4021 2023-07-21T22:37:36.814 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.814 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 4020, 2023-07-21T22:37:36.814 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 4020, 2023-07-21T22:37:36.815 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2023-07-21T22:37:36.815 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2023-07-21T22:37:36.815 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2023-07-21T22:37:36.815 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 2239, 2023-07-21T22:37:36.815 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2023-07-21T22:37:36.816 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 1779, 2023-07-21T22:37:36.816 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2023-07-21T22:37:36.816 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2023-07-21T22:37:36.816 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2023-07-21T22:37:36.816 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2023-07-21T22:37:36.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2023-07-21T22:37:36.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2023-07-21T22:37:36.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2023-07-21T22:37:36.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2023-07-21T22:37:36.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2023-07-21T22:37:36.817 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2023-07-21T22:37:36.818 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2023-07-21T22:37:36.818 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2023-07-21T22:37:36.818 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:37:36.818 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2023-07-21T22:37:36.818 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:37:36.819 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2023-07-21T22:37:36.819 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2023-07-21T22:37:36.819 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2023-07-21T22:37:36.819 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:37:36.819 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2023-07-21T22:37:36.820 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:37:36.820 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2023-07-21T22:37:36.820 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2023-07-21T22:37:36.820 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2023-07-21T22:37:36.820 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2023-07-21T22:37:36.820 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2023-07-21T22:37:36.821 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2023-07-21T22:37:36.821 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2023-07-21T22:37:36.821 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2023-07-21T22:37:36.821 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2023-07-21T22:37:36.821 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2023-07-21T22:37:36.822 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2023-07-21T22:37:36.822 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2023-07-21T22:37:36.822 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2023-07-21T22:37:36.822 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2023-07-21T22:37:36.822 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2023-07-21T22:37:36.823 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2023-07-21T22:37:36.823 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2023-07-21T22:37:36.823 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 114, 2023-07-21T22:37:36.823 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2023-07-21T22:37:36.823 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 25, 2023-07-21T22:37:36.823 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 11, 2023-07-21T22:37:36.824 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2023-07-21T22:37:36.824 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 1, 2023-07-21T22:37:36.824 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2023-07-21T22:37:36.824 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2023-07-21T22:37:36.824 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2023-07-21T22:37:36.825 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2023-07-21T22:37:36.825 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.825 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2023-07-21T22:37:36.825 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2023-07-21T22:37:36.825 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:37:36.826 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2023-07-21T22:37:36.826 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2023-07-21T22:37:36.826 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.826 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2023-07-21T22:37:36.826 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2023-07-21T22:37:36.827 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.827 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2023-07-21T22:37:36.827 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2023-07-21T22:37:36.827 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:37:36.827 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2023-07-21T22:37:36.827 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2023-07-21T22:37:36.828 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2023-07-21T22:37:36.828 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2023-07-21T22:37:36.828 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.828 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:36.828 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:36.829 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:36.829 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:36.829 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 4394, 2023-07-21T22:37:36.829 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 295826573, 2023-07-21T22:37:36.829 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:36.830 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 4394, 2023-07-21T22:37:36.830 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:36.830 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:36.830 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4394, 2023-07-21T22:37:36.830 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 295826573, 2023-07-21T22:37:36.831 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:36.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.831 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.831 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.831 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.832 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.832 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2023-07-21T22:37:36.832 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:36.832 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:36.832 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:36.832 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:36.833 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:36.833 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:36.833 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:36.833 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 4020, 2023-07-21T22:37:36.833 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 293393636, 2023-07-21T22:37:36.834 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4020, 2023-07-21T22:37:36.834 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 293393636, 2023-07-21T22:37:36.834 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:36.834 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.834 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.835 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.835 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.835 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.835 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2023-07-21T22:37:36.835 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:36.836 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2023-07-21T22:37:36.836 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:36.836 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:36.836 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:36.836 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:36.837 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:36.837 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 4020, 2023-07-21T22:37:36.837 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 4020, 2023-07-21T22:37:36.837 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 4020, 2023-07-21T22:37:36.837 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 4020, 2023-07-21T22:37:36.837 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:36.838 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.838 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.838 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.838 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.838 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.839 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:37:36.839 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:36.839 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2023-07-21T22:37:36.839 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:36.839 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:36.840 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:36.840 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:36.840 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:36.840 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:36.840 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:36.841 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2023-07-21T22:37:36.841 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2023-07-21T22:37:36.841 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:36.841 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.841 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.841 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.842 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.842 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:36.842 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x557b6423a460": { 2023-07-21T22:37:36.842 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:36.842 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2023-07-21T22:37:36.843 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:36.843 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:36.843 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:36.843 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:36.843 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:36.844 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:36.844 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:36.844 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2023-07-21T22:37:36.844 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2023-07-21T22:37:36.844 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:36.844 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:36.845 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:36.845 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:36.845 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.845 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:36.845 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-21T22:37:37.071 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-07-21T22:37:37.072 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-07-21T22:37:37.072 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2023-07-21T22:37:37.073 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:37:37.073 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:37:37.073 DEBUG:tasks.check_counter:Getting stats from h 2023-07-21T22:37:37.073 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-21T22:37:37.623 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:37:38.369 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:38 smithi104 ceph-mon[129021]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2023-07-21T22:37:38.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:38 smithi149 ceph-mon[129268]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2023-07-21T22:37:38.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:38 smithi066 ceph-mon[126379]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.2 MiB/s wr, 54 op/s 2023-07-21T22:37:38.926 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-21T22:37:38.926 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:38.927 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 179408, 2023-07-21T22:37:38.927 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 274185, 2023-07-21T22:37:38.927 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 63037107, 2023-07-21T22:37:38.927 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 373548506, 2023-07-21T22:37:38.927 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 21, 2023-07-21T22:37:38.928 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2023-07-21T22:37:38.928 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 11.799364816, 2023-07-21T22:37:38.928 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 5.282445202, 2023-07-21T22:37:38.928 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 7.345838975, 2023-07-21T22:37:38.928 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.752203914, 2023-07-21T22:37:38.929 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:38.929 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 274182, 2023-07-21T22:37:38.929 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 33.689458277, 2023-07-21T22:37:38.929 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000122872 2023-07-21T22:37:38.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.929 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:38.930 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 333871, 2023-07-21T22:37:38.930 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.054749268, 2023-07-21T22:37:38.930 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000163 2023-07-21T22:37:38.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.930 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 57091312, 2023-07-21T22:37:38.931 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 227403440 2023-07-21T22:37:38.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.931 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:38.931 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1943478, 2023-07-21T22:37:38.931 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1912313, 2023-07-21T22:37:38.932 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 340620748, 2023-07-21T22:37:38.932 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 383512907, 2023-07-21T22:37:38.932 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 39, 2023-07-21T22:37:38.932 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2023-07-21T22:37:38.932 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 66.203859120, 2023-07-21T22:37:38.933 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 28.557409330, 2023-07-21T22:37:38.933 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 46.286117924, 2023-07-21T22:37:38.933 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.454063453, 2023-07-21T22:37:38.933 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:38.933 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1912312, 2023-07-21T22:37:38.933 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 19.500154403, 2023-07-21T22:37:38.934 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000010197 2023-07-21T22:37:38.934 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.934 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:38.934 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3816445, 2023-07-21T22:37:38.934 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.324972857, 2023-07-21T22:37:38.935 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000085 2023-07-21T22:37:38.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.935 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1984, 2023-07-21T22:37:38.935 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 124624 2023-07-21T22:37:38.935 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.936 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:38.936 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 38005, 2023-07-21T22:37:38.936 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 33743, 2023-07-21T22:37:38.936 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 14069221, 2023-07-21T22:37:38.936 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 84435018, 2023-07-21T22:37:38.936 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 9, 2023-07-21T22:37:38.937 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2023-07-21T22:37:38.937 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 2.251867208, 2023-07-21T22:37:38.937 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.774361139, 2023-07-21T22:37:38.937 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 1.044468843, 2023-07-21T22:37:38.937 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.547264160, 2023-07-21T22:37:38.938 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:38.938 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 33742, 2023-07-21T22:37:38.938 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 4.259417898, 2023-07-21T22:37:38.938 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000126234 2023-07-21T22:37:38.938 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.939 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:38.939 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 37462, 2023-07-21T22:37:38.939 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.005511071, 2023-07-21T22:37:38.939 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000147 2023-07-21T22:37:38.939 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.939 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1754240, 2023-07-21T22:37:38.940 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 621376 2023-07-21T22:37:38.940 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.940 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2023-07-21T22:37:38.940 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2023-07-21T22:37:38.940 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2023-07-21T22:37:38.941 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.941 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2023-07-21T22:37:38.941 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2023-07-21T22:37:38.941 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2023-07-21T22:37:38.941 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 41101, 2023-07-21T22:37:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 53.675358615, 2023-07-21T22:37:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001305938 2023-07-21T22:37:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2023-07-21T22:37:38.943 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2023-07-21T22:37:38.943 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2023-07-21T22:37:38.943 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 25158, 2023-07-21T22:37:38.943 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 5.083492272, 2023-07-21T22:37:38.943 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000202062 2023-07-21T22:37:38.943 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:38.944 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.944 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2023-07-21T22:37:38.944 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 111154, 2023-07-21T22:37:38.944 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 105192, 2023-07-21T22:37:38.944 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2023-07-21T22:37:38.945 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 105192, 2023-07-21T22:37:38.945 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 371.486962526, 2023-07-21T22:37:38.945 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003531513 2023-07-21T22:37:38.945 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.945 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2023-07-21T22:37:38.946 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 5962, 2023-07-21T22:37:38.946 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 14, 2023-07-21T22:37:38.946 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2023-07-21T22:37:38.946 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 91, 2023-07-21T22:37:38.946 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 9, 2023-07-21T22:37:38.946 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 7, 2023-07-21T22:37:38.947 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 2040, 2023-07-21T22:37:38.947 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 590, 2023-07-21T22:37:38.947 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 1268, 2023-07-21T22:37:38.947 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 182, 2023-07-21T22:37:38.947 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 1196, 2023-07-21T22:37:38.948 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 898, 2023-07-21T22:37:38.948 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 212, 2023-07-21T22:37:38.948 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 212, 2023-07-21T22:37:38.948 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 11, 2023-07-21T22:37:38.948 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 169420, 2023-07-21T22:37:38.949 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 100673, 2023-07-21T22:37:38.949 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 5949, 2023-07-21T22:37:38.949 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 79, 2023-07-21T22:37:38.949 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:37:38.949 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2023-07-21T22:37:38.950 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 1531, 2023-07-21T22:37:38.950 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 43306, 2023-07-21T22:37:38.950 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2023-07-21T22:37:38.950 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 18, 2023-07-21T22:37:38.950 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 613, 2023-07-21T22:37:38.951 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 15, 2023-07-21T22:37:38.951 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 949, 2023-07-21T22:37:38.951 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2023-07-21T22:37:38.951 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 486, 2023-07-21T22:37:38.951 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2023-07-21T22:37:38.951 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 704, 2023-07-21T22:37:38.952 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 211879339, 2023-07-21T22:37:38.952 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2023-07-21T22:37:38.952 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 486, 2023-07-21T22:37:38.952 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2023-07-21T22:37:38.952 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:37:38.953 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:37:38.953 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 30, 2023-07-21T22:37:38.953 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:37:38.953 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 30, 2023-07-21T22:37:38.953 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 456, 2023-07-21T22:37:38.954 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 10, 2023-07-21T22:37:38.954 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 13511, 2023-07-21T22:37:38.954 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 74802, 2023-07-21T22:37:38.954 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:37:38.954 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 740, 2023-07-21T22:37:38.955 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:37:38.955 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 36095, 2023-07-21T22:37:38.955 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 14565, 2023-07-21T22:37:38.955 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 3632, 2023-07-21T22:37:38.955 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 82796 2023-07-21T22:37:38.956 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.956 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2023-07-21T22:37:38.956 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 11, 2023-07-21T22:37:38.956 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 127, 2023-07-21T22:37:38.956 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 72, 2023-07-21T22:37:38.957 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 202, 2023-07-21T22:37:38.957 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 87, 2023-07-21T22:37:38.957 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2023-07-21T22:37:38.957 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2023-07-21T22:37:38.957 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:37:38.957 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 14833, 2023-07-21T22:37:38.958 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 14833, 2023-07-21T22:37:38.958 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2023-07-21T22:37:38.958 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2023-07-21T22:37:38.958 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2023-07-21T22:37:38.958 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:37:38.959 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:37:38.959 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2023-07-21T22:37:38.959 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2023-07-21T22:37:38.959 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:37:38.959 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 26, 2023-07-21T22:37:38.960 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2023-07-21T22:37:38.960 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 16, 2023-07-21T22:37:38.960 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2023-07-21T22:37:38.960 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2023-07-21T22:37:38.960 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.960 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2023-07-21T22:37:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 88264, 2023-07-21T22:37:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2023-07-21T22:37:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2023-07-21T22:37:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 88264, 2023-07-21T22:37:38.961 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2023-07-21T22:37:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2023-07-21T22:37:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 112, 2023-07-21T22:37:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2023-07-21T22:37:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2023-07-21T22:37:38.962 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 112, 2023-07-21T22:37:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2023-07-21T22:37:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2023-07-21T22:37:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 4194304, 2023-07-21T22:37:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 297572011, 2023-07-21T22:37:38.963 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2023-07-21T22:37:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2023-07-21T22:37:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 22221, 2023-07-21T22:37:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 129.782214953, 2023-07-21T22:37:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005840520 2023-07-21T22:37:38.964 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2023-07-21T22:37:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2023-07-21T22:37:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 826, 2023-07-21T22:37:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 16327, 2023-07-21T22:37:38.965 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 15501, 2023-07-21T22:37:38.966 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 99, 2023-07-21T22:37:38.966 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 285, 2023-07-21T22:37:38.966 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 186, 2023-07-21T22:37:38.966 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 2040, 2023-07-21T22:37:38.966 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 29921, 2023-07-21T22:37:38.967 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 27881, 2023-07-21T22:37:38.967 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 211, 2023-07-21T22:37:38.967 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 16581, 2023-07-21T22:37:38.967 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 16370, 2023-07-21T22:37:38.967 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 77840, 2023-07-21T22:37:38.968 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 198964 2023-07-21T22:37:38.968 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.968 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2023-07-21T22:37:38.968 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 154632, 2023-07-21T22:37:38.968 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 321, 2023-07-21T22:37:38.969 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 111154, 2023-07-21T22:37:38.969 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 2202, 2023-07-21T22:37:38.969 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 7801, 2023-07-21T22:37:38.969 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2023-07-21T22:37:38.969 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 14791, 2023-07-21T22:37:38.969 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 43.776933558, 2023-07-21T22:37:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002959700 2023-07-21T22:37:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2023-07-21T22:37:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 898, 2023-07-21T22:37:38.970 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 10.065367951, 2023-07-21T22:37:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.011208650 2023-07-21T22:37:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2023-07-21T22:37:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.971 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2023-07-21T22:37:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.972 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2023-07-21T22:37:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 22630, 2023-07-21T22:37:38.973 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 85.913470211, 2023-07-21T22:37:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003796441 2023-07-21T22:37:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2023-07-21T22:37:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.974 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2023-07-21T22:37:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2023-07-21T22:37:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000158598, 2023-07-21T22:37:38.975 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000158598 2023-07-21T22:37:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2023-07-21T22:37:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.976 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2023-07-21T22:37:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.977 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2023-07-21T22:37:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.978 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2023-07-21T22:37:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.979 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2023-07-21T22:37:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.980 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2023-07-21T22:37:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.981 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2023-07-21T22:37:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.982 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2023-07-21T22:37:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.983 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2023-07-21T22:37:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 48040, 2023-07-21T22:37:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 156.735744471, 2023-07-21T22:37:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003262609 2023-07-21T22:37:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.984 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2023-07-21T22:37:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3274, 2023-07-21T22:37:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 9.935514327, 2023-07-21T22:37:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003034671 2023-07-21T22:37:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.985 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2023-07-21T22:37:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2023-07-21T22:37:38.986 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.987 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.987 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.987 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.987 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2023-07-21T22:37:38.987 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.988 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.988 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.988 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.988 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2023-07-21T22:37:38.988 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.989 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.989 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.989 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.989 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2023-07-21T22:37:38.989 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.990 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.990 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.990 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.990 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2023-07-21T22:37:38.990 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 193, 2023-07-21T22:37:38.991 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.296109847, 2023-07-21T22:37:38.991 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001534247 2023-07-21T22:37:38.991 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.991 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2023-07-21T22:37:38.991 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.991 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.992 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.992 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.992 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2023-07-21T22:37:38.992 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 532, 2023-07-21T22:37:38.992 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.676709112, 2023-07-21T22:37:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001272009 2023-07-21T22:37:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2023-07-21T22:37:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.993 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.994 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.994 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.994 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2023-07-21T22:37:38.994 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.994 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2023-07-21T22:37:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.995 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2023-07-21T22:37:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 14833, 2023-07-21T22:37:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 64.086954451, 2023-07-21T22:37:38.996 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004320565 2023-07-21T22:37:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:37:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:37:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2023-07-21T22:37:38.997 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:38.998 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2023-07-21T22:37:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2023-07-21T22:37:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 4, 2023-07-21T22:37:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 3, 2023-07-21T22:37:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2023-07-21T22:37:38.999 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2023-07-21T22:37:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 26404, 2023-07-21T22:37:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 26404, 2023-07-21T22:37:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 986 2023-07-21T22:37:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.000 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2023-07-21T22:37:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:39.001 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:39.002 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:39.003 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:39.004 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:39.005 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:39.005 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:39.005 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:39.005 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:39.005 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:39.006 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2023-07-21T22:37:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:39.007 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 361879, 2023-07-21T22:37:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 678, 2023-07-21T22:37:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 616, 2023-07-21T22:37:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 7, 2023-07-21T22:37:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2023-07-21T22:37:39.008 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2023-07-21T22:37:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:39.009 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 9608, 2023-07-21T22:37:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 184, 2023-07-21T22:37:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:39.010 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2023-07-21T22:37:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 3881858, 2023-07-21T22:37:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 39248, 2023-07-21T22:37:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2023-07-21T22:37:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:39.011 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2023-07-21T22:37:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2023-07-21T22:37:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2023-07-21T22:37:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2023-07-21T22:37:39.012 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 54379, 2023-07-21T22:37:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 296935529, 2023-07-21T22:37:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 33, 2023-07-21T22:37:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 54379, 2023-07-21T22:37:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2023-07-21T22:37:39.013 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 54379, 2023-07-21T22:37:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 445.509999758, 2023-07-21T22:37:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.008192684 2023-07-21T22:37:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2023-07-21T22:37:39.014 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2023-07-21T22:37:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 54379, 2023-07-21T22:37:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 54838 2023-07-21T22:37:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 54379, 2023-07-21T22:37:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 1108, 2023-07-21T22:37:39.015 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 53271, 2023-07-21T22:37:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2023-07-21T22:37:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2023-07-21T22:37:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2023-07-21T22:37:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 10, 2023-07-21T22:37:39.016 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 608, 2023-07-21T22:37:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 22829, 2023-07-21T22:37:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 349, 2023-07-21T22:37:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2023-07-21T22:37:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2023-07-21T22:37:39.017 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2023-07-21T22:37:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 9, 2023-07-21T22:37:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 29803, 2023-07-21T22:37:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2023-07-21T22:37:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2023-07-21T22:37:39.018 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2023-07-21T22:37:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 486, 2023-07-21T22:37:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 20, 2023-07-21T22:37:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:37:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2023-07-21T22:37:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:37:39.019 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2023-07-21T22:37:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2023-07-21T22:37:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2023-07-21T22:37:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:37:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2023-07-21T22:37:39.020 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:37:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2023-07-21T22:37:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2023-07-21T22:37:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2023-07-21T22:37:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2023-07-21T22:37:39.021 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2023-07-21T22:37:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2023-07-21T22:37:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2023-07-21T22:37:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2023-07-21T22:37:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2023-07-21T22:37:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2023-07-21T22:37:39.022 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2023-07-21T22:37:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2023-07-21T22:37:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2023-07-21T22:37:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2023-07-21T22:37:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2023-07-21T22:37:39.023 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2023-07-21T22:37:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2023-07-21T22:37:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 114, 2023-07-21T22:37:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2023-07-21T22:37:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 25, 2023-07-21T22:37:39.024 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2023-07-21T22:37:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2023-07-21T22:37:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2023-07-21T22:37:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2023-07-21T22:37:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 522, 2023-07-21T22:37:39.025 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 28, 2023-07-21T22:37:39.026 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 174 2023-07-21T22:37:39.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.026 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2023-07-21T22:37:39.026 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2023-07-21T22:37:39.026 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:37:39.026 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2023-07-21T22:37:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2023-07-21T22:37:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2023-07-21T22:37:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 111357 2023-07-21T22:37:39.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2023-07-21T22:37:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2023-07-21T22:37:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 16, 2023-07-21T22:37:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2023-07-21T22:37:39.028 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 8, 2023-07-21T22:37:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 14833, 2023-07-21T22:37:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2023-07-21T22:37:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:39.029 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 2160887, 2023-07-21T22:37:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 216631433, 2023-07-21T22:37:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:39.030 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 2160887, 2023-07-21T22:37:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2160887, 2023-07-21T22:37:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 216631433, 2023-07-21T22:37:39.031 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:39.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2023-07-21T22:37:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:39.033 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 54379, 2023-07-21T22:37:39.034 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 298554838, 2023-07-21T22:37:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 54379, 2023-07-21T22:37:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 298554838, 2023-07-21T22:37:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:39.035 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2023-07-21T22:37:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:39.036 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2023-07-21T22:37:39.037 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:39.037 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2023-07-21T22:37:39.037 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2023-07-21T22:37:39.037 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:39.037 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 54379, 2023-07-21T22:37:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 54379, 2023-07-21T22:37:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 54379, 2023-07-21T22:37:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 54379, 2023-07-21T22:37:39.038 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:39.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:37:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2023-07-21T22:37:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:39.040 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 14833, 2023-07-21T22:37:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1619533, 2023-07-21T22:37:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 14833, 2023-07-21T22:37:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:39.041 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 608, 2023-07-21T22:37:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1619533, 2023-07-21T22:37:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:39.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55f57cb340a0": { 2023-07-21T22:37:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 1747, 2023-07-21T22:37:39.043 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2023-07-21T22:37:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 88264, 2023-07-21T22:37:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 293377707, 2023-07-21T22:37:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:39.044 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 88264, 2023-07-21T22:37:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 22221, 2023-07-21T22:37:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 293375960, 2023-07-21T22:37:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:39.045 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:39.046 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:39.046 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-21T22:37:39.268 INFO:tasks.check_counter:Daemon mds.h mds.exported=18 2023-07-21T22:37:39.269 INFO:tasks.check_counter:Daemon mds.h mds.imported=15 2023-07-21T22:37:39.269 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-07-21T22:37:39.269 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:37:39.269 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:37:39.270 DEBUG:tasks.check_counter:Getting stats from k 2023-07-21T22:37:39.270 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-21T22:37:39.820 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T22:37:40.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:40 smithi104 ceph-mon[129021]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 740 KiB/s wr, 37 op/s 2023-07-21T22:37:40.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:40 smithi149 ceph-mon[129268]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 740 KiB/s wr, 37 op/s 2023-07-21T22:37:40.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:40 smithi066 ceph-mon[126379]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 740 KiB/s wr, 37 op/s 2023-07-21T22:37:41.160 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2023-07-21T22:37:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 0, 2023-07-21T22:37:41.161 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2, 2023-07-21T22:37:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:37:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 0, 2023-07-21T22:37:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2023-07-21T22:37:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2023-07-21T22:37:41.162 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.001043654, 2023-07-21T22:37:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:37:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000190453, 2023-07-21T22:37:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:41.163 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:41.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:37:41.165 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:37:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 317, 2023-07-21T22:37:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 306, 2023-07-21T22:37:41.166 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1584576, 2023-07-21T22:37:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 174912, 2023-07-21T22:37:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2023-07-21T22:37:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2023-07-21T22:37:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.048626307, 2023-07-21T22:37:41.167 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.014489490, 2023-07-21T22:37:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.035121258, 2023-07-21T22:37:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.007413427, 2023-07-21T22:37:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 305, 2023-07-21T22:37:41.168 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012942953, 2023-07-21T22:37:41.169 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000042435 2023-07-21T22:37:41.170 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:41.171 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1584576, 2023-07-21T22:37:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 174912 2023-07-21T22:37:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.172 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 6, 2023-07-21T22:37:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 235, 2023-07-21T22:37:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 3264, 2023-07-21T22:37:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 192768, 2023-07-21T22:37:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2023-07-21T22:37:41.173 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.012619398, 2023-07-21T22:37:41.174 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.007902978, 2023-07-21T22:37:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.002266616, 2023-07-21T22:37:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 233, 2023-07-21T22:37:41.175 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018683183, 2023-07-21T22:37:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000080185 2023-07-21T22:37:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:41.176 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-21T22:37:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 192768 2023-07-21T22:37:41.177 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2023-07-21T22:37:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2023-07-21T22:37:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2023-07-21T22:37:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.178 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2023-07-21T22:37:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:41.179 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:41.180 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:41.181 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:41.182 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:41.184 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:41.184 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:41.184 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:41.184 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:41.184 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:41.185 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:41.185 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2023-07-21T22:37:41.185 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:41.185 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:41.185 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:41.185 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:41.186 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148995, 2023-07-21T22:37:41.186 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 31, 2023-07-21T22:37:41.186 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:41.186 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:41.186 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2023-07-21T22:37:41.187 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2023-07-21T22:37:41.187 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:41.187 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:41.187 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:41.187 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:41.188 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2023-07-21T22:37:41.188 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2023-07-21T22:37:41.188 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:41.188 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:41.188 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:41.189 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2023-07-21T22:37:41.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2023-07-21T22:37:41.189 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2023-07-21T22:37:41.189 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:41.189 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2023-07-21T22:37:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2023-07-21T22:37:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2023-07-21T22:37:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2023-07-21T22:37:41.190 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2023-07-21T22:37:41.191 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2023-07-21T22:37:41.191 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 319, 2023-07-21T22:37:41.191 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1539791, 2023-07-21T22:37:41.191 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:41.191 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 319, 2023-07-21T22:37:41.192 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2023-07-21T22:37:41.192 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2023-07-21T22:37:41.192 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 319, 2023-07-21T22:37:41.192 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1539791, 2023-07-21T22:37:41.192 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2023-07-21T22:37:41.193 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2023-07-21T22:37:41.193 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2023-07-21T22:37:41.193 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2023-07-21T22:37:41.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:41.193 INFO:teuthology.orchestra.run.smithi104.stdout: } 2023-07-21T22:37:41.193 INFO:teuthology.orchestra.run.smithi104.stdout:} 2023-07-21T22:37:41.473 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:41 smithi104 ceph-mon[129021]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 757 KiB/s wr, 41 op/s 2023-07-21T22:37:41.474 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-07-21T22:37:41.475 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-07-21T22:37:41.475 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-07-21T22:37:41.475 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-07-21T22:37:41.475 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-07-21T22:37:41.476 DEBUG:tasks.check_counter:Getting stats from c 2023-07-21T22:37:41.476 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-21T22:37:41.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:41 smithi149 ceph-mon[129268]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 757 KiB/s wr, 41 op/s 2023-07-21T22:37:41.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:41 smithi066 ceph-mon[126379]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 757 KiB/s wr, 41 op/s 2023-07-21T22:37:42.146 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:37:42.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:42 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:42.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:42 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:42.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:42 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:43.945 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-07-21T22:37:43.945 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:43.945 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 2540, 2023-07-21T22:37:43.946 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2500, 2023-07-21T22:37:43.946 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 224683868, 2023-07-21T22:37:43.946 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 852158, 2023-07-21T22:37:43.946 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2023-07-21T22:37:43.946 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:43.947 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.352404997, 2023-07-21T22:37:43.947 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.066129675, 2023-07-21T22:37:43.947 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.201756813, 2023-07-21T22:37:43.947 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.075746765, 2023-07-21T22:37:43.947 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:43.948 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2499, 2023-07-21T22:37:43.948 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.094159751, 2023-07-21T22:37:43.948 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000037678 2023-07-21T22:37:43.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.949 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:43.949 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.949 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.949 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.950 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1740576, 2023-07-21T22:37:43.950 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 178528 2023-07-21T22:37:43.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.950 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:43.951 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1138, 2023-07-21T22:37:43.951 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1365, 2023-07-21T22:37:43.951 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 243581684, 2023-07-21T22:37:43.951 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 897366, 2023-07-21T22:37:43.951 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2023-07-21T22:37:43.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2023-07-21T22:37:43.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.201759748, 2023-07-21T22:37:43.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.033187783, 2023-07-21T22:37:43.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.129439240, 2023-07-21T22:37:43.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.028893053, 2023-07-21T22:37:43.952 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:43.953 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1363, 2023-07-21T22:37:43.953 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.077592400, 2023-07-21T22:37:43.953 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000056927 2023-07-21T22:37:43.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.953 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:43.954 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.954 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.954 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.954 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-21T22:37:43.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 545248 2023-07-21T22:37:43.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.955 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:43.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1346, 2023-07-21T22:37:43.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1348, 2023-07-21T22:37:43.955 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 277479018, 2023-07-21T22:37:43.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 418613, 2023-07-21T22:37:43.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:43.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2023-07-21T22:37:43.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.222928217, 2023-07-21T22:37:43.956 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.030592772, 2023-07-21T22:37:43.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.147868467, 2023-07-21T22:37:43.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.035108095, 2023-07-21T22:37:43.957 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:43.957 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1346, 2023-07-21T22:37:43.957 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.048891095, 2023-07-21T22:37:43.958 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000036323 2023-07-21T22:37:43.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.958 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:43.958 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.958 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.958 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.959 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:37:43.959 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:37:43.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.959 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-07-21T22:37:43.960 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2023-07-21T22:37:43.960 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-07-21T22:37:43.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.960 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2023-07-21T22:37:43.960 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-07-21T22:37:43.961 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-07-21T22:37:43.961 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7479, 2023-07-21T22:37:43.961 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.262457007, 2023-07-21T22:37:43.961 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000035092 2023-07-21T22:37:43.961 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:43.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.962 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2023-07-21T22:37:43.962 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-07-21T22:37:43.962 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-07-21T22:37:43.962 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.962 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.963 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.963 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:43.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.963 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2023-07-21T22:37:43.963 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2023-07-21T22:37:43.964 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2023-07-21T22:37:43.964 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2023-07-21T22:37:43.964 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.964 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.964 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.965 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2023-07-21T22:37:43.965 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2023-07-21T22:37:43.965 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2023-07-21T22:37:43.965 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2023-07-21T22:37:43.966 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2023-07-21T22:37:43.966 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2023-07-21T22:37:43.966 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2023-07-21T22:37:43.966 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 15154, 2023-07-21T22:37:43.966 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 123, 2023-07-21T22:37:43.966 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 15012, 2023-07-21T22:37:43.967 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 19, 2023-07-21T22:37:43.967 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 14977, 2023-07-21T22:37:43.967 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 5321, 2023-07-21T22:37:43.967 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2023-07-21T22:37:43.967 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2023-07-21T22:37:43.968 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 10, 2023-07-21T22:37:43.968 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2023-07-21T22:37:43.968 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2023-07-21T22:37:43.968 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2023-07-21T22:37:43.968 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2023-07-21T22:37:43.969 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:37:43.969 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2023-07-21T22:37:43.969 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2023-07-21T22:37:43.969 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2023-07-21T22:37:43.969 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2023-07-21T22:37:43.969 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2023-07-21T22:37:43.970 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2023-07-21T22:37:43.970 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2023-07-21T22:37:43.970 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2023-07-21T22:37:43.970 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2023-07-21T22:37:43.970 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:37:43.971 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2023-07-21T22:37:43.971 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 725, 2023-07-21T22:37:43.971 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 218096948, 2023-07-21T22:37:43.971 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2023-07-21T22:37:43.971 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:37:43.972 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2023-07-21T22:37:43.972 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:37:43.972 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:37:43.972 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:37:43.972 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:37:43.973 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:37:43.973 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2023-07-21T22:37:43.973 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:37:43.973 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:37:43.973 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:37:43.973 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:37:43.974 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:37:43.974 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:37:43.974 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2023-07-21T22:37:43.974 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:37:43.974 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2023-07-21T22:37:43.975 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2023-07-21T22:37:43.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.975 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2023-07-21T22:37:43.975 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2023-07-21T22:37:43.975 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2023-07-21T22:37:43.976 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2023-07-21T22:37:43.976 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2023-07-21T22:37:43.976 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2023-07-21T22:37:43.976 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2023-07-21T22:37:43.976 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2023-07-21T22:37:43.977 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:37:43.977 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2023-07-21T22:37:43.977 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2023-07-21T22:37:43.977 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2023-07-21T22:37:43.977 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2023-07-21T22:37:43.977 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2023-07-21T22:37:43.978 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:37:43.978 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:37:43.978 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2023-07-21T22:37:43.978 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2023-07-21T22:37:43.978 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:37:43.979 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2023-07-21T22:37:43.979 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2023-07-21T22:37:43.979 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:37:43.979 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2023-07-21T22:37:43.979 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2023-07-21T22:37:43.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.980 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2023-07-21T22:37:43.980 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2023-07-21T22:37:43.980 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2023-07-21T22:37:43.980 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2023-07-21T22:37:43.981 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 0, 2023-07-21T22:37:43.981 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2023-07-21T22:37:43.981 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2023-07-21T22:37:43.981 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2023-07-21T22:37:43.981 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2023-07-21T22:37:43.982 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2023-07-21T22:37:43.982 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 129, 2023-07-21T22:37:43.982 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2023-07-21T22:37:43.982 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2023-07-21T22:37:43.982 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 247479150, 2023-07-21T22:37:43.983 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 0, 2023-07-21T22:37:43.983 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 746885457, 2023-07-21T22:37:43.983 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2023-07-21T22:37:43.983 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.983 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.984 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.984 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 113065 2023-07-21T22:37:43.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.984 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2023-07-21T22:37:43.985 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 661, 2023-07-21T22:37:43.985 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 21396, 2023-07-21T22:37:43.985 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 20735, 2023-07-21T22:37:43.985 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 82, 2023-07-21T22:37:43.985 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 225, 2023-07-21T22:37:43.985 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 143, 2023-07-21T22:37:43.986 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 15154, 2023-07-21T22:37:43.986 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 22201, 2023-07-21T22:37:43.986 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 7047, 2023-07-21T22:37:43.986 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2023-07-21T22:37:43.986 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2023-07-21T22:37:43.987 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2023-07-21T22:37:43.987 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 72892, 2023-07-21T22:37:43.987 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207156 2023-07-21T22:37:43.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.987 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2023-07-21T22:37:43.987 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2023-07-21T22:37:43.988 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2023-07-21T22:37:43.988 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2023-07-21T22:37:43.988 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2023-07-21T22:37:43.988 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2023-07-21T22:37:43.988 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2023-07-21T22:37:43.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.989 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.989 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.989 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2023-07-21T22:37:43.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.990 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.990 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.990 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2023-07-21T22:37:43.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.991 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.991 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2023-07-21T22:37:43.991 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.992 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.992 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.993 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2023-07-21T22:37:43.993 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.993 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.993 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.993 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2023-07-21T22:37:43.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.994 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.994 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.994 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2023-07-21T22:37:43.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.995 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.995 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.995 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2023-07-21T22:37:43.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.996 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.996 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.996 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2023-07-21T22:37:43.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.997 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.997 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.997 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2023-07-21T22:37:43.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.998 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.998 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.998 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2023-07-21T22:37:43.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:43.999 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:43.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:43.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:43.999 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2023-07-21T22:37:43.999 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.000 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.000 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.000 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2023-07-21T22:37:44.000 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.001 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.001 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.001 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2023-07-21T22:37:44.001 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.002 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.002 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.002 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2023-07-21T22:37:44.002 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.003 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.003 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.003 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2023-07-21T22:37:44.003 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.004 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.004 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.004 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2023-07-21T22:37:44.004 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.004 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.005 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.005 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2023-07-21T22:37:44.005 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.005 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.006 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.006 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2023-07-21T22:37:44.006 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.006 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.007 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.007 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2023-07-21T22:37:44.007 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.007 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2023-07-21T22:37:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.008 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.009 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2023-07-21T22:37:44.009 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.009 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.009 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.010 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2023-07-21T22:37:44.010 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.010 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.010 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.011 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2023-07-21T22:37:44.011 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.011 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.011 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.012 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2023-07-21T22:37:44.012 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.012 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.012 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2023-07-21T22:37:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.014 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2023-07-21T22:37:44.014 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.014 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.014 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.015 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2023-07-21T22:37:44.015 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.015 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.015 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.016 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2023-07-21T22:37:44.016 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.016 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.016 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.017 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:37:44.017 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:37:44.017 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2023-07-21T22:37:44.017 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.017 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.018 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.018 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.018 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2023-07-21T22:37:44.018 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2023-07-21T22:37:44.018 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 6, 2023-07-21T22:37:44.019 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 4, 2023-07-21T22:37:44.019 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 4, 2023-07-21T22:37:44.019 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2023-07-21T22:37:44.019 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2023-07-21T22:37:44.019 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2023-07-21T22:37:44.020 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 5772 2023-07-21T22:37:44.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.020 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-07-21T22:37:44.020 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:44.020 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:44.021 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:44.021 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:44.021 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:44.021 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:44.021 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:44.022 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:44.022 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:44.022 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:44.022 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:44.022 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:44.022 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:44.023 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:44.023 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:44.023 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:44.023 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:44.023 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:44.024 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:44.024 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:44.024 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:44.024 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:44.024 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:44.025 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:44.025 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:44.025 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:44.025 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:44.025 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:44.026 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:44.026 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:44.026 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:44.026 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-07-21T22:37:44.026 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:44.026 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:44.027 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:44.027 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:44.027 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 267463, 2023-07-21T22:37:44.027 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 903, 2023-07-21T22:37:44.027 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:44.028 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:44.028 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-07-21T22:37:44.028 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-07-21T22:37:44.028 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:44.028 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:44.029 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:44.029 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:44.029 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9608, 2023-07-21T22:37:44.029 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 184, 2023-07-21T22:37:44.029 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:44.030 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:44.030 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:44.030 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-07-21T22:37:44.030 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 12622257, 2023-07-21T22:37:44.030 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 80397, 2023-07-21T22:37:44.031 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:44.031 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-07-21T22:37:44.031 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:44.031 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-07-21T22:37:44.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.031 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2023-07-21T22:37:44.032 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2023-07-21T22:37:44.032 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2023-07-21T22:37:44.032 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 4644, 2023-07-21T22:37:44.032 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 26, 2023-07-21T22:37:44.032 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 4, 2023-07-21T22:37:44.033 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 4644, 2023-07-21T22:37:44.033 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2023-07-21T22:37:44.033 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4644, 2023-07-21T22:37:44.033 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 7.100859029, 2023-07-21T22:37:44.033 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001529039 2023-07-21T22:37:44.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.034 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2023-07-21T22:37:44.034 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2023-07-21T22:37:44.034 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4644, 2023-07-21T22:37:44.034 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4645 2023-07-21T22:37:44.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.035 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 4644, 2023-07-21T22:37:44.035 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 4644, 2023-07-21T22:37:44.035 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2023-07-21T22:37:44.035 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2023-07-21T22:37:44.035 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2023-07-21T22:37:44.036 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 2639, 2023-07-21T22:37:44.036 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2023-07-21T22:37:44.036 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 2002, 2023-07-21T22:37:44.036 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2023-07-21T22:37:44.036 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2023-07-21T22:37:44.037 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2023-07-21T22:37:44.037 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2023-07-21T22:37:44.037 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2023-07-21T22:37:44.037 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2023-07-21T22:37:44.037 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2023-07-21T22:37:44.038 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2023-07-21T22:37:44.038 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2023-07-21T22:37:44.038 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2023-07-21T22:37:44.038 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 2, 2023-07-21T22:37:44.038 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2023-07-21T22:37:44.038 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:37:44.039 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2023-07-21T22:37:44.039 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:37:44.039 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2023-07-21T22:37:44.039 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2023-07-21T22:37:44.039 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2023-07-21T22:37:44.040 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:37:44.040 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2023-07-21T22:37:44.040 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:37:44.040 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2023-07-21T22:37:44.040 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2023-07-21T22:37:44.041 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2023-07-21T22:37:44.041 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2023-07-21T22:37:44.041 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2023-07-21T22:37:44.041 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2023-07-21T22:37:44.041 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2023-07-21T22:37:44.042 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2023-07-21T22:37:44.042 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2023-07-21T22:37:44.042 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2023-07-21T22:37:44.042 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2023-07-21T22:37:44.042 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2023-07-21T22:37:44.042 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2023-07-21T22:37:44.043 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2023-07-21T22:37:44.043 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2023-07-21T22:37:44.043 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2023-07-21T22:37:44.043 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2023-07-21T22:37:44.043 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 114, 2023-07-21T22:37:44.044 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2023-07-21T22:37:44.044 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 25, 2023-07-21T22:37:44.044 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 11, 2023-07-21T22:37:44.044 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2023-07-21T22:37:44.044 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2023-07-21T22:37:44.045 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2023-07-21T22:37:44.045 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2023-07-21T22:37:44.045 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2023-07-21T22:37:44.045 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2023-07-21T22:37:44.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.046 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2023-07-21T22:37:44.046 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2023-07-21T22:37:44.046 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:37:44.046 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2023-07-21T22:37:44.046 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2023-07-21T22:37:44.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.047 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2023-07-21T22:37:44.047 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2023-07-21T22:37:44.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.047 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2023-07-21T22:37:44.047 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2023-07-21T22:37:44.048 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:37:44.048 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2023-07-21T22:37:44.048 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2023-07-21T22:37:44.048 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2023-07-21T22:37:44.048 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2023-07-21T22:37:44.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.049 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:44.049 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:44.049 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:44.049 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:44.050 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 5020, 2023-07-21T22:37:44.050 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 745270748, 2023-07-21T22:37:44.050 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:44.050 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 5020, 2023-07-21T22:37:44.050 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:44.050 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:44.051 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 5020, 2023-07-21T22:37:44.051 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 745270748, 2023-07-21T22:37:44.051 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:44.051 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.051 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.052 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.052 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.052 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2023-07-21T22:37:44.052 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:44.053 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:44.053 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:44.053 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:44.053 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:44.053 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:44.053 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:44.054 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4644, 2023-07-21T22:37:44.054 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 742726294, 2023-07-21T22:37:44.054 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4644, 2023-07-21T22:37:44.054 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 742726294, 2023-07-21T22:37:44.054 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:44.055 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.055 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.055 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.055 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.056 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2023-07-21T22:37:44.056 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:44.056 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2023-07-21T22:37:44.056 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:44.056 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4644, 2023-07-21T22:37:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 4644, 2023-07-21T22:37:44.058 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4644, 2023-07-21T22:37:44.058 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 4644, 2023-07-21T22:37:44.058 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:44.058 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.058 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.058 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.059 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.059 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.059 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:37:44.059 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:44.059 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-07-21T22:37:44.060 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:44.060 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:44.060 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:44.060 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:44.060 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:44.061 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:44.061 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:44.061 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2023-07-21T22:37:44.061 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2023-07-21T22:37:44.061 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:44.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.062 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.062 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.062 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:44.062 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55fd109c6460": { 2023-07-21T22:37:44.063 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:44.063 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-07-21T22:37:44.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:44.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:44.063 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:44.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:44.064 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:44.064 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:44.064 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:44.064 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2023-07-21T22:37:44.065 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2023-07-21T22:37:44.065 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:44.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:44.065 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:44.065 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:44.065 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.066 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:44.066 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-07-21T22:37:44.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:44 smithi149 ceph-mon[129268]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-07-21T22:37:44.397 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-21T22:37:44.398 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-21T22:37:44.398 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-07-21T22:37:44.398 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:37:44.398 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:37:44.399 DEBUG:tasks.check_counter:Getting stats from f 2023-07-21T22:37:44.399 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-21T22:37:44.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:44 smithi066 ceph-mon[126379]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-07-21T22:37:44.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:44 smithi104 ceph-mon[129021]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 9 op/s 2023-07-21T22:37:44.996 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:37:46.346 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-07-21T22:37:46.347 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:46.347 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1061, 2023-07-21T22:37:46.347 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1063, 2023-07-21T22:37:46.347 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 138693758, 2023-07-21T22:37:46.348 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 330035, 2023-07-21T22:37:46.348 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 8, 2023-07-21T22:37:46.348 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2023-07-21T22:37:46.348 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.163660810, 2023-07-21T22:37:46.349 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.022865246, 2023-07-21T22:37:46.349 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.101414789, 2023-07-21T22:37:46.349 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.030195623, 2023-07-21T22:37:46.349 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:46.349 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1061, 2023-07-21T22:37:46.349 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.049717201, 2023-07-21T22:37:46.350 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000046858 2023-07-21T22:37:46.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.350 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:46.350 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.350 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.351 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.351 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:37:46.351 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:37:46.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.352 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:46.352 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1727, 2023-07-21T22:37:46.352 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1687, 2023-07-21T22:37:46.352 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 139689151, 2023-07-21T22:37:46.352 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 600029, 2023-07-21T22:37:46.353 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 6, 2023-07-21T22:37:46.353 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2023-07-21T22:37:46.353 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.220823719, 2023-07-21T22:37:46.353 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.048366398, 2023-07-21T22:37:46.353 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.126895913, 2023-07-21T22:37:46.354 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.046333353, 2023-07-21T22:37:46.354 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:46.354 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1686, 2023-07-21T22:37:46.354 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.070789033, 2023-07-21T22:37:46.354 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000041986 2023-07-21T22:37:46.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.355 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:46.355 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.355 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.355 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.356 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1740960, 2023-07-21T22:37:46.356 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 179552 2023-07-21T22:37:46.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.356 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:46.357 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1690, 2023-07-21T22:37:46.357 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1918, 2023-07-21T22:37:46.357 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 80218750, 2023-07-21T22:37:46.357 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 1070700, 2023-07-21T22:37:46.357 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 8, 2023-07-21T22:37:46.358 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2023-07-21T22:37:46.358 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.210408911, 2023-07-21T22:37:46.358 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.048923826, 2023-07-21T22:37:46.358 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.087183205, 2023-07-21T22:37:46.358 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.057273547, 2023-07-21T22:37:46.358 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:46.359 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1916, 2023-07-21T22:37:46.359 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.106125083, 2023-07-21T22:37:46.359 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000055388 2023-07-21T22:37:46.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.359 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:46.360 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.360 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.360 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.360 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-21T22:37:46.361 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 546976 2023-07-21T22:37:46.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.361 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-07-21T22:37:46.361 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2023-07-21T22:37:46.361 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-07-21T22:37:46.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.362 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2023-07-21T22:37:46.362 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-07-21T22:37:46.362 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-07-21T22:37:46.362 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6779, 2023-07-21T22:37:46.363 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.178518948, 2023-07-21T22:37:46.363 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000026334 2023-07-21T22:37:46.363 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.363 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2023-07-21T22:37:46.363 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-07-21T22:37:46.364 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-07-21T22:37:46.364 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.364 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.364 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.364 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.365 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2023-07-21T22:37:46.365 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 0, 2023-07-21T22:37:46.365 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2023-07-21T22:37:46.365 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2023-07-21T22:37:46.366 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.366 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.366 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.366 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2023-07-21T22:37:46.367 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2023-07-21T22:37:46.367 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2023-07-21T22:37:46.367 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2023-07-21T22:37:46.367 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 0, 2023-07-21T22:37:46.367 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2023-07-21T22:37:46.368 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2023-07-21T22:37:46.368 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 23041, 2023-07-21T22:37:46.368 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 548, 2023-07-21T22:37:46.368 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 11645, 2023-07-21T22:37:46.368 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 10848, 2023-07-21T22:37:46.369 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 22258, 2023-07-21T22:37:46.369 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 225, 2023-07-21T22:37:46.369 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2023-07-21T22:37:46.369 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2023-07-21T22:37:46.369 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 8, 2023-07-21T22:37:46.370 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 0, 2023-07-21T22:37:46.370 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 0, 2023-07-21T22:37:46.370 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2023-07-21T22:37:46.370 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2023-07-21T22:37:46.370 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:37:46.370 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2023-07-21T22:37:46.371 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2023-07-21T22:37:46.371 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2023-07-21T22:37:46.371 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2023-07-21T22:37:46.371 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2023-07-21T22:37:46.371 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2023-07-21T22:37:46.372 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2023-07-21T22:37:46.372 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2023-07-21T22:37:46.372 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2023-07-21T22:37:46.372 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2023-07-21T22:37:46.372 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2023-07-21T22:37:46.373 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 0, 2023-07-21T22:37:46.373 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 0, 2023-07-21T22:37:46.373 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 0, 2023-07-21T22:37:46.373 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T22:37:46.373 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2023-07-21T22:37:46.374 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:37:46.374 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:37:46.374 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2023-07-21T22:37:46.374 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:37:46.374 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T22:37:46.375 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2023-07-21T22:37:46.375 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2023-07-21T22:37:46.375 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T22:37:46.375 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2023-07-21T22:37:46.375 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:37:46.375 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T22:37:46.376 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:37:46.376 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2023-07-21T22:37:46.376 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2023-07-21T22:37:46.376 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2023-07-21T22:37:46.377 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2023-07-21T22:37:46.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.377 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2023-07-21T22:37:46.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2023-07-21T22:37:46.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2023-07-21T22:37:46.377 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2023-07-21T22:37:46.378 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2023-07-21T22:37:46.378 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2023-07-21T22:37:46.378 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2023-07-21T22:37:46.378 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2023-07-21T22:37:46.378 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:37:46.379 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2023-07-21T22:37:46.379 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2023-07-21T22:37:46.379 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2023-07-21T22:37:46.379 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2023-07-21T22:37:46.379 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2023-07-21T22:37:46.380 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:37:46.380 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:37:46.380 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2023-07-21T22:37:46.380 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2023-07-21T22:37:46.380 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:37:46.381 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2023-07-21T22:37:46.381 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2023-07-21T22:37:46.381 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2023-07-21T22:37:46.381 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2023-07-21T22:37:46.381 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2023-07-21T22:37:46.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.382 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2023-07-21T22:37:46.382 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 0, 2023-07-21T22:37:46.382 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2023-07-21T22:37:46.382 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2023-07-21T22:37:46.382 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 0, 2023-07-21T22:37:46.383 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2023-07-21T22:37:46.383 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2023-07-21T22:37:46.383 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 0, 2023-07-21T22:37:46.383 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2023-07-21T22:37:46.383 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2023-07-21T22:37:46.384 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 129, 2023-07-21T22:37:46.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2023-07-21T22:37:46.384 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2023-07-21T22:37:46.384 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 11338273, 2023-07-21T22:37:46.385 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 0, 2023-07-21T22:37:46.385 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 359898584, 2023-07-21T22:37:46.385 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2023-07-21T22:37:46.385 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.386 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.386 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.386 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 104752 2023-07-21T22:37:46.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2023-07-21T22:37:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 1293, 2023-07-21T22:37:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 23345, 2023-07-21T22:37:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 22052, 2023-07-21T22:37:46.387 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 87, 2023-07-21T22:37:46.388 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 140, 2023-07-21T22:37:46.388 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 53, 2023-07-21T22:37:46.388 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 23041, 2023-07-21T22:37:46.388 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 23788, 2023-07-21T22:37:46.388 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 747, 2023-07-21T22:37:46.389 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2023-07-21T22:37:46.389 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2023-07-21T22:37:46.389 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2023-07-21T22:37:46.389 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 76896, 2023-07-21T22:37:46.389 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207156 2023-07-21T22:37:46.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.390 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2023-07-21T22:37:46.390 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 0, 2023-07-21T22:37:46.390 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2023-07-21T22:37:46.390 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 0, 2023-07-21T22:37:46.391 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 0, 2023-07-21T22:37:46.391 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2023-07-21T22:37:46.391 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2023-07-21T22:37:46.391 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.391 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.391 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.392 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2023-07-21T22:37:46.392 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.392 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.393 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2023-07-21T22:37:46.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.393 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.393 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.394 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2023-07-21T22:37:46.394 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.394 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.394 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.395 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2023-07-21T22:37:46.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.395 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.395 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.396 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2023-07-21T22:37:46.396 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.396 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.396 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.397 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2023-07-21T22:37:46.397 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.397 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.397 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.398 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2023-07-21T22:37:46.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.398 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.398 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.399 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2023-07-21T22:37:46.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.399 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.399 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.400 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2023-07-21T22:37:46.400 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.400 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.400 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.401 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2023-07-21T22:37:46.401 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.401 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.401 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.402 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2023-07-21T22:37:46.402 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.402 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.402 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.403 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2023-07-21T22:37:46.403 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.404 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.404 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.405 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2023-07-21T22:37:46.405 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.405 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.406 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.406 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2023-07-21T22:37:46.407 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.407 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.407 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.408 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2023-07-21T22:37:46.408 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.408 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.409 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.409 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2023-07-21T22:37:46.410 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.410 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.410 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.411 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2023-07-21T22:37:46.411 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.412 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.412 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.413 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2023-07-21T22:37:46.413 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.413 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.414 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.414 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2023-07-21T22:37:46.415 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.415 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.415 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.416 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2023-07-21T22:37:46.417 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.417 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.417 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.417 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2023-07-21T22:37:46.418 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.418 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.418 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.418 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2023-07-21T22:37:46.419 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.419 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.420 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.420 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2023-07-21T22:37:46.420 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.420 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.421 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.421 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2023-07-21T22:37:46.421 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.421 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.422 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.422 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2023-07-21T22:37:46.422 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.422 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.423 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.423 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2023-07-21T22:37:46.423 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.423 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.424 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.424 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2023-07-21T22:37:46.424 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.424 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.425 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.425 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2023-07-21T22:37:46.425 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.425 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.426 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.426 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:37:46.426 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:37:46.426 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2023-07-21T22:37:46.427 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.427 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.427 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.427 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.428 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2023-07-21T22:37:46.428 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2023-07-21T22:37:46.428 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2023-07-21T22:37:46.428 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2023-07-21T22:37:46.428 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2023-07-21T22:37:46.429 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2023-07-21T22:37:46.429 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2023-07-21T22:37:46.429 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2023-07-21T22:37:46.429 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 5775 2023-07-21T22:37:46.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.429 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-07-21T22:37:46.430 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:46.430 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:46.430 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:46.430 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:46.430 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:46.431 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:46.431 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:46.431 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:46.431 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:46.431 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:46.432 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:46.432 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:46.432 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:46.432 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:46.432 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:46.433 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:46.433 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:46.433 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:46.433 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:46.433 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:46.433 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:46.434 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:46.434 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:46.434 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:46.434 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:46.434 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:46.435 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:46.435 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:46.435 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:46.435 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:46.435 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:46.436 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-07-21T22:37:46.436 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:46.436 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:46.436 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:46.436 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:46.437 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 291010, 2023-07-21T22:37:46.437 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 1575, 2023-07-21T22:37:46.437 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:46.437 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:46.437 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-07-21T22:37:46.438 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-07-21T22:37:46.438 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:46.438 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:46.438 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:46.438 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:46.439 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9608, 2023-07-21T22:37:46.439 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 184, 2023-07-21T22:37:46.439 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:46.439 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:46.439 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:46.439 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-07-21T22:37:46.440 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 19616553, 2023-07-21T22:37:46.440 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 127513, 2023-07-21T22:37:46.440 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:46.440 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-07-21T22:37:46.440 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:46.441 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-07-21T22:37:46.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.441 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2023-07-21T22:37:46.441 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2023-07-21T22:37:46.441 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2023-07-21T22:37:46.442 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 4097, 2023-07-21T22:37:46.442 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 21, 2023-07-21T22:37:46.442 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 3, 2023-07-21T22:37:46.442 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 4097, 2023-07-21T22:37:46.442 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2023-07-21T22:37:46.443 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4097, 2023-07-21T22:37:46.443 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 6.914863286, 2023-07-21T22:37:46.443 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001687786 2023-07-21T22:37:46.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.443 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2023-07-21T22:37:46.444 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2023-07-21T22:37:46.444 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4097, 2023-07-21T22:37:46.444 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4098 2023-07-21T22:37:46.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.444 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 4097, 2023-07-21T22:37:46.445 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 4097, 2023-07-21T22:37:46.445 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 0, 2023-07-21T22:37:46.445 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2023-07-21T22:37:46.445 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2023-07-21T22:37:46.445 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 2321, 2023-07-21T22:37:46.446 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 0, 2023-07-21T22:37:46.446 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 1774, 2023-07-21T22:37:46.446 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 0, 2023-07-21T22:37:46.446 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 0, 2023-07-21T22:37:46.446 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2023-07-21T22:37:46.447 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2023-07-21T22:37:46.447 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2023-07-21T22:37:46.447 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2023-07-21T22:37:46.447 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 0, 2023-07-21T22:37:46.447 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2023-07-21T22:37:46.447 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2023-07-21T22:37:46.448 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2023-07-21T22:37:46.448 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 1, 2023-07-21T22:37:46.448 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 0, 2023-07-21T22:37:46.448 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:37:46.448 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2023-07-21T22:37:46.449 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:37:46.449 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2023-07-21T22:37:46.449 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2023-07-21T22:37:46.449 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2023-07-21T22:37:46.449 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:37:46.450 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2023-07-21T22:37:46.450 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:37:46.450 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2023-07-21T22:37:46.450 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2023-07-21T22:37:46.450 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2023-07-21T22:37:46.451 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2023-07-21T22:37:46.451 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2023-07-21T22:37:46.451 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2023-07-21T22:37:46.451 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2023-07-21T22:37:46.451 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2023-07-21T22:37:46.451 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2023-07-21T22:37:46.452 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2023-07-21T22:37:46.452 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2023-07-21T22:37:46.452 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2023-07-21T22:37:46.452 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2023-07-21T22:37:46.452 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2023-07-21T22:37:46.453 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2023-07-21T22:37:46.453 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2023-07-21T22:37:46.453 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2023-07-21T22:37:46.453 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 114, 2023-07-21T22:37:46.453 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2023-07-21T22:37:46.454 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 25, 2023-07-21T22:37:46.454 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 11, 2023-07-21T22:37:46.454 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2023-07-21T22:37:46.454 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2023-07-21T22:37:46.454 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2023-07-21T22:37:46.455 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 0, 2023-07-21T22:37:46.455 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2023-07-21T22:37:46.455 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 0 2023-07-21T22:37:46.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.455 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2023-07-21T22:37:46.455 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 0, 2023-07-21T22:37:46.456 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:37:46.456 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2023-07-21T22:37:46.456 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2023-07-21T22:37:46.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.456 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2023-07-21T22:37:46.457 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2023-07-21T22:37:46.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.457 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2023-07-21T22:37:46.457 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2023-07-21T22:37:46.457 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T22:37:46.458 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2023-07-21T22:37:46.458 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2023-07-21T22:37:46.458 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2023-07-21T22:37:46.458 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2023-07-21T22:37:46.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.459 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:46.459 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:46.459 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:46.459 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:46.459 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 4474, 2023-07-21T22:37:46.460 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 358176919, 2023-07-21T22:37:46.460 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:46.460 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 4474, 2023-07-21T22:37:46.460 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:46.460 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:46.461 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4474, 2023-07-21T22:37:46.461 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 358176919, 2023-07-21T22:37:46.461 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:46.461 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.461 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.462 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.462 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.462 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2023-07-21T22:37:46.462 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:46.462 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:46.463 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:46.463 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:46.463 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:46.463 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:46.463 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:46.464 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4097, 2023-07-21T22:37:46.464 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 355724951, 2023-07-21T22:37:46.464 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4097, 2023-07-21T22:37:46.464 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 355724951, 2023-07-21T22:37:46.464 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:46.465 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.465 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.465 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.465 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.466 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2023-07-21T22:37:46.466 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:46.466 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2023-07-21T22:37:46.466 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:46.466 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:46.467 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:46.467 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:46.467 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:46.467 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 4097, 2023-07-21T22:37:46.467 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 4097, 2023-07-21T22:37:46.468 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 4097, 2023-07-21T22:37:46.468 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 4097, 2023-07-21T22:37:46.468 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:46.468 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.468 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.469 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.469 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:37:46.469 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:46.470 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-07-21T22:37:46.470 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:46.470 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:46.470 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:46.470 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:46.471 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:46.471 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:46.471 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:46.471 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2023-07-21T22:37:46.471 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2023-07-21T22:37:46.472 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:46.472 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.472 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.472 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.472 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:46.473 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x562288bc2460": { 2023-07-21T22:37:46.473 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:46.473 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-07-21T22:37:46.473 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:46.473 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:46.474 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:46.474 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:46.474 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:46.474 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:46.474 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:46.475 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2023-07-21T22:37:46.475 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2023-07-21T22:37:46.475 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:46.475 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:46.475 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:46.476 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:46.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:46.476 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-07-21T22:37:46.477 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:46 smithi066 ceph-mon[126379]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-07-21T22:37:46.477 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:46 smithi149 ceph-mon[129268]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-07-21T22:37:46.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:46 smithi104 ceph-mon[129021]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 62 KiB/s wr, 13 op/s 2023-07-21T22:37:46.756 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-21T22:37:46.757 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-21T22:37:46.757 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-07-21T22:37:46.757 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:37:46.757 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:37:46.758 DEBUG:tasks.check_counter:Getting stats from i 2023-07-21T22:37:46.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-21T22:37:47.322 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:37:48.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:48 smithi066 ceph-mon[126379]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 8 op/s 2023-07-21T22:37:48.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:48 smithi149 ceph-mon[129268]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 8 op/s 2023-07-21T22:37:48.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:48 smithi104 ceph-mon[129021]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 8 op/s 2023-07-21T22:37:48.762 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-07-21T22:37:48.763 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:48.763 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 35465, 2023-07-21T22:37:48.763 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 35468, 2023-07-21T22:37:48.763 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 8889394, 2023-07-21T22:37:48.763 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 180119445, 2023-07-21T22:37:48.764 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 46, 2023-07-21T22:37:48.764 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 8, 2023-07-21T22:37:48.764 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 2.664711785, 2023-07-21T22:37:48.764 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.769981673, 2023-07-21T22:37:48.764 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.516402347, 2023-07-21T22:37:48.765 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 1.221478988, 2023-07-21T22:37:48.765 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:48.765 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 35465, 2023-07-21T22:37:48.765 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 3.309787156, 2023-07-21T22:37:48.765 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000093325 2023-07-21T22:37:48.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.766 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:48.766 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 49, 2023-07-21T22:37:48.766 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000005975, 2023-07-21T22:37:48.766 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000121 2023-07-21T22:37:48.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.767 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1824, 2023-07-21T22:37:48.767 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 1616 2023-07-21T22:37:48.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.767 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:48.768 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 44270, 2023-07-21T22:37:48.768 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 48918, 2023-07-21T22:37:48.768 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 11645177, 2023-07-21T22:37:48.768 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 141430880, 2023-07-21T22:37:48.768 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 9, 2023-07-21T22:37:48.769 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2023-07-21T22:37:48.769 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 3.149804796, 2023-07-21T22:37:48.769 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 1.105551884, 2023-07-21T22:37:48.769 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 1.054670855, 2023-07-21T22:37:48.769 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.950970908, 2023-07-21T22:37:48.770 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:48.770 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 48919, 2023-07-21T22:37:48.770 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 4.232967704, 2023-07-21T22:37:48.770 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000086530 2023-07-21T22:37:48.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.771 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:48.771 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 37359, 2023-07-21T22:37:48.771 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005982088, 2023-07-21T22:37:48.771 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000160 2023-07-21T22:37:48.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.772 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-21T22:37:48.772 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 550880 2023-07-21T22:37:48.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.772 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:48.772 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 298581, 2023-07-21T22:37:48.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 412543, 2023-07-21T22:37:48.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 107733358, 2023-07-21T22:37:48.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 326494546, 2023-07-21T22:37:48.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 12, 2023-07-21T22:37:48.773 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 7, 2023-07-21T22:37:48.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 17.098834996, 2023-07-21T22:37:48.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 7.592800539, 2023-07-21T22:37:48.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 11.575745041, 2023-07-21T22:37:48.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.437060699, 2023-07-21T22:37:48.774 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:48.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 412540, 2023-07-21T22:37:48.775 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 30.381765624, 2023-07-21T22:37:48.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000073645 2023-07-21T22:37:48.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.775 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:48.775 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 597462, 2023-07-21T22:37:48.776 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.079900549, 2023-07-21T22:37:48.776 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000133 2023-07-21T22:37:48.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.776 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 63034736, 2023-07-21T22:37:48.776 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 238676608 2023-07-21T22:37:48.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.777 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-07-21T22:37:48.777 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2023-07-21T22:37:48.777 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-07-21T22:37:48.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.778 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2023-07-21T22:37:48.778 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-07-21T22:37:48.778 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-07-21T22:37:48.778 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 57341, 2023-07-21T22:37:48.778 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 60.189695686, 2023-07-21T22:37:48.779 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001049679 2023-07-21T22:37:48.779 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.779 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2023-07-21T22:37:48.779 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2023-07-21T22:37:48.780 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2023-07-21T22:37:48.780 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 36797, 2023-07-21T22:37:48.780 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 6.108329473, 2023-07-21T22:37:48.780 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000166000 2023-07-21T22:37:48.780 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.781 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2023-07-21T22:37:48.781 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 123216, 2023-07-21T22:37:48.781 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 118560, 2023-07-21T22:37:48.781 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2023-07-21T22:37:48.781 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 118560, 2023-07-21T22:37:48.782 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 386.443818374, 2023-07-21T22:37:48.782 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003259478 2023-07-21T22:37:48.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.782 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2023-07-21T22:37:48.782 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 4656, 2023-07-21T22:37:48.783 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 1, 2023-07-21T22:37:48.783 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2023-07-21T22:37:48.783 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 66, 2023-07-21T22:37:48.783 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 6, 2023-07-21T22:37:48.783 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 6, 2023-07-21T22:37:48.784 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 2398, 2023-07-21T22:37:48.784 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 425, 2023-07-21T22:37:48.784 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 1187, 2023-07-21T22:37:48.784 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 786, 2023-07-21T22:37:48.784 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 1791, 2023-07-21T22:37:48.785 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 836, 2023-07-21T22:37:48.785 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 259, 2023-07-21T22:37:48.785 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 259, 2023-07-21T22:37:48.785 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 13, 2023-07-21T22:37:48.785 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 216288, 2023-07-21T22:37:48.785 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 112336, 2023-07-21T22:37:48.786 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 4651, 2023-07-21T22:37:48.786 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 65, 2023-07-21T22:37:48.786 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2023-07-21T22:37:48.786 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2023-07-21T22:37:48.786 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 2156, 2023-07-21T22:37:48.787 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 7134, 2023-07-21T22:37:48.787 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2023-07-21T22:37:48.787 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 12, 2023-07-21T22:37:48.787 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 554, 2023-07-21T22:37:48.787 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 18, 2023-07-21T22:37:48.788 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 1419, 2023-07-21T22:37:48.788 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2023-07-21T22:37:48.788 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 629, 2023-07-21T22:37:48.788 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2023-07-21T22:37:48.788 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 704, 2023-07-21T22:37:48.789 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 211879339, 2023-07-21T22:37:48.789 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2023-07-21T22:37:48.789 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 629, 2023-07-21T22:37:48.789 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2023-07-21T22:37:48.789 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T22:37:48.790 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2023-07-21T22:37:48.790 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 40, 2023-07-21T22:37:48.790 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T22:37:48.790 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 40, 2023-07-21T22:37:48.790 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 589, 2023-07-21T22:37:48.791 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 15, 2023-07-21T22:37:48.791 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 8385, 2023-07-21T22:37:48.791 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 87349, 2023-07-21T22:37:48.791 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T22:37:48.791 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 542, 2023-07-21T22:37:48.792 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T22:37:48.792 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 30903, 2023-07-21T22:37:48.792 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 16498, 2023-07-21T22:37:48.792 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 3490, 2023-07-21T22:37:48.792 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 98249 2023-07-21T22:37:48.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.793 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2023-07-21T22:37:48.793 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 58, 2023-07-21T22:37:48.793 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 41, 2023-07-21T22:37:48.793 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 30, 2023-07-21T22:37:48.793 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 82, 2023-07-21T22:37:48.794 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 146, 2023-07-21T22:37:48.794 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2023-07-21T22:37:48.794 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2023-07-21T22:37:48.794 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2023-07-21T22:37:48.794 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 21675, 2023-07-21T22:37:48.795 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 21675, 2023-07-21T22:37:48.795 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2023-07-21T22:37:48.795 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2023-07-21T22:37:48.795 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2023-07-21T22:37:48.795 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2023-07-21T22:37:48.796 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2023-07-21T22:37:48.796 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2023-07-21T22:37:48.796 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2023-07-21T22:37:48.796 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T22:37:48.796 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 15, 2023-07-21T22:37:48.797 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2023-07-21T22:37:48.797 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 12, 2023-07-21T22:37:48.797 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2023-07-21T22:37:48.797 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2023-07-21T22:37:48.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.798 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2023-07-21T22:37:48.798 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 104782, 2023-07-21T22:37:48.798 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 4924, 2023-07-21T22:37:48.798 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 4924, 2023-07-21T22:37:48.798 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 99858, 2023-07-21T22:37:48.798 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2023-07-21T22:37:48.799 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 1024, 2023-07-21T22:37:48.799 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 133, 2023-07-21T22:37:48.799 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 5, 2023-07-21T22:37:48.799 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 5, 2023-07-21T22:37:48.799 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 129, 2023-07-21T22:37:48.800 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2023-07-21T22:37:48.800 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 1, 2023-07-21T22:37:48.800 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 11338273, 2023-07-21T22:37:48.800 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 360103676, 2023-07-21T22:37:48.800 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2023-07-21T22:37:48.801 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2023-07-21T22:37:48.801 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 30479, 2023-07-21T22:37:48.801 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 170.274669405, 2023-07-21T22:37:48.801 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005586622 2023-07-21T22:37:48.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.802 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2023-07-21T22:37:48.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.802 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2023-07-21T22:37:48.802 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 1342, 2023-07-21T22:37:48.802 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 23558, 2023-07-21T22:37:48.802 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 22216, 2023-07-21T22:37:48.803 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 121, 2023-07-21T22:37:48.803 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 222, 2023-07-21T22:37:48.803 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 101, 2023-07-21T22:37:48.803 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 2398, 2023-07-21T22:37:48.803 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 37170, 2023-07-21T22:37:48.804 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 34772, 2023-07-21T22:37:48.804 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 259, 2023-07-21T22:37:48.804 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 22776, 2023-07-21T22:37:48.804 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 22517, 2023-07-21T22:37:48.804 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 80684, 2023-07-21T22:37:48.805 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 198964 2023-07-21T22:37:48.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.805 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2023-07-21T22:37:48.805 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 196868, 2023-07-21T22:37:48.805 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 475, 2023-07-21T22:37:48.806 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 123216, 2023-07-21T22:37:48.806 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 2388, 2023-07-21T22:37:48.806 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 14798, 2023-07-21T22:37:48.806 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2023-07-21T22:37:48.806 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 21722, 2023-07-21T22:37:48.807 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 79.103134632, 2023-07-21T22:37:48.807 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003641613 2023-07-21T22:37:48.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.807 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2023-07-21T22:37:48.807 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1866, 2023-07-21T22:37:48.807 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 22.835319148, 2023-07-21T22:37:48.808 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.012237577 2023-07-21T22:37:48.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.808 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2023-07-21T22:37:48.808 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.808 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.809 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.809 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2023-07-21T22:37:48.809 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.809 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.810 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.810 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2023-07-21T22:37:48.810 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 28629, 2023-07-21T22:37:48.810 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 47.525583704, 2023-07-21T22:37:48.811 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001660050 2023-07-21T22:37:48.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.811 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2023-07-21T22:37:48.811 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.811 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.812 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.812 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.812 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2023-07-21T22:37:48.812 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2023-07-21T22:37:48.812 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000147501, 2023-07-21T22:37:48.812 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000147501 2023-07-21T22:37:48.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.813 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2023-07-21T22:37:48.813 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.813 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.813 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.814 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.814 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2023-07-21T22:37:48.814 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.814 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.814 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.815 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2023-07-21T22:37:48.815 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.815 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.815 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.816 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.816 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2023-07-21T22:37:48.816 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.816 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.816 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.817 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.817 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2023-07-21T22:37:48.817 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.817 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.817 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.818 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.818 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2023-07-21T22:37:48.818 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.818 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.818 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.818 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.819 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2023-07-21T22:37:48.819 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.819 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.819 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.820 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2023-07-21T22:37:48.820 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2023-07-21T22:37:48.820 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000400096, 2023-07-21T22:37:48.820 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000400096 2023-07-21T22:37:48.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.821 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2023-07-21T22:37:48.821 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40646, 2023-07-21T22:37:48.821 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 155.736008981, 2023-07-21T22:37:48.821 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003831521 2023-07-21T22:37:48.821 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.822 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2023-07-21T22:37:48.822 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3668, 2023-07-21T22:37:48.822 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 14.446655278, 2023-07-21T22:37:48.822 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003938564 2023-07-21T22:37:48.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.822 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2023-07-21T22:37:48.823 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.823 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.823 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.823 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.823 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2023-07-21T22:37:48.824 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.824 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.824 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.824 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2023-07-21T22:37:48.825 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.825 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.825 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.825 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2023-07-21T22:37:48.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.826 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.826 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2023-07-21T22:37:48.826 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.827 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.827 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.827 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2023-07-21T22:37:48.827 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 24, 2023-07-21T22:37:48.828 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.253703048, 2023-07-21T22:37:48.828 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.010570960 2023-07-21T22:37:48.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.828 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2023-07-21T22:37:48.828 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.829 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.829 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.829 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2023-07-21T22:37:48.829 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 328, 2023-07-21T22:37:48.830 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.961132391, 2023-07-21T22:37:48.830 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002930281 2023-07-21T22:37:48.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.830 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2023-07-21T22:37:48.830 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.831 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.831 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.831 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2023-07-21T22:37:48.831 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.832 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.832 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.832 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2023-07-21T22:37:48.832 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.833 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.833 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.833 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2023-07-21T22:37:48.833 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 21675, 2023-07-21T22:37:48.833 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 65.581733595, 2023-07-21T22:37:48.834 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003025685 2023-07-21T22:37:48.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.834 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2023-07-21T22:37:48.834 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2023-07-21T22:37:48.834 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2023-07-21T22:37:48.835 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.835 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.835 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.835 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.836 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2023-07-21T22:37:48.836 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2023-07-21T22:37:48.836 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 3, 2023-07-21T22:37:48.836 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2023-07-21T22:37:48.836 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2023-07-21T22:37:48.837 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2023-07-21T22:37:48.837 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 30426, 2023-07-21T22:37:48.837 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 30426, 2023-07-21T22:37:48.837 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 975 2023-07-21T22:37:48.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.837 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-07-21T22:37:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:48.838 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:48.839 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:48.840 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:48.840 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:48.840 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:48.840 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:48.840 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:48.841 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:48.841 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:48.841 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:48.841 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:48.841 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:48.842 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:48.842 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:48.842 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:48.842 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:48.842 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:48.842 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:48.843 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:48.843 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:48.843 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:48.843 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:48.843 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:48.844 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-07-21T22:37:48.844 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:48.844 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:48.844 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:48.844 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:48.845 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 789007, 2023-07-21T22:37:48.845 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 1332, 2023-07-21T22:37:48.845 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 264, 2023-07-21T22:37:48.845 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 3, 2023-07-21T22:37:48.845 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-07-21T22:37:48.846 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-07-21T22:37:48.846 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:48.846 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:48.846 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:48.846 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:48.846 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 9608, 2023-07-21T22:37:48.847 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 184, 2023-07-21T22:37:48.847 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:48.847 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:48.847 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:48.847 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-07-21T22:37:48.848 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 5528645, 2023-07-21T22:37:48.848 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 55697, 2023-07-21T22:37:48.848 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:48.848 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-07-21T22:37:48.848 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:48.849 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-07-21T22:37:48.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.849 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2023-07-21T22:37:48.849 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2023-07-21T22:37:48.849 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2023-07-21T22:37:48.850 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 76842, 2023-07-21T22:37:48.850 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 360172227, 2023-07-21T22:37:48.850 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 28, 2023-07-21T22:37:48.850 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 76842, 2023-07-21T22:37:48.850 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2023-07-21T22:37:48.851 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 76842, 2023-07-21T22:37:48.851 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 529.100545361, 2023-07-21T22:37:48.851 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006885564 2023-07-21T22:37:48.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.851 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2023-07-21T22:37:48.851 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2023-07-21T22:37:48.852 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 76842, 2023-07-21T22:37:48.852 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 77278 2023-07-21T22:37:48.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.852 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 76842, 2023-07-21T22:37:48.852 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 1405, 2023-07-21T22:37:48.853 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 75437, 2023-07-21T22:37:48.853 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2023-07-21T22:37:48.853 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2023-07-21T22:37:48.853 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2023-07-21T22:37:48.853 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 16, 2023-07-21T22:37:48.854 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 775, 2023-07-21T22:37:48.854 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 31254, 2023-07-21T22:37:48.854 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 435, 2023-07-21T22:37:48.854 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2023-07-21T22:37:48.854 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2023-07-21T22:37:48.855 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2023-07-21T22:37:48.855 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 6, 2023-07-21T22:37:48.855 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 43478, 2023-07-21T22:37:48.855 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2023-07-21T22:37:48.855 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2023-07-21T22:37:48.855 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2023-07-21T22:37:48.856 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 630, 2023-07-21T22:37:48.856 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 32, 2023-07-21T22:37:48.856 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2023-07-21T22:37:48.856 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2023-07-21T22:37:48.856 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2023-07-21T22:37:48.857 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2023-07-21T22:37:48.857 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2023-07-21T22:37:48.857 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2023-07-21T22:37:48.857 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T22:37:48.857 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2023-07-21T22:37:48.858 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2023-07-21T22:37:48.858 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2023-07-21T22:37:48.858 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2023-07-21T22:37:48.858 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2023-07-21T22:37:48.858 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2023-07-21T22:37:48.859 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2023-07-21T22:37:48.859 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2023-07-21T22:37:48.859 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2023-07-21T22:37:48.859 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2023-07-21T22:37:48.859 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2023-07-21T22:37:48.859 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2023-07-21T22:37:48.860 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2023-07-21T22:37:48.860 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2023-07-21T22:37:48.860 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2023-07-21T22:37:48.860 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2023-07-21T22:37:48.860 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2023-07-21T22:37:48.861 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2023-07-21T22:37:48.861 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2023-07-21T22:37:48.861 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 114, 2023-07-21T22:37:48.861 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2023-07-21T22:37:48.861 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 25, 2023-07-21T22:37:48.862 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2023-07-21T22:37:48.862 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2023-07-21T22:37:48.862 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2023-07-21T22:37:48.862 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2023-07-21T22:37:48.863 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 467, 2023-07-21T22:37:48.863 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 2, 2023-07-21T22:37:48.863 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 183 2023-07-21T22:37:48.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.863 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2023-07-21T22:37:48.864 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2023-07-21T22:37:48.864 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2023-07-21T22:37:48.864 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2023-07-21T22:37:48.864 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2023-07-21T22:37:48.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.865 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2023-07-21T22:37:48.865 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 123550 2023-07-21T22:37:48.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.865 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2023-07-21T22:37:48.865 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2023-07-21T22:37:48.866 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 16, 2023-07-21T22:37:48.866 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2023-07-21T22:37:48.866 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 8, 2023-07-21T22:37:48.866 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 21675, 2023-07-21T22:37:48.866 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2023-07-21T22:37:48.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.867 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:48.867 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:48.867 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:48.867 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:48.867 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 378312, 2023-07-21T22:37:48.868 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 87079998, 2023-07-21T22:37:48.868 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:48.868 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 378312, 2023-07-21T22:37:48.868 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:48.868 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:48.869 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 378312, 2023-07-21T22:37:48.869 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 87079998, 2023-07-21T22:37:48.869 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:48.869 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.869 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.870 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.870 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.870 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2023-07-21T22:37:48.870 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:48.871 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:48.871 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:48.871 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:48.871 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:48.871 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:48.871 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:48.872 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 76842, 2023-07-21T22:37:48.872 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 362537762, 2023-07-21T22:37:48.872 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 76842, 2023-07-21T22:37:48.872 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 362537762, 2023-07-21T22:37:48.872 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:48.873 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.873 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.873 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.873 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.874 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2023-07-21T22:37:48.874 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:48.874 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2023-07-21T22:37:48.874 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:48.874 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2023-07-21T22:37:48.875 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2023-07-21T22:37:48.875 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:48.875 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2023-07-21T22:37:48.875 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 76842, 2023-07-21T22:37:48.875 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 76842, 2023-07-21T22:37:48.876 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 76842, 2023-07-21T22:37:48.876 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 76842, 2023-07-21T22:37:48.876 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:48.876 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.876 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.876 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2023-07-21T22:37:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-07-21T22:37:48.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:48.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 21675, 2023-07-21T22:37:48.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 2365551, 2023-07-21T22:37:48.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:48.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 21675, 2023-07-21T22:37:48.879 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:48.879 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:48.879 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 775, 2023-07-21T22:37:48.879 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2365551, 2023-07-21T22:37:48.879 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:48.880 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.880 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.880 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.880 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:48.880 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x5584243300a0": { 2023-07-21T22:37:48.881 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 3804, 2023-07-21T22:37:48.881 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2023-07-21T22:37:48.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:48.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 104782, 2023-07-21T22:37:48.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 355909372, 2023-07-21T22:37:48.882 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:48.882 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 104782, 2023-07-21T22:37:48.882 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:48.882 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:48.882 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 30479, 2023-07-21T22:37:48.883 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 355905568, 2023-07-21T22:37:48.883 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:48.883 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:48.883 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:48.883 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:48.884 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.884 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:48.884 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-07-21T22:37:49.269 INFO:tasks.check_counter:Daemon mds.i mds.exported=12 2023-07-21T22:37:49.269 INFO:tasks.check_counter:Daemon mds.i mds.imported=18 2023-07-21T22:37:49.269 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-07-21T22:37:49.270 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-07-21T22:37:49.270 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T22:37:49.270 DEBUG:tasks.check_counter:Getting stats from l 2023-07-21T22:37:49.270 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea shell --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-21T22:37:49.827 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T22:37:50.394 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:50 smithi149 ceph-mon[129268]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 8 op/s 2023-07-21T22:37:50.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:50 smithi066 ceph-mon[126379]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 8 op/s 2023-07-21T22:37:50.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:50 smithi104 ceph-mon[129021]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 8 op/s 2023-07-21T22:37:51.222 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2023-07-21T22:37:51.223 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T22:37:51.223 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 0, 2023-07-21T22:37:51.223 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 2, 2023-07-21T22:37:51.224 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 0, 2023-07-21T22:37:51.224 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 0, 2023-07-21T22:37:51.224 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 4, 2023-07-21T22:37:51.224 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 0, 2023-07-21T22:37:51.224 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.001075115, 2023-07-21T22:37:51.225 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T22:37:51.225 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.000198300, 2023-07-21T22:37:51.225 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:51.225 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:51.225 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:51.226 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:51.226 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:51.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.226 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:51.226 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:51.226 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:51.227 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:51.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.227 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T22:37:51.227 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T22:37:51.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.228 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T22:37:51.228 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 320, 2023-07-21T22:37:51.228 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 309, 2023-07-21T22:37:51.228 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1585392, 2023-07-21T22:37:51.228 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 176688, 2023-07-21T22:37:51.229 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 2, 2023-07-21T22:37:51.229 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 1, 2023-07-21T22:37:51.229 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.054479147, 2023-07-21T22:37:51.229 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.016489972, 2023-07-21T22:37:51.229 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.038618735, 2023-07-21T22:37:51.230 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.008146465, 2023-07-21T22:37:51.230 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:51.230 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 308, 2023-07-21T22:37:51.230 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.020338900, 2023-07-21T22:37:51.230 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000066035 2023-07-21T22:37:51.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.231 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:51.231 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:51.231 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:51.231 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:51.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.232 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1585392, 2023-07-21T22:37:51.232 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 176688 2023-07-21T22:37:51.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.232 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T22:37:51.232 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 6, 2023-07-21T22:37:51.233 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 238, 2023-07-21T22:37:51.233 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3264, 2023-07-21T22:37:51.233 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 193296, 2023-07-21T22:37:51.233 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2023-07-21T22:37:51.233 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2023-07-21T22:37:51.234 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.012477813, 2023-07-21T22:37:51.234 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.008378261, 2023-07-21T22:37:51.234 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.001566083, 2023-07-21T22:37:51.234 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T22:37:51.234 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T22:37:51.235 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 236, 2023-07-21T22:37:51.235 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.025723211, 2023-07-21T22:37:51.235 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000108996 2023-07-21T22:37:51.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.235 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2023-07-21T22:37:51.235 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:51.236 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:51.236 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:51.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.236 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 3264, 2023-07-21T22:37:51.236 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 193296 2023-07-21T22:37:51.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.237 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2023-07-21T22:37:51.237 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 0, 2023-07-21T22:37:51.237 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2023-07-21T22:37:51.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.238 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2023-07-21T22:37:51.238 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2023-07-21T22:37:51.238 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2023-07-21T22:37:51.238 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T22:37:51.238 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2023-07-21T22:37:51.239 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T22:37:51.239 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2023-07-21T22:37:51.239 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T22:37:51.239 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T22:37:51.239 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T22:37:51.239 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T22:37:51.240 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T22:37:51.240 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2023-07-21T22:37:51.240 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T22:37:51.240 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T22:37:51.240 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2023-07-21T22:37:51.241 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2023-07-21T22:37:51.241 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2023-07-21T22:37:51.241 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2023-07-21T22:37:51.241 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T22:37:51.241 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T22:37:51.242 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T22:37:51.242 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T22:37:51.242 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T22:37:51.242 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2023-07-21T22:37:51.242 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2023-07-21T22:37:51.243 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2023-07-21T22:37:51.243 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T22:37:51.243 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T22:37:51.243 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2023-07-21T22:37:51.243 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2023-07-21T22:37:51.244 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2023-07-21T22:37:51.244 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2023-07-21T22:37:51.244 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T22:37:51.244 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2023-07-21T22:37:51.244 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T22:37:51.245 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2023-07-21T22:37:51.245 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 148977, 2023-07-21T22:37:51.245 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 30, 2023-07-21T22:37:51.245 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2023-07-21T22:37:51.245 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2023-07-21T22:37:51.245 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2023-07-21T22:37:51.246 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2023-07-21T22:37:51.246 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2023-07-21T22:37:51.246 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2023-07-21T22:37:51.246 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2023-07-21T22:37:51.246 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2023-07-21T22:37:51.247 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 0, 2023-07-21T22:37:51.247 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 0, 2023-07-21T22:37:51.247 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T22:37:51.247 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2023-07-21T22:37:51.247 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2023-07-21T22:37:51.248 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2023-07-21T22:37:51.248 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 928, 2023-07-21T22:37:51.248 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 2, 2023-07-21T22:37:51.248 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2023-07-21T22:37:51.248 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2023-07-21T22:37:51.249 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2023-07-21T22:37:51.249 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2023-07-21T22:37:51.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2023-07-21T22:37:51.249 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T22:37:51.249 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2023-07-21T22:37:51.250 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2023-07-21T22:37:51.250 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2023-07-21T22:37:51.250 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 322, 2023-07-21T22:37:51.250 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1540181, 2023-07-21T22:37:51.250 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2023-07-21T22:37:51.250 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 322, 2023-07-21T22:37:51.251 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2023-07-21T22:37:51.251 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2023-07-21T22:37:51.251 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 322, 2023-07-21T22:37:51.251 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1540181, 2023-07-21T22:37:51.251 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2023-07-21T22:37:51.252 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2023-07-21T22:37:51.252 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2023-07-21T22:37:51.252 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2023-07-21T22:37:51.252 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:51.252 INFO:teuthology.orchestra.run.smithi149.stdout: } 2023-07-21T22:37:51.253 INFO:teuthology.orchestra.run.smithi149.stdout:} 2023-07-21T22:37:51.677 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-21T22:37:51.677 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-21T22:37:51.677 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-07-21T22:37:51.678 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-07-21T22:37:51.678 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-07-21T22:37:51.678 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-07-21T22:37:51.690 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T22:37:51.691 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:37:51.691 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:37:51.744 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:37:51.745 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:37:51.794 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:37:52.046 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 -- 172.21.15.66:0/2895065011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17101004a0 msgr2=0x7f171010c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 --2- 172.21.15.66:0/2895065011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17101004a0 0x7f171010c930 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f170800b0a0 tx=0x7f170800b3b0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 -- 172.21.15.66:0/2895065011 shutdown_connections 2023-07-21T22:37:52.047 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 --2- 172.21.15.66:0/2895065011 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17101004a0 0x7f171010c930 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 --2- 172.21.15.66:0/2895065011 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1710105b00 0x7f1710105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 --2- 172.21.15.66:0/2895065011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17100ffaf0 0x7f17100fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 -- 172.21.15.66:0/2895065011 >> 172.21.15.66:0/2895065011 conn(0x7f17100fb630 msgr2=0x7f17100fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:52.048 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 -- 172.21.15.66:0/2895065011 shutdown_connections 2023-07-21T22:37:52.049 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.001+0000 7f1714bff700 1 -- 172.21.15.66:0/2895065011 wait complete. 2023-07-21T22:37:52.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 Processor -- start 2023-07-21T22:37:52.050 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 -- start start 2023-07-21T22:37:52.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 0x7f1710072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17101004a0 0x7f1710072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1710105b00 0x7f17101a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1710073580 con 0x7f17100ffaf0 2023-07-21T22:37:52.051 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101a4390 con 0x7f17101004a0 2023-07-21T22:37:52.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f1714bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17101a44d0 con 0x7f1710105b00 2023-07-21T22:37:52.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f170e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 0x7f1710072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f170e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 0x7f1710072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54566/0 (socket says 172.21.15.66:54566) 2023-07-21T22:37:52.052 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.003+0000 7f170e59c700 1 -- 172.21.15.66:0/2459741621 learned_addr learned my addr 172.21.15.66:0/2459741621 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:52.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170ed9d700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1710105b00 0x7f17101a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f1707fff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17101004a0 0x7f1710072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170e59c700 1 -- 172.21.15.66:0/2459741621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1710105b00 msgr2=0x7f17101a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170e59c700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1710105b00 0x7f17101a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.053 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170e59c700 1 -- 172.21.15.66:0/2459741621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17101004a0 msgr2=0x7f1710072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170e59c700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17101004a0 0x7f1710072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170e59c700 1 -- 172.21.15.66:0/2459741621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1708009d00 con 0x7f17100ffaf0 2023-07-21T22:37:52.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f170e59c700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 0x7f1710072810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f170000e9a0 tx=0x7f170000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:52.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700009790 con 0x7f17100ffaf0 2023-07-21T22:37:52.054 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1700010430 con 0x7f17100ffaf0 2023-07-21T22:37:52.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17101a4730 con 0x7f17100ffaf0 2023-07-21T22:37:52.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1700018680 con 0x7f17100ffaf0 2023-07-21T22:37:52.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.004+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17101a4c10 con 0x7f17100ffaf0 2023-07-21T22:37:52.055 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.006+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1710067300 con 0x7f17100ffaf0 2023-07-21T22:37:52.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.008+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f17000105a0 con 0x7f17100ffaf0 2023-07-21T22:37:52.056 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.009+0000 7f17077fe700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f16fc0a1300 0x7f16fc0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.009+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1700014070 con 0x7f17100ffaf0 2023-07-21T22:37:52.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.009+0000 7f1707fff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f16fc0a1300 0x7f16fc0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.057 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.010+0000 7f1707fff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f16fc0a1300 0x7f16fc0a37b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1710074000 tx=0x7f16f8009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:52.058 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.013+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1700051770 con 0x7f17100ffaf0 2023-07-21T22:37:52.270 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.226+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1710073d40 con 0x7f17100ffaf0 2023-07-21T22:37:52.272 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.227+0000 7f17077fe700 1 -- 172.21.15.66:0/2459741621 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+25149 (secure 0 0 0) 0x7f1700096260 con 0x7f17100ffaf0 2023-07-21T22:37:52.272 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:37:52.273 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":114,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:37:13.087333+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/4012457980":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/4235245168":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1497274919":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6801/1262827329":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6800/1262827329":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3171200415":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:37:52.274 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f16fc0a1300 msgr2=0x7f16fc0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f16fc0a1300 0x7f16fc0a37b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1710074000 tx=0x7f16f8009260 comp rx=0 tx=0).stop 2023-07-21T22:37:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 msgr2=0x7f1710072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 0x7f1710072810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f170000e9a0 tx=0x7f170000ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.275 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 shutdown_connections 2023-07-21T22:37:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f16fc0a1300 0x7f16fc0a37b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1710105b00 0x7f17101a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f17100ffaf0 0x7f1710072810 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 --2- 172.21.15.66:0/2459741621 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17101004a0 0x7f1710072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.276 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 >> 172.21.15.66:0/2459741621 conn(0x7f17100fb630 msgr2=0x7f17100fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:52.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 shutdown_connections 2023-07-21T22:37:52.277 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.230+0000 7f1714bff700 1 -- 172.21.15.66:0/2459741621 wait complete. 2023-07-21T22:37:52.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T22:14:54.912397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T22:19:05.097209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T22:19:06.344719+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-07-21T22:19:10.054857+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 113, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-07-21T22:37:52.295 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T22:37:52.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:52 smithi066 ceph-mon[126379]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-07-21T22:37:52.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:52 smithi066 ceph-mon[126379]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:52.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:52 smithi149 ceph-mon[129268]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-07-21T22:37:52.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:52 smithi149 ceph-mon[129268]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:52.506 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 -- 172.21.15.66:0/1902124846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0ffb00 msgr2=0x7f65ec0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 --2- 172.21.15.66:0/1902124846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0ffb00 0x7f65ec0fff70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0098f0 tx=0x7f65dc009c00 comp rx=0 tx=0).stop 2023-07-21T22:37:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 -- 172.21.15.66:0/1902124846 shutdown_connections 2023-07-21T22:37:52.507 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 --2- 172.21.15.66:0/1902124846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec105b10 0x7f65ec105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 --2- 172.21.15.66:0/1902124846 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec1004b0 0x7f65ec10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 --2- 172.21.15.66:0/1902124846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec0ffb00 0x7f65ec0fff70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 -- 172.21.15.66:0/1902124846 >> 172.21.15.66:0/1902124846 conn(0x7f65ec0fb600 msgr2=0x7f65ec0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:52.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 -- 172.21.15.66:0/1902124846 shutdown_connections 2023-07-21T22:37:52.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.462+0000 7f65f37ff700 1 -- 172.21.15.66:0/1902124846 wait complete. 2023-07-21T22:37:52.510 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 Processor -- start 2023-07-21T22:37:52.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 -- start start 2023-07-21T22:37:52.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 0x7f65ec072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec1004b0 0x7f65ec072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec105b10 0x7f65ec1a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.511 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec073580 con 0x7f65ec105b10 2023-07-21T22:37:52.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1a4370 con 0x7f65ec0ffb00 2023-07-21T22:37:52.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.463+0000 7f65f37ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec1a44b0 con 0x7f65ec1004b0 2023-07-21T22:37:52.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 0x7f65ec072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f0d9a700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec1004b0 0x7f65ec072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 0x7f65ec072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:46942/0 (socket says 172.21.15.66:46942) 2023-07-21T22:37:52.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 -- 172.21.15.66:0/3925107700 learned_addr learned my addr 172.21.15.66:0/3925107700 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:52.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f1d9c700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec105b10 0x7f65ec1a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 -- 172.21.15.66:0/3925107700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec1004b0 msgr2=0x7f65ec072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec1004b0 0x7f65ec072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 -- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec105b10 msgr2=0x7f65ec1a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec105b10 0x7f65ec1a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 -- 172.21.15.66:0/3925107700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4009c40 con 0x7f65ec0ffb00 2023-07-21T22:37:52.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f0d9a700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec1004b0 0x7f65ec072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:52.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.464+0000 7f65f159b700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 0x7f65ec072810 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e9c0 tx=0x7f65e800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:52.515 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.465+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8004a30 con 0x7f65ec0ffb00 2023-07-21T22:37:52.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.465+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc009520 con 0x7f65ec0ffb00 2023-07-21T22:37:52.516 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.465+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65e800cc80 con 0x7f65ec0ffb00 2023-07-21T22:37:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.465+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8005440 con 0x7f65ec0ffb00 2023-07-21T22:37:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.465+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec1a4ab0 con 0x7f65ec0ffb00 2023-07-21T22:37:52.517 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.466+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec067290 con 0x7f65ec0ffb00 2023-07-21T22:37:52.519 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.468+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f65e8004b90 con 0x7f65ec0ffb00 2023-07-21T22:37:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.469+0000 7f65e27fc700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f65d80a12c0 0x7f65d80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.520 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.470+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f65e8014070 con 0x7f65ec0ffb00 2023-07-21T22:37:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.470+0000 7f65f0d9a700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f65d80a12c0 0x7f65d80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.470+0000 7f65f0d9a700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f65d80a12c0 0x7f65d80a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005e90 tx=0x7f65dc019040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:52.521 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.471+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f65e8096340 con 0x7f65ec0ffb00 2023-07-21T22:37:52.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:52 smithi104 ceph-mon[129021]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 66 KiB/s wr, 11 op/s 2023-07-21T22:37:52.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:52 smithi104 ceph-mon[129021]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T22:37:52.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.686+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65ec1a4da0 con 0x7f65ec0ffb00 2023-07-21T22:37:52.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.687+0000 7f65e27fc700 1 -- 172.21.15.66:0/3925107700 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65e8099960 con 0x7f65ec0ffb00 2023-07-21T22:37:52.731 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2023-07-21T22:37:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f65d80a12c0 msgr2=0x7f65d80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f65d80a12c0 0x7f65d80a3770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005e90 tx=0x7f65dc019040 comp rx=0 tx=0).stop 2023-07-21T22:37:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 msgr2=0x7f65ec072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.734 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 0x7f65ec072810 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f65e800e9c0 tx=0x7f65e800ecd0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 shutdown_connections 2023-07-21T22:37:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec0ffb00 0x7f65ec072810 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f65d80a12c0 0x7f65d80a3770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f65ec105b10 0x7f65ec1a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.735 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 --2- 172.21.15.66:0/3925107700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65ec1004b0 0x7f65ec072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 >> 172.21.15.66:0/3925107700 conn(0x7f65ec0fb600 msgr2=0x7f65ec0fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 shutdown_connections 2023-07-21T22:37:52.736 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.689+0000 7f65f37ff700 1 -- 172.21.15.66:0/3925107700 wait complete. 2023-07-21T22:37:52.753 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T22:37:52.964 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.919+0000 7f0c7ffff700 1 -- 172.21.15.66:0/16254220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 msgr2=0x7f0c780fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.919+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/16254220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c780fff50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009990 tx=0x7f0c68009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 -- 172.21.15.66:0/16254220 shutdown_connections 2023-07-21T22:37:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/16254220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c78105af0 0x7f0c78105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.965 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/16254220 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78100490 0x7f0c7810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/16254220 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c780fff50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 -- 172.21.15.66:0/16254220 >> 172.21.15.66:0/16254220 conn(0x7f0c780fb5e0 msgr2=0x7f0c780fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 -- 172.21.15.66:0/16254220 shutdown_connections 2023-07-21T22:37:52.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.920+0000 7f0c7ffff700 1 -- 172.21.15.66:0/16254220 wait complete. 2023-07-21T22:37:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 Processor -- start 2023-07-21T22:37:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 -- start start 2023-07-21T22:37:52.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c78197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c78100490 0x7f0c78198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.969 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78105af0 0x7f0c7819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819cdf0 con 0x7f0c780ffae0 2023-07-21T22:37:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819cf60 con 0x7f0c78100490 2023-07-21T22:37:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.921+0000 7f0c7ffff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7819d120 con 0x7f0c78105af0 2023-07-21T22:37:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c78197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c78197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54616/0 (socket says 172.21.15.66:54616) 2023-07-21T22:37:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 -- 172.21.15.66:0/1254293095 learned_addr learned my addr 172.21.15.66:0/1254293095 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7d59a700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c78100490 0x7f0c78198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7e59c700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78105af0 0x7f0c7819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 -- 172.21.15.66:0/1254293095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78105af0 msgr2=0x7f0c7819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.971 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78105af0 0x7f0c7819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 -- 172.21.15.66:0/1254293095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c78100490 msgr2=0x7f0c78198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c78100490 0x7f0c78198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 -- 172.21.15.66:0/1254293095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c680095f0 con 0x7f0c780ffae0 2023-07-21T22:37:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7e59c700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78105af0 0x7f0c7819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:37:52.972 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7dd9b700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c78197e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9a0 tx=0x7f0c7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:52.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74009790 con 0x7f0c780ffae0 2023-07-21T22:37:52.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.922+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7819d400 con 0x7f0c780ffae0 2023-07-21T22:37:52.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.923+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c7400cbd0 con 0x7f0c780ffae0 2023-07-21T22:37:52.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.923+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c74018680 con 0x7f0c780ffae0 2023-07-21T22:37:52.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.923+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c781a4c40 con 0x7f0c780ffae0 2023-07-21T22:37:52.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.924+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c78067290 con 0x7f0c780ffae0 2023-07-21T22:37:52.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.926+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0c7400cd40 con 0x7f0c780ffae0 2023-07-21T22:37:52.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.927+0000 7f0c6effd700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c640a12c0 0x7f0c640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:52.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.927+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0c74014070 con 0x7f0c780ffae0 2023-07-21T22:37:52.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.927+0000 7f0c7d59a700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c640a12c0 0x7f0c640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:52.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.928+0000 7f0c7d59a700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c640a12c0 0x7f0c640a3770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c78199420 tx=0x7f0c68005fd0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:52.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:52.931+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0c74051770 con 0x7f0c780ffae0 2023-07-21T22:37:53.191 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.146+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c78199160 con 0x7f0c780ffae0 2023-07-21T22:37:53.192 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.147+0000 7f0c6effd700 1 -- 172.21.15.66:0/1254293095 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c78199160 con 0x7f0c780ffae0 2023-07-21T22:37:53.192 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:37:53.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.149+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c640a12c0 msgr2=0x7f0c640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.194 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.149+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c640a12c0 0x7f0c640a3770 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0c78199420 tx=0x7f0c68005fd0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.149+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 msgr2=0x7f0c78197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.149+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c78197e10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400e9a0 tx=0x7f0c7400ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 shutdown_connections 2023-07-21T22:37:53.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f0c640a12c0 0x7f0c640a3770 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.195 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c78105af0 0x7f0c7819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f0c780ffae0 0x7f0c78197e10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 --2- 172.21.15.66:0/1254293095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c78100490 0x7f0c78198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 >> 172.21.15.66:0/1254293095 conn(0x7f0c780fb5e0 msgr2=0x7f0c7810ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:53.196 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 shutdown_connections 2023-07-21T22:37:53.197 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.150+0000 7f0c7ffff700 1 -- 172.21.15.66:0/1254293095 wait complete. 2023-07-21T22:37:53.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T22:37:53.427 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 -- 172.21.15.66:0/2999192969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 msgr2=0x7f17701010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 --2- 172.21.15.66:0/2999192969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17701010b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1760009990 tx=0x7f1760009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.428 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 -- 172.21.15.66:0/2999192969 shutdown_connections 2023-07-21T22:37:53.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 --2- 172.21.15.66:0/2999192969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f177010a0b0 0x7f177010c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.429 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 --2- 172.21.15.66:0/2999192969 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17701010b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 --2- 172.21.15.66:0/2999192969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17701015f0 0x7f1770109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.382+0000 7f17749ff700 1 -- 172.21.15.66:0/2999192969 >> 172.21.15.66:0/2999192969 conn(0x7f17700fa530 msgr2=0x7f17700fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:53.430 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.383+0000 7f17749ff700 1 -- 172.21.15.66:0/2999192969 shutdown_connections 2023-07-21T22:37:53.431 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.383+0000 7f17749ff700 1 -- 172.21.15.66:0/2999192969 wait complete. 2023-07-21T22:37:53.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.383+0000 7f17749ff700 1 Processor -- start 2023-07-21T22:37:53.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 -- start start 2023-07-21T22:37:53.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17700fe930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.434 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17701015f0 0x7f17700fee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f177010a0b0 0x7f17701a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700ff6a0 con 0x7f1770100ce0 2023-07-21T22:37:53.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17701a4340 con 0x7f17701015f0 2023-07-21T22:37:53.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f17749ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17701a4480 con 0x7f177010a0b0 2023-07-21T22:37:53.435 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f176e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17700fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f176e59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17700fe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54622/0 (socket says 172.21.15.66:54622) 2023-07-21T22:37:53.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f176e59c700 1 -- 172.21.15.66:0/3068888854 learned_addr learned my addr 172.21.15.66:0/3068888854 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:53.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f176ed9d700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f177010a0b0 0x7f17701a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.436 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.384+0000 7f1767fff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17701015f0 0x7f17700fee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f176e59c700 1 -- 172.21.15.66:0/3068888854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f177010a0b0 msgr2=0x7f17701a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f176e59c700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f177010a0b0 0x7f17701a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f176e59c700 1 -- 172.21.15.66:0/3068888854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17701015f0 msgr2=0x7f17700fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f176e59c700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17701015f0 0x7f17700fee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.437 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f176e59c700 1 -- 172.21.15.66:0/3068888854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17600095f0 con 0x7f1770100ce0 2023-07-21T22:37:53.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f176e59c700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17700fe930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1760005910 tx=0x7f1760004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:53.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176001d070 con 0x7f1770100ce0 2023-07-21T22:37:53.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17701a4680 con 0x7f1770100ce0 2023-07-21T22:37:53.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1760022470 con 0x7f1770100ce0 2023-07-21T22:37:53.438 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760017650 con 0x7f1770100ce0 2023-07-21T22:37:53.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.385+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17701a4b00 con 0x7f1770100ce0 2023-07-21T22:37:53.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.387+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1770066a00 con 0x7f1770100ce0 2023-07-21T22:37:53.439 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.388+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f17600225e0 con 0x7f1770100ce0 2023-07-21T22:37:53.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.390+0000 7f17677fe700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f175c0a12c0 0x7f175c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.390+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f17600ce300 con 0x7f1770100ce0 2023-07-21T22:37:53.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.390+0000 7f1767fff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f175c0a12c0 0x7f175c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.440 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.391+0000 7f1767fff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f175c0a12c0 0x7f175c0a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1770100120 tx=0x7f1758009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:53.441 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.394+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f176005c770 con 0x7f1770100ce0 2023-07-21T22:37:53.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:53 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2459741621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:53.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:53 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3925107700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:37:53.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:53 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2459741621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:53.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:53 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3925107700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:37:53.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:53 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2459741621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:53.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:53 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3925107700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T22:37:53.649 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.604+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17701a4de0 con 0x7f1770100ce0 2023-07-21T22:37:53.650 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.605+0000 7f17677fe700 1 -- 172.21.15.66:0/3068888854 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f17600973a0 con 0x7f1770100ce0 2023-07-21T22:37:53.650 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 64 2023-07-21T22:37:53.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f175c0a12c0 msgr2=0x7f175c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.653 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f175c0a12c0 0x7f175c0a3770 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1770100120 tx=0x7f1758009260 comp rx=0 tx=0).stop 2023-07-21T22:37:53.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 msgr2=0x7f17700fe930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.654 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17700fe930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1760005910 tx=0x7f1760004710 comp rx=0 tx=0).stop 2023-07-21T22:37:53.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 shutdown_connections 2023-07-21T22:37:53.655 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f175c0a12c0 0x7f175c0a3770 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f177010a0b0 0x7f17701a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.656 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1770100ce0 0x7f17700fe930 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 --2- 172.21.15.66:0/3068888854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17701015f0 0x7f17700fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 >> 172.21.15.66:0/3068888854 conn(0x7f17700fa530 msgr2=0x7f177010c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:53.657 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 shutdown_connections 2023-07-21T22:37:53.658 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.608+0000 7f17749ff700 1 -- 172.21.15.66:0/3068888854 wait complete. 2023-07-21T22:37:53.672 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-07-21T22:37:53.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 -- 172.21.15.66:0/1559060118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f80ffad0 msgr2=0x7f72f80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 --2- 172.21.15.66:0/1559060118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f80ffad0 0x7f72f80fff40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009990 tx=0x7f72e8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.884 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 -- 172.21.15.66:0/1559060118 shutdown_connections 2023-07-21T22:37:53.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 --2- 172.21.15.66:0/1559060118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72f8105ae0 0x7f72f8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 --2- 172.21.15.66:0/1559060118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8100480 0x7f72f810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 --2- 172.21.15.66:0/1559060118 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f80ffad0 0x7f72f80fff40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 -- 172.21.15.66:0/1559060118 >> 172.21.15.66:0/1559060118 conn(0x7f72f80fb5f0 msgr2=0x7f72f80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:53.885 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 -- 172.21.15.66:0/1559060118 shutdown_connections 2023-07-21T22:37:53.886 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.839+0000 7f72ffdff700 1 -- 172.21.15.66:0/1559060118 wait complete. 2023-07-21T22:37:53.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.840+0000 7f72ffdff700 1 Processor -- start 2023-07-21T22:37:53.887 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.840+0000 7f72ffdff700 1 -- start start 2023-07-21T22:37:53.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72ffdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72f80ffad0 0x7f72f8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72ffdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f8100480 0x7f72f8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72ffdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 0x7f72f819c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72ffdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f819ce30 con 0x7f72f8100480 2023-07-21T22:37:53.888 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72ffdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f819cfa0 con 0x7f72f80ffad0 2023-07-21T22:37:53.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72ffdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f819d160 con 0x7f72f8105ae0 2023-07-21T22:37:53.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72fd39a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f8100480 0x7f72f8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72fe39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 0x7f72f819c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.889 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72fe39c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 0x7f72f819c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:54846/0 (socket says 172.21.15.66:54846) 2023-07-21T22:37:53.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72fe39c700 1 -- 172.21.15.66:0/963654707 learned_addr learned my addr 172.21.15.66:0/963654707 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:53.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.841+0000 7f72fdb9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72f80ffad0 0x7f72f8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fe39c700 1 -- 172.21.15.66:0/963654707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72f80ffad0 msgr2=0x7f72f8197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fe39c700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72f80ffad0 0x7f72f8197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.890 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fe39c700 1 -- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f8100480 msgr2=0x7f72f8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:53.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fe39c700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f8100480 0x7f72f8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:53.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fe39c700 1 -- 172.21.15.66:0/963654707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e80095f0 con 0x7f72f8105ae0 2023-07-21T22:37:53.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fd39a700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f8100480 0x7f72f8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:53.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72fe39c700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 0x7f72f819c7f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72f0007ee0 tx=0x7f72f000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:53.891 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0017070 con 0x7f72f8105ae0 2023-07-21T22:37:53.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f819d440 con 0x7f72f8105ae0 2023-07-21T22:37:53.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72f000f040 con 0x7f72f8105ae0 2023-07-21T22:37:53.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0005550 con 0x7f72f8105ae0 2023-07-21T22:37:53.892 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.842+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f81a4c80 con 0x7f72f8105ae0 2023-07-21T22:37:53.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.843+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f8067290 con 0x7f72f8105ae0 2023-07-21T22:37:53.893 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.846+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f72f000db20 con 0x7f72f8105ae0 2023-07-21T22:37:53.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.847+0000 7f72eeffd700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f72e40a12c0 0x7f72e40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:53.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.847+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f72f00ccea0 con 0x7f72f8105ae0 2023-07-21T22:37:53.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.847+0000 7f72fdb9b700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f72e40a12c0 0x7f72e40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:53.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.848+0000 7f72fdb9b700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f72e40a12c0 0x7f72e40a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009be0 tx=0x7f72f4009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:53.894 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:53.849+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f72f0095eb0 con 0x7f72f8105ae0 2023-07-21T22:37:54.102 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.058+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f72f81a4e10 con 0x7f72f8105ae0 2023-07-21T22:37:54.103 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.059+0000 7f72eeffd700 1 -- 172.21.15.66:0/963654707 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v114) v1 ==== 103+0+11 (secure 0 0 0) 0x7f72f00994d0 con 0x7f72f8105ae0 2023-07-21T22:37:54.104 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 32 2023-07-21T22:37:54.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f72e40a12c0 msgr2=0x7f72e40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.106 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f72e40a12c0 0x7f72e40a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009be0 tx=0x7f72f4009260 comp rx=0 tx=0).stop 2023-07-21T22:37:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 msgr2=0x7f72f819c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 0x7f72f819c7f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72f0007ee0 tx=0x7f72f000d3b0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 shutdown_connections 2023-07-21T22:37:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72f80ffad0 0x7f72f8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.107 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f72e40a12c0 0x7f72e40a3770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f72f8105ae0 0x7f72f819c7f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 --2- 172.21.15.66:0/963654707 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f72f8100480 0x7f72f8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.061+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 >> 172.21.15.66:0/963654707 conn(0x7f72f80fb5f0 msgr2=0x7f72f810ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:54.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.062+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 shutdown_connections 2023-07-21T22:37:54.108 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.062+0000 7f72ffdff700 1 -- 172.21.15.66:0/963654707 wait complete. 2023-07-21T22:37:54.126 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 -- 172.21.15.66:0/429302508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee40ffad0 msgr2=0x7f9ee40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.335 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 --2- 172.21.15.66:0/429302508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee40ffad0 0x7f9ee40fff40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009990 tx=0x7f9ed4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 -- 172.21.15.66:0/429302508 shutdown_connections 2023-07-21T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 --2- 172.21.15.66:0/429302508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee4105ae0 0x7f9ee4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 --2- 172.21.15.66:0/429302508 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4100480 0x7f9ee410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.336 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 --2- 172.21.15.66:0/429302508 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee40ffad0 0x7f9ee40fff40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 -- 172.21.15.66:0/429302508 >> 172.21.15.66:0/429302508 conn(0x7f9ee40fb5f0 msgr2=0x7f9ee40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 -- 172.21.15.66:0/429302508 shutdown_connections 2023-07-21T22:37:54.337 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.290+0000 7f9eebfff700 1 -- 172.21.15.66:0/429302508 wait complete. 2023-07-21T22:37:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.291+0000 7f9eebfff700 1 Processor -- start 2023-07-21T22:37:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 -- start start 2023-07-21T22:37:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee40ffad0 0x7f9ee4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:54.339 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 0x7f9ee4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4105ae0 0x7f9ee41a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee4073580 con 0x7f9ee4100480 2023-07-21T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41a4330 con 0x7f9ee40ffad0 2023-07-21T22:37:54.340 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eebfff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee41a4470 con 0x7f9ee4105ae0 2023-07-21T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9ee959a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 0x7f9ee4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:54.341 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9eea59c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4105ae0 0x7f9ee41a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9ee959a700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 0x7f9ee4072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54676/0 (socket says 172.21.15.66:54676) 2023-07-21T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9ee959a700 1 -- 172.21.15.66:0/3442360378 learned_addr learned my addr 172.21.15.66:0/3442360378 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:54.342 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.292+0000 7f9ee9d9b700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee40ffad0 0x7f9ee4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:54.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee959a700 1 -- 172.21.15.66:0/3442360378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4105ae0 msgr2=0x7f9ee41a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.343 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee959a700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4105ae0 0x7f9ee41a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee959a700 1 -- 172.21.15.66:0/3442360378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee40ffad0 msgr2=0x7f9ee4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.344 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee959a700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee40ffad0 0x7f9ee4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee959a700 1 -- 172.21.15.66:0/3442360378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40095f0 con 0x7f9ee4100480 2023-07-21T22:37:54.345 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee9d9b700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee40ffad0 0x7f9ee4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9eea59c700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4105ae0 0x7f9ee41a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9ee959a700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 0x7f9ee4072d50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005910 tx=0x7f9ed4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:54.346 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed401d070 con 0x7f9ee4100480 2023-07-21T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ed4022470 con 0x7f9ee4100480 2023-07-21T22:37:54.347 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee41a4670 con 0x7f9ee4100480 2023-07-21T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4017650 con 0x7f9ee4100480 2023-07-21T22:37:54.348 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.293+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee41a4af0 con 0x7f9ee4100480 2023-07-21T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.294+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee4067290 con 0x7f9ee4100480 2023-07-21T22:37:54.350 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.297+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9ed40225e0 con 0x7f9ee4100480 2023-07-21T22:37:54.351 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.298+0000 7f9edaffd700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9ed00a1340 0x7f9ed00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:54.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.298+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9ed40ce730 con 0x7f9ee4100480 2023-07-21T22:37:54.352 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.298+0000 7f9ee9d9b700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9ed00a1340 0x7f9ed00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:54.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.299+0000 7f9ee9d9b700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9ed00a1340 0x7f9ed00a37f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009fd0 tx=0x7f9ee0009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:54.353 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.300+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ed40977d0 con 0x7f9ee4100480 2023-07-21T22:37:54.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:54 smithi066 ceph-mon[126379]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 7 op/s 2023-07-21T22:37:54.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:54 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1254293095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:54 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3068888854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.446 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:54 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/963654707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:54 smithi149 ceph-mon[129268]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 7 op/s 2023-07-21T22:37:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:54 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1254293095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:54 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3068888854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:54 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/963654707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.573 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.529+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ee4073e30 con 0x7f9ee4100480 2023-07-21T22:37:54.574 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.529+0000 7f9edaffd700 1 -- 172.21.15.66:0/3442360378 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v114) v1 ==== 81+0+942 (secure 0 0 0) 0x7f9ed409adf0 con 0x7f9ee4100480 2023-07-21T22:37:54.574 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/4012457980 2023-07-22T22:21:28.704079+0000 2023-07-21T22:37:54.575 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/4235245168 2023-07-22T22:21:28.704079+0000 2023-07-21T22:37:54.575 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/662946744 2023-07-22T22:12:36.098613+0000 2023-07-21T22:37:54.575 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1497274919 2023-07-22T22:21:28.704079+0000 2023-07-21T22:37:54.575 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/3065709937 2023-07-22T22:12:36.098613+0000 2023-07-21T22:37:54.575 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/1244827991 2023-07-22T22:12:36.098613+0000 2023-07-21T22:37:54.576 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/1262827329 2023-07-22T22:21:28.704079+0000 2023-07-21T22:37:54.576 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/979491945 2023-07-22T22:12:09.674594+0000 2023-07-21T22:37:54.576 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6800/1262827329 2023-07-22T22:21:28.704079+0000 2023-07-21T22:37:54.576 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/3065709937 2023-07-22T22:12:36.098613+0000 2023-07-21T22:37:54.577 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3161416321 2023-07-22T22:12:09.674594+0000 2023-07-21T22:37:54.577 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3483118396 2023-07-22T22:12:09.674594+0000 2023-07-21T22:37:54.577 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:6801/979491945 2023-07-22T22:12:09.674594+0000 2023-07-21T22:37:54.577 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/690385827 2023-07-22T22:12:09.674594+0000 2023-07-21T22:37:54.577 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3171200415 2023-07-22T22:21:28.704079+0000 2023-07-21T22:37:54.578 INFO:teuthology.orchestra.run.smithi066.stdout:172.21.15.66:0/3486180168 2023-07-22T22:12:36.098613+0000 2023-07-21T22:37:54.578 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9ed00a1340 msgr2=0x7f9ed00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9ed00a1340 0x7f9ed00a37f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009fd0 tx=0x7f9ee0009260 comp rx=0 tx=0).stop 2023-07-21T22:37:54.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 msgr2=0x7f9ee4072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:54.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 0x7f9ee4072d50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4005910 tx=0x7f9ed4004710 comp rx=0 tx=0).stop 2023-07-21T22:37:54.579 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 shutdown_connections 2023-07-21T22:37:54.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ee40ffad0 0x7f9ee4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f9ed00a1340 0x7f9ed00a37f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.580 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4105ae0 0x7f9ee41a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 --2- 172.21.15.66:0/3442360378 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f9ee4100480 0x7f9ee4072d50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:54.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 >> 172.21.15.66:0/3442360378 conn(0x7f9ee40fb5f0 msgr2=0x7f9ee40fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:54.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 shutdown_connections 2023-07-21T22:37:54.581 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:54.532+0000 7f9eebfff700 1 -- 172.21.15.66:0/3442360378 wait complete. 2023-07-21T22:37:54.581 INFO:teuthology.orchestra.run.smithi066.stderr:listed 16 entries 2023-07-21T22:37:54.595 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-07-21T22:37:54.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:54.596 DEBUG:teuthology.orchestra.run.smithi066:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-07-21T22:37:54.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:54 smithi104 ceph-mon[129021]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 7 op/s 2023-07-21T22:37:54.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:54 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1254293095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:54 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3068888854' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:54 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/963654707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-07-21T22:37:54.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:37:54.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:54.898 DEBUG:teuthology.orchestra.run.smithi066:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-21T22:37:54.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:54.963 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-07-21T22:37:55.007 INFO:teuthology.orchestra.run.smithi066.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-21T22:37:55.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:55.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-21T22:37:55.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:55.123 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2023-07-21T22:37:55.240 INFO:tasks.kclient:Unmounting kernel clients... 2023-07-21T22:37:55.240 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2023-07-21T22:37:55.240 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T22:37:55.287 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345544k,nr_inodes=4086386,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=36,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/f195dfcf28b313b19d0329c0851220e341c9f759bbc8bc05e200e9faf3bcc614/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/f195dfcf28b313b19d0329c0851220e341c9f759bbc8bc05e200e9faf3bcc614/diff,workdir=/var/lib/containers/storage/overlay/f195dfcf28b313b19d0329c0851220e341c9f759bbc8bc05e200e9faf3bcc614/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/90e49de362f6bc5504f494aaab2e74201ac190bdf20e23fbd93999aacd9f4edb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/90e49de362f6bc5504f494aaab2e74201ac190bdf20e23fbd93999aacd9f4edb/diff,workdir=/var/lib/containers/storage/overlay/90e49de362f6bc5504f494aaab2e74201ac190bdf20e23fbd93999aacd9f4edb/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/15a9d33d5473bfd379ba1c137a013171b9ca464b3123c35875419a6eb0e7a224/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/15a9d33d5473bfd379ba1c137a013171b9ca464b3123c35875419a6eb0e7a224/diff,workdir=/var/lib/containers/storage/overlay/15a9d33d5473bfd379ba1c137a013171b9ca464b3123c35875419a6eb0e7a224/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/36a57b275defa83b199effeee05f905c13bf5bd2404e9861172998b0f05fd53e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/36a57b275defa83b199effeee05f905c13bf5bd2404e9861172998b0f05fd53e/diff,workdir=/var/lib/containers/storage/overlay/36a57b275defa83b199effeee05f905c13bf5bd2404e9861172998b0f05fd53e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c7a77712388c74fe4865fc42108b9f1ea23693fd0b18043d83c1fdf58d8b1cce/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/c7a77712388c74fe4865fc42108b9f1ea23693fd0b18043d83c1fdf58d8b1cce/diff,workdir=/var/lib/containers/storage/overlay/c7a77712388c74fe4865fc42108b9f1ea23693fd0b18043d83c1fdf58d8b1cce/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e6d09e6fad4c90249eb3e664660f7b5c12dc9c78a673c42a7c6529ab262f400e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/e6d09e6fad4c90249eb3e664660f7b5c12dc9c78a673c42a7c6529ab262f400e/diff,workdir=/var/lib/containers/storage/overlay/e6d09e6fad4c90249eb3e664660f7b5c12dc9c78a673c42a7c6529ab262f400e/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5df2c2fa8e834e56934ad7d38be3242d278cc63d635e2a35b403a97ac549ec17/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/5df2c2fa8e834e56934ad7d38be3242d278cc63d635e2a35b403a97ac549ec17/diff,workdir=/var/lib/containers/storage/overlay/5df2c2fa8e834e56934ad7d38be3242d278cc63d635e2a35b403a97ac549ec17/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f01a3214be4ee089522ad8dc528bf588a66cafa4d5ae5bb973d4e73c64ac3991/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/f01a3214be4ee089522ad8dc528bf588a66cafa4d5ae5bb973d4e73c64ac3991/diff,workdir=/var/lib/containers/storage/overlay/f01a3214be4ee089522ad8dc528bf588a66cafa4d5ae5bb973d4e73c64ac3991/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c09625e1968e2db893ea0928d955fd09b938c0653e19f73848700b1d214466b8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/c09625e1968e2db893ea0928d955fd09b938c0653e19f73848700b1d214466b8/diff,workdir=/var/lib/containers/storage/overlay/c09625e1968e2db893ea0928d955fd09b938c0653e19f73848700b1d214466b8/work,redirect_dir=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fc3a6afa657237e6695c1753ac947df80c3fbaceafd11b5b4ae710006d3dbcd4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/3DWHBHTD6BBLKNG36OWEQXZIWN:/var/lib/containers/storage/overlay/l/VDIYYAE54YZ4FP6JA7E4LFQ4JX,upperdir=/var/lib/containers/storage/overlay/fc3a6afa657237e6695c1753ac947df80c3fbaceafd11b5b4ae710006d3dbcd4/diff,workdir=/var/lib/containers/storage/overlay/fc3a6afa657237e6695c1753ac947df80c3fbaceafd11b5b4ae710006d3dbcd4/work,redirect_dir=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-07-21T22:37:55.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:55.288 DEBUG:teuthology.orchestra.run.smithi066:> ip netns list 2023-07-21T22:37:55.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T22:37:55.373 DEBUG:teuthology.orchestra.run.smithi066:> sudo ip link delete ceph-brx 2023-07-21T22:37:55.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:55 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3442360378' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:37:55.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:55 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3442360378' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:37:55.491 INFO:teuthology.orchestra.run.smithi066.stderr:Cannot find device "ceph-brx" 2023-07-21T22:37:55.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:37:55.495 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-21T22:37:55.506 INFO:tasks.cephadm:Teardown begin 2023-07-21T22:37:55.507 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-21T22:37:55.508 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:37:55.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:55 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3442360378' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T22:37:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 -- 172.21.15.66:0/2622687148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc8105b60 msgr2=0x7f1fc8105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/2622687148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc8105b60 0x7f1fc8105f30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4009990 tx=0x7f1fc4009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:55.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 -- 172.21.15.66:0/2622687148 shutdown_connections 2023-07-21T22:37:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/2622687148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc8105b60 0x7f1fc8105f30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/2622687148 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc81004e0 0x7f1fc810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/2622687148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fc80ffb30 0x7f1fc80fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 -- 172.21.15.66:0/2622687148 >> 172.21.15.66:0/2622687148 conn(0x7f1fc80fb610 msgr2=0x7f1fc80fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:55.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 -- 172.21.15.66:0/2622687148 shutdown_connections 2023-07-21T22:37:55.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.699+0000 7f1fd03ff700 1 -- 172.21.15.66:0/2622687148 wait complete. 2023-07-21T22:37:55.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.700+0000 7f1fd03ff700 1 Processor -- start 2023-07-21T22:37:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.700+0000 7f1fd03ff700 1 -- start start 2023-07-21T22:37:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fd03ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc80ffb30 0x7f1fc8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fd03ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 0x7f1fc8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:55.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fd03ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fc8105b60 0x7f1fc81a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:55.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fd03ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc8073580 con 0x7f1fc80ffb30 2023-07-21T22:37:55.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fd03ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc81a4280 con 0x7f1fc81004e0 2023-07-21T22:37:55.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fd03ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc81a43c0 con 0x7f1fc8105b60 2023-07-21T22:37:55.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fce19b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc80ffb30 0x7f1fc8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:55.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 0x7f1fc8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fce99c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fc8105b60 0x7f1fc81a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 0x7f1fc8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47030/0 (socket says 172.21.15.66:47030) 2023-07-21T22:37:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 -- 172.21.15.66:0/4246905909 learned_addr learned my addr 172.21.15.66:0/4246905909 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 -- 172.21.15.66:0/4246905909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fc8105b60 msgr2=0x7f1fc81a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:55.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fc8105b60 0x7f1fc81a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:55.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 -- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc80ffb30 msgr2=0x7f1fc8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:55.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc80ffb30 0x7f1fc8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:55.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.701+0000 7f1fcd99a700 1 -- 172.21.15.66:0/4246905909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc40095f0 con 0x7f1fc81004e0 2023-07-21T22:37:55.751 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.702+0000 7f1fcd99a700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 0x7f1fc8072d50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800e9a0 tx=0x7f1fb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:55.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.702+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8009790 con 0x7f1fc81004e0 2023-07-21T22:37:55.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.702+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1fb8010430 con 0x7f1fc81004e0 2023-07-21T22:37:55.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.702+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8018680 con 0x7f1fc81004e0 2023-07-21T22:37:55.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.702+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc81a4620 con 0x7f1fc81004e0 2023-07-21T22:37:55.752 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.702+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc81a4ab0 con 0x7f1fc81004e0 2023-07-21T22:37:55.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.703+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc8067300 con 0x7f1fc81004e0 2023-07-21T22:37:55.753 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.705+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1fb80105a0 con 0x7f1fc81004e0 2023-07-21T22:37:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.706+0000 7f1fbf7fe700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1fb40a1380 0x7f1fb40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.707+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1fb8014070 con 0x7f1fc81004e0 2023-07-21T22:37:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.707+0000 7f1fce19b700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1fb40a1380 0x7f1fb40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:55.754 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.707+0000 7f1fce19b700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1fb40a1380 0x7f1fb40a3830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400b2e0 tx=0x7f1fc4005fb0 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:55.755 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.708+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1fb80962b0 con 0x7f1fc81004e0 2023-07-21T22:37:56.010 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.966+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fc8073d40 con 0x7f1fc81004e0 2023-07-21T22:37:56.011 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.968+0000 7f1fbf7fe700 1 -- 172.21.15.66:0/4246905909 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f1fb80998d0 con 0x7f1fc81004e0 2023-07-21T22:37:56.012 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:37:56.012 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:37:51.117015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":250,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:37:56.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1fb40a1380 msgr2=0x7f1fb40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1fb40a1380 0x7f1fb40a3830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400b2e0 tx=0x7f1fc4005fb0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.014 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 msgr2=0x7f1fc8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 0x7f1fc8072d50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800e9a0 tx=0x7f1fb800ecb0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 shutdown_connections 2023-07-21T22:37:56.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f1fb40a1380 0x7f1fb40a3830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.015 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1fc8105b60 0x7f1fc81a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f1fc80ffb30 0x7f1fc8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 --2- 172.21.15.66:0/4246905909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc81004e0 0x7f1fc8072d50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.969+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 >> 172.21.15.66:0/4246905909 conn(0x7f1fc80fb610 msgr2=0x7f1fc81036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.016 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.970+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 shutdown_connections 2023-07-21T22:37:56.021 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:55.970+0000 7f1fd03ff700 1 -- 172.21.15.66:0/4246905909 wait complete. 2023-07-21T22:37:56.022 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 53 2023-07-21T22:37:56.033 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T22:37:56.242 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.197+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/1464776487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f840731c0 msgr2=0x7f5f8410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.197+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/1464776487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f840731c0 0x7f5f8410bdd0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5f740098f0 tx=0x7f5f74009c00 comp rx=0 tx=0).stop 2023-07-21T22:37:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/1464776487 shutdown_connections 2023-07-21T22:37:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/1464776487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f8410c310 0x7f5f8410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/1464776487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840728b0 0x7f5f84072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.243 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/1464776487 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f840731c0 0x7f5f8410bdd0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/1464776487 >> 172.21.15.66:0/1464776487 conn(0x7f5f840fb590 msgr2=0x7f5f840fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/1464776487 shutdown_connections 2023-07-21T22:37:56.244 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.198+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/1464776487 wait complete. 2023-07-21T22:37:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 Processor -- start 2023-07-21T22:37:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 -- start start 2023-07-21T22:37:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 0x7f5f84197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.246 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840731c0 0x7f5f84198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f8410c310 0x7f5f8419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8419cde0 con 0x7f5f8410c310 2023-07-21T22:37:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8419cf50 con 0x7f5f840728b0 2023-07-21T22:37:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8b5ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8419d110 con 0x7f5f840731c0 2023-07-21T22:37:56.247 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8939b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 0x7f5f84197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f89b9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f8410c310 0x7f5f8419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8939b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 0x7f5f84197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47058/0 (socket says 172.21.15.66:47058) 2023-07-21T22:37:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.199+0000 7f5f8939b700 1 -- 172.21.15.66:0/293191514 learned_addr learned my addr 172.21.15.66:0/293191514 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:56.248 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f88b9a700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840731c0 0x7f5f84198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8939b700 1 -- 172.21.15.66:0/293191514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840731c0 msgr2=0x7f5f84198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8939b700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840731c0 0x7f5f84198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8939b700 1 -- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f8410c310 msgr2=0x7f5f8419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8939b700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f8410c310 0x7f5f8419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.249 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8939b700 1 -- 172.21.15.66:0/293191514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f800095f0 con 0x7f5f840728b0 2023-07-21T22:37:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f88b9a700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840731c0 0x7f5f84198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T22:37:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f89b9c700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f8410c310 0x7f5f8419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8939b700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 0x7f5f84197e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000ec10 tx=0x7f5f8000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000cd40 con 0x7f5f840728b0 2023-07-21T22:37:56.250 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f800044e0 con 0x7f5f840728b0 2023-07-21T22:37:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80010690 con 0x7f5f840728b0 2023-07-21T22:37:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f74009520 con 0x7f5f840728b0 2023-07-21T22:37:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.200+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f841a4a50 con 0x7f5f840728b0 2023-07-21T22:37:56.251 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.201+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f84067300 con 0x7f5f840728b0 2023-07-21T22:37:56.252 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.203+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5f80003710 con 0x7f5f840728b0 2023-07-21T22:37:56.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.205+0000 7f5f7a7fc700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f700a1380 0x7f5f700a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.205+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f80014070 con 0x7f5f840728b0 2023-07-21T22:37:56.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.205+0000 7f5f88b9a700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f700a1380 0x7f5f700a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.206+0000 7f5f88b9a700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f700a1380 0x7f5f700a3830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f84199410 tx=0x7f5f74019040 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.253 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.207+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5f80096d40 con 0x7f5f840728b0 2023-07-21T22:37:56.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:56 smithi066 ceph-mon[126379]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-07-21T22:37:56.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:56 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/4246905909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:37:56.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:56 smithi149 ceph-mon[129268]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-07-21T22:37:56.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:56 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/4246905909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:37:56.508 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.463+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f84199150 con 0x7f5f840728b0 2023-07-21T22:37:56.509 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.464+0000 7f5f7a7fc700 1 -- 172.21.15.66:0/293191514 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+10825 (secure 0 0 0) 0x7f5f8000f3c0 con 0x7f5f840728b0 2023-07-21T22:37:56.509 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:37:56.510 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14433,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6835/1071874116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6834","nonce":1071874116},{"type":"v1","addr":"172.21.15.66:6835","nonce":1071874116}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6839/1234257491","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6838","nonce":1234257491},{"type":"v1","addr":"172.21.15.66:6839","nonce":1234257491}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14508,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6841/3476312972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6840","nonce":3476312972},{"type":"v1","addr":"172.21.15.66:6841","nonce":3476312972}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/2009309286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2009309286},{"type":"v1","addr":"172.21.15.149:6833","nonce":2009309286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24461,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/3341651114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":3341651114},{"type":"v1","addr":"172.21.15.104:6833","nonce":3341651114}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24485,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.66:6837/1892710488","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6836","nonce":1892710488},{"type":"v1","addr":"172.21.15.66:6837","nonce":1892710488}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":53,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T22:19:06.899289+0000","modified":"2023-07-21T22:37:51.117015+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24530,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24490":{"gid":24490,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6835/1642617037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1642617037},{"type":"v1","addr":"172.21.15.149:6835","nonce":1642617037}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1273148910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1273148910},{"type":"v1","addr":"172.21.15.104:6835","nonce":1273148910}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.149:6837/1037928792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1037928792},{"type":"v1","addr":"172.21.15.149:6837","nonce":1037928792}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":250,"addr":"172.21.15.149:6839/2751755690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":2751755690},{"type":"v1","addr":"172.21.15.149:6839","nonce":2751755690}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24530":{"gid":24530,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.104:6837/1237242254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":1237242254},{"type":"v1","addr":"172.21.15.104:6837","nonce":1237242254}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.104:6839/1299576617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":1299576617},{"type":"v1","addr":"172.21.15.104:6839","nonce":1299576617}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T22:37:56.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f700a1380 msgr2=0x7f5f700a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f700a1380 0x7f5f700a3830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f84199410 tx=0x7f5f74019040 comp rx=0 tx=0).stop 2023-07-21T22:37:56.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 msgr2=0x7f5f84197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.512 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 0x7f5f84197e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000ec10 tx=0x7f5f8000ef20 comp rx=0 tx=0).stop 2023-07-21T22:37:56.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 shutdown_connections 2023-07-21T22:37:56.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f840728b0 0x7f5f84197e00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5f700a1380 0x7f5f700a3830 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5f8410c310 0x7f5f8419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.513 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 --2- 172.21.15.66:0/293191514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f840731c0 0x7f5f84198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 >> 172.21.15.66:0/293191514 conn(0x7f5f840fb590 msgr2=0x7f5f8410e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 shutdown_connections 2023-07-21T22:37:56.514 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.467+0000 7f5f8b5ff700 1 -- 172.21.15.66:0/293191514 wait complete. 2023-07-21T22:37:56.514 INFO:teuthology.orchestra.run.smithi066.stderr:dumped fsmap epoch 53 2023-07-21T22:37:56.531 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T22:37:56.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:56 smithi104 ceph-mon[129021]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 172 KiB/s wr, 12 op/s 2023-07-21T22:37:56.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:56 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/4246905909' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:37:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 -- 172.21.15.66:0/1727614789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc105ae0 msgr2=0x7fc1bc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/1727614789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc105eb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009990 tx=0x7fc1b8009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.739 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 -- 172.21.15.66:0/1727614789 shutdown_connections 2023-07-21T22:37:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/1727614789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc100480 0x7fc1bc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/1727614789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc105eb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/1727614789 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc0ffad0 0x7fc1bc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.740 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 -- 172.21.15.66:0/1727614789 >> 172.21.15.66:0/1727614789 conn(0x7fc1bc0fb590 msgr2=0x7fc1bc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 -- 172.21.15.66:0/1727614789 shutdown_connections 2023-07-21T22:37:56.741 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.694+0000 7fc1c37ff700 1 -- 172.21.15.66:0/1727614789 wait complete. 2023-07-21T22:37:56.742 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.695+0000 7fc1c37ff700 1 Processor -- start 2023-07-21T22:37:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.695+0000 7fc1c37ff700 1 -- start start 2023-07-21T22:37:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c37ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc0ffad0 0x7fc1bc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c37ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc100480 0x7fc1bc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.743 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c37ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc1a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c37ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc073580 con 0x7fc1bc105ae0 2023-07-21T22:37:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c37ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1a4220 con 0x7fc1bc100480 2023-07-21T22:37:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c37ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1a4360 con 0x7fc1bc0ffad0 2023-07-21T22:37:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc0ffad0 0x7fc1bc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.744 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c1d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc1a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c159b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc0ffad0 0x7fc1bc072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:54914/0 (socket says 172.21.15.66:54914) 2023-07-21T22:37:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c1d9c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc1a3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54734/0 (socket says 172.21.15.66:54734) 2023-07-21T22:37:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c159b700 1 -- 172.21.15.66:0/2268496075 learned_addr learned my addr 172.21.15.66:0/2268496075 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.696+0000 7fc1c0d9a700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc100480 0x7fc1bc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.745 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c1d9c700 1 -- 172.21.15.66:0/2268496075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc0ffad0 msgr2=0x7fc1bc072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c1d9c700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc0ffad0 0x7fc1bc072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c1d9c700 1 -- 172.21.15.66:0/2268496075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc100480 msgr2=0x7fc1bc072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c1d9c700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc100480 0x7fc1bc072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c1d9c700 1 -- 172.21.15.66:0/2268496075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac009520 con 0x7fc1bc105ae0 2023-07-21T22:37:56.746 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c1d9c700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc1a3ce0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400efc0 tx=0x7fc1b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4018070 con 0x7fc1bc105ae0 2023-07-21T22:37:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1b400f040 con 0x7fc1bc105ae0 2023-07-21T22:37:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b80095f0 con 0x7fc1bc105ae0 2023-07-21T22:37:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400cd40 con 0x7fc1bc105ae0 2023-07-21T22:37:56.747 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.697+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc1a4900 con 0x7fc1bc105ae0 2023-07-21T22:37:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.698+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc067300 con 0x7fc1bc105ae0 2023-07-21T22:37:56.748 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.700+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fc1b4003a00 con 0x7fc1bc105ae0 2023-07-21T22:37:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.702+0000 7fc1b27fc700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc1a80a1380 0x7fc1a80a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.702+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc1b40ccb30 con 0x7fc1bc105ae0 2023-07-21T22:37:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.702+0000 7fc1c159b700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc1a80a1380 0x7fc1a80a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:56.749 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.703+0000 7fc1c159b700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc1a80a1380 0x7fc1a80a3830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8006010 tx=0x7fc1b8005960 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:56.750 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.705+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc1b4095c30 con 0x7fc1bc105ae0 2023-07-21T22:37:56.968 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.924+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1bc1a4c50 con 0x7fc1bc105ae0 2023-07-21T22:37:56.970 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.925+0000 7fc1b27fc700 1 -- 172.21.15.66:0/2268496075 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+25149 (secure 0 0 0) 0x7fc1b4099250 con 0x7fc1bc105ae0 2023-07-21T22:37:56.971 INFO:teuthology.orchestra.run.smithi066.stdout: 2023-07-21T22:37:56.971 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":114,"fsid":"6bcb348c-2813-11ee-9b34-001a4aab830c","created":"2023-07-21T22:11:36.433200+0000","modified":"2023-07-21T22:37:13.087333+0000","last_up_change":"2023-07-21T22:18:18.232740+0000","last_in_change":"2023-07-21T22:18:01.204137+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T22:14:54.912397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T22:19:05.097209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T22:19:06.344719+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-07-21T22:19:10.054857+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":113,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"165f7676-76ef-4250-b74b-48159136d422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6803","nonce":1273732852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6805","nonce":1273732852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6809","nonce":1273732852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":1273732852},{"type":"v1","addr":"172.21.15.66:6807","nonce":1273732852}]},"public_addr":"172.21.15.66:6803/1273732852","cluster_addr":"172.21.15.66:6805/1273732852","heartbeat_back_addr":"172.21.15.66:6809/1273732852","heartbeat_front_addr":"172.21.15.66:6807/1273732852","state":["exists","up"]},{"osd":1,"uuid":"77b37050-b8b8-4218-b605-b362f6375262","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6801","nonce":1799730222}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6803","nonce":1799730222}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6807","nonce":1799730222}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":1799730222},{"type":"v1","addr":"172.21.15.104:6805","nonce":1799730222}]},"public_addr":"172.21.15.104:6801/1799730222","cluster_addr":"172.21.15.104:6803/1799730222","heartbeat_back_addr":"172.21.15.104:6807/1799730222","heartbeat_front_addr":"172.21.15.104:6805/1799730222","state":["exists","up"]},{"osd":2,"uuid":"502c7d03-af4c-4cbb-ab10-7bd70bb07c12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6801","nonce":943388416}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6803","nonce":943388416}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6807","nonce":943388416}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":943388416},{"type":"v1","addr":"172.21.15.149:6805","nonce":943388416}]},"public_addr":"172.21.15.149:6801/943388416","cluster_addr":"172.21.15.149:6803/943388416","heartbeat_back_addr":"172.21.15.149:6807/943388416","heartbeat_front_addr":"172.21.15.149:6805/943388416","state":["exists","up"]},{"osd":3,"uuid":"b3f495e8-3d23-4125-96ea-0e4459f72a13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6811","nonce":4147082736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6813","nonce":4147082736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6817","nonce":4147082736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":4147082736},{"type":"v1","addr":"172.21.15.66:6815","nonce":4147082736}]},"public_addr":"172.21.15.66:6811/4147082736","cluster_addr":"172.21.15.66:6813/4147082736","heartbeat_back_addr":"172.21.15.66:6817/4147082736","heartbeat_front_addr":"172.21.15.66:6815/4147082736","state":["exists","up"]},{"osd":4,"uuid":"0864a013-76a0-4e35-91a1-2405091d259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6809","nonce":3974235171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6811","nonce":3974235171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6815","nonce":3974235171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3974235171},{"type":"v1","addr":"172.21.15.104:6813","nonce":3974235171}]},"public_addr":"172.21.15.104:6809/3974235171","cluster_addr":"172.21.15.104:6811/3974235171","heartbeat_back_addr":"172.21.15.104:6815/3974235171","heartbeat_front_addr":"172.21.15.104:6813/3974235171","state":["exists","up"]},{"osd":5,"uuid":"d8b253aa-c95f-488c-af70-0d2f6a83e718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6809","nonce":2736316265}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6811","nonce":2736316265}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6815","nonce":2736316265}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":2736316265},{"type":"v1","addr":"172.21.15.149:6813","nonce":2736316265}]},"public_addr":"172.21.15.149:6809/2736316265","cluster_addr":"172.21.15.149:6811/2736316265","heartbeat_back_addr":"172.21.15.149:6815/2736316265","heartbeat_front_addr":"172.21.15.149:6813/2736316265","state":["exists","up"]},{"osd":6,"uuid":"60ab1523-87ee-4f6c-a457-cc311b885a18","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6819","nonce":2608304075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6821","nonce":2608304075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6825","nonce":2608304075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2608304075},{"type":"v1","addr":"172.21.15.66:6823","nonce":2608304075}]},"public_addr":"172.21.15.66:6819/2608304075","cluster_addr":"172.21.15.66:6821/2608304075","heartbeat_back_addr":"172.21.15.66:6825/2608304075","heartbeat_front_addr":"172.21.15.66:6823/2608304075","state":["exists","up"]},{"osd":7,"uuid":"8896ebbe-b361-411e-a5fc-f818267ad8e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6817","nonce":1910018308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6819","nonce":1910018308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6823","nonce":1910018308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1910018308},{"type":"v1","addr":"172.21.15.104:6821","nonce":1910018308}]},"public_addr":"172.21.15.104:6817/1910018308","cluster_addr":"172.21.15.104:6819/1910018308","heartbeat_back_addr":"172.21.15.104:6823/1910018308","heartbeat_front_addr":"172.21.15.104:6821/1910018308","state":["exists","up"]},{"osd":8,"uuid":"f7e61e60-51c6-484e-a89b-3e079f2bdb15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6817","nonce":1996550641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6819","nonce":1996550641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6823","nonce":1996550641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":1996550641},{"type":"v1","addr":"172.21.15.149:6821","nonce":1996550641}]},"public_addr":"172.21.15.149:6817/1996550641","cluster_addr":"172.21.15.149:6819/1996550641","heartbeat_back_addr":"172.21.15.149:6823/1996550641","heartbeat_front_addr":"172.21.15.149:6821/1996550641","state":["exists","up"]},{"osd":9,"uuid":"a6812986-f4de-4f88-8aec-7f112164c075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6827","nonce":825018601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6829","nonce":825018601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6833","nonce":825018601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":825018601},{"type":"v1","addr":"172.21.15.66:6831","nonce":825018601}]},"public_addr":"172.21.15.66:6827/825018601","cluster_addr":"172.21.15.66:6829/825018601","heartbeat_back_addr":"172.21.15.66:6833/825018601","heartbeat_front_addr":"172.21.15.66:6831/825018601","state":["exists","up"]},{"osd":10,"uuid":"24a4c4d2-8441-4081-a35b-b758c5753b8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6825","nonce":3265864999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6827","nonce":3265864999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6831","nonce":3265864999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":3265864999},{"type":"v1","addr":"172.21.15.104:6829","nonce":3265864999}]},"public_addr":"172.21.15.104:6825/3265864999","cluster_addr":"172.21.15.104:6827/3265864999","heartbeat_back_addr":"172.21.15.104:6831/3265864999","heartbeat_front_addr":"172.21.15.104:6829/3265864999","state":["exists","up"]},{"osd":11,"uuid":"f3e27952-c49b-4512-a879-f0b53e0aea22","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6825","nonce":2237094047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6827","nonce":2237094047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6831","nonce":2237094047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2237094047},{"type":"v1","addr":"172.21.15.149:6829","nonce":2237094047}]},"public_addr":"172.21.15.149:6825/2237094047","cluster_addr":"172.21.15.149:6827/2237094047","heartbeat_back_addr":"172.21.15.149:6831/2237094047","heartbeat_front_addr":"172.21.15.149:6829/2237094047","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:08.231764+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:30.011253+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:14:51.067402+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:16.979924+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:38.515254+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:15:59.356876+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:25.382312+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:16:46.777653+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:07.728364+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:33.837180+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:17:56.250414+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T22:18:17.161687+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.10","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/4012457980":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/4235245168":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/662946744":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1497274919":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6800/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/1244827991":"2023-07-22T22:12:36.098613+0000","172.21.15.66:6801/1262827329":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6800/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6800/1262827329":"2023-07-22T22:21:28.704079+0000","172.21.15.66:6801/3065709937":"2023-07-22T22:12:36.098613+0000","172.21.15.66:0/3161416321":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3483118396":"2023-07-22T22:12:09.674594+0000","172.21.15.66:6801/979491945":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/690385827":"2023-07-22T22:12:09.674594+0000","172.21.15.66:0/3171200415":"2023-07-22T22:21:28.704079+0000","172.21.15.66:0/3486180168":"2023-07-22T22:12:36.098613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T22:37:56.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc1a80a1380 msgr2=0x7fc1a80a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc1a80a1380 0x7fc1a80a3830 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8006010 tx=0x7fc1b8005960 comp rx=0 tx=0).stop 2023-07-21T22:37:56.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 msgr2=0x7fc1bc1a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:56.973 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc1a3ce0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc1b400efc0 tx=0x7fc1b400c3a0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 shutdown_connections 2023-07-21T22:37:56.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fc1a80a1380 0x7fc1a80a3830 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fc1bc105ae0 0x7fc1bc1a3ce0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc1bc0ffad0 0x7fc1bc072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.974 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 --2- 172.21.15.66:0/2268496075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1bc100480 0x7fc1bc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 >> 172.21.15.66:0/2268496075 conn(0x7fc1bc0fb590 msgr2=0x7fc1bc0fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 shutdown_connections 2023-07-21T22:37:56.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:56.928+0000 7fc1c37ff700 1 -- 172.21.15.66:0/2268496075 wait complete. 2023-07-21T22:37:56.992 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-21T22:37:57.203 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 -- 172.21.15.66:0/725672152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 msgr2=0x7ff51410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 --2- 172.21.15.66:0/725672152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51410e700 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009f90 tx=0x7ff50c00b330 comp rx=0 tx=0).stop 2023-07-21T22:37:57.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 -- 172.21.15.66:0/725672152 shutdown_connections 2023-07-21T22:37:57.204 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 --2- 172.21.15.66:0/725672152 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51410e700 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 --2- 172.21.15.66:0/725672152 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140728b0 0x7ff514072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 --2- 172.21.15.66:0/725672152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140731c0 0x7ff51410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 -- 172.21.15.66:0/725672152 >> 172.21.15.66:0/725672152 conn(0x7ff5140fb5e0 msgr2=0x7ff5140fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:57.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 -- 172.21.15.66:0/725672152 shutdown_connections 2023-07-21T22:37:57.205 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.159+0000 7ff518dff700 1 -- 172.21.15.66:0/725672152 wait complete. 2023-07-21T22:37:57.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 Processor -- start 2023-07-21T22:37:57.207 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 -- start start 2023-07-21T22:37:57.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 0x7ff514197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.208 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 0x7ff5141984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51419cf50 con 0x7ff51410c320 2023-07-21T22:37:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51419d0c0 con 0x7ff5140728b0 2023-07-21T22:37:57.209 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.160+0000 7ff518dff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51419d280 con 0x7ff5140731c0 2023-07-21T22:37:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 0x7ff514197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff511d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 0x7ff5141984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff511d9b700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 0x7ff5141984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.66:54928/0 (socket says 172.21.15.66:54928) 2023-07-21T22:37:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 0x7ff514197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47106/0 (socket says 172.21.15.66:47106) 2023-07-21T22:37:57.210 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff512d9d700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff511d9b700 1 -- 172.21.15.66:0/3573133420 learned_addr learned my addr 172.21.15.66:0/3573133420 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 -- 172.21.15.66:0/3573133420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 msgr2=0x7ff5141984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 0x7ff5141984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 -- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 msgr2=0x7ff51419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.211 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 -- 172.21.15.66:0/3573133420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c009c40 con 0x7ff5140728b0 2023-07-21T22:37:57.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff512d9d700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:57.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff511d9b700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 0x7ff5141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:57.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.161+0000 7ff51259c700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 0x7ff514197f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9c0 tx=0x7ff50400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:57.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.162+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50400c9c0 con 0x7ff5140728b0 2023-07-21T22:37:57.212 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.162+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff504010430 con 0x7ff5140728b0 2023-07-21T22:37:57.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.162+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51419d560 con 0x7ff5140728b0 2023-07-21T22:37:57.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.162+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504018680 con 0x7ff5140728b0 2023-07-21T22:37:57.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.162+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5141a4da0 con 0x7ff5140728b0 2023-07-21T22:37:57.213 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.163+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff514106c40 con 0x7ff5140728b0 2023-07-21T22:37:57.214 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.165+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff504010860 con 0x7ff5140728b0 2023-07-21T22:37:57.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.166+0000 7ff5037fe700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4fc0a1340 0x7ff4fc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.166+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff504014070 con 0x7ff5140728b0 2023-07-21T22:37:57.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.166+0000 7ff511d9b700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4fc0a1340 0x7ff4fc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.167+0000 7ff511d9b700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4fc0a1340 0x7ff4fc0a37f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff514199580 tx=0x7ff508009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:57.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.170+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff504051770 con 0x7ff5140728b0 2023-07-21T22:37:57.444 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:57 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/293191514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:37:57.445 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:57 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/2268496075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:57.471 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.426+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff514067290 con 0x7ff5140728b0 2023-07-21T22:37:57.481 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.437+0000 7ff5037fe700 1 -- 172.21.15.66:0/3573133420 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff5040960d0 con 0x7ff5140728b0 2023-07-21T22:37:57.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4fc0a1340 msgr2=0x7ff4fc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4fc0a1340 0x7ff4fc0a37f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff514199580 tx=0x7ff508009260 comp rx=0 tx=0).stop 2023-07-21T22:37:57.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 msgr2=0x7ff514197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 0x7ff514197f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9c0 tx=0x7ff50400ecd0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 shutdown_connections 2023-07-21T22:37:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5140728b0 0x7ff514197f70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7ff4fc0a1340 0x7ff4fc0a37f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7ff51410c320 0x7ff51419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 --2- 172.21.15.66:0/3573133420 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5140731c0 0x7ff5141984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 >> 172.21.15.66:0/3573133420 conn(0x7ff5140fb5e0 msgr2=0x7ff514103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.439+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 shutdown_connections 2023-07-21T22:37:57.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.440+0000 7ff518dff700 1 -- 172.21.15.66:0/3573133420 wait complete. 2023-07-21T22:37:57.487 INFO:teuthology.orchestra.run.smithi066.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-21T22:37:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/293191514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:37:57.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/2268496075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:57.503 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-21T22:37:57.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/293191514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T22:37:57.519 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/2268496075' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T22:37:57.520 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e[153688]: ignoring --setuser ceph since I am not root 2023-07-21T22:37:57.521 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e[153688]: ignoring --setgroup ceph since I am not root 2023-07-21T22:37:57.716 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 -- 172.21.15.66:0/4128554481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 msgr2=0x7fbf3c105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 --2- 172.21.15.66:0/4128554481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c105ec0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009990 tx=0x7fbf2c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 -- 172.21.15.66:0/4128554481 shutdown_connections 2023-07-21T22:37:57.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 --2- 172.21.15.66:0/4128554481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c105ec0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.717 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 --2- 172.21.15.66:0/4128554481 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c100490 0x7fbf3c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 --2- 172.21.15.66:0/4128554481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.671+0000 7fbf41bff700 1 -- 172.21.15.66:0/4128554481 >> 172.21.15.66:0/4128554481 conn(0x7fbf3c0fb5e0 msgr2=0x7fbf3c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:57.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.672+0000 7fbf41bff700 1 -- 172.21.15.66:0/4128554481 shutdown_connections 2023-07-21T22:37:57.718 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.672+0000 7fbf41bff700 1 -- 172.21.15.66:0/4128554481 wait complete. 2023-07-21T22:37:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 Processor -- start 2023-07-21T22:37:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 -- start start 2023-07-21T22:37:57.720 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3c100490 0x7fbf3c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c1a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c073580 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c1a4340 con 0x7fbf3c105af0 2023-07-21T22:37:57.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf41bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c1a4480 con 0x7fbf3c100490 2023-07-21T22:37:57.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf3b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf3b7fe700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54772/0 (socket says 172.21.15.66:54772) 2023-07-21T22:37:57.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.673+0000 7fbf3b7fe700 1 -- 172.21.15.66:0/3278168010 learned_addr learned my addr 172.21.15.66:0/3278168010 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:57.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3bfff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c1a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3affd700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3c100490 0x7fbf3c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3b7fe700 1 -- 172.21.15.66:0/3278168010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3c100490 msgr2=0x7fbf3c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3b7fe700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3c100490 0x7fbf3c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3b7fe700 1 -- 172.21.15.66:0/3278168010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 msgr2=0x7fbf3c1a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:57.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3b7fe700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c1a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:57.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3b7fe700 1 -- 172.21.15.66:0/3278168010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c0095f0 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3bfff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c1a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:37:57.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.674+0000 7fbf3b7fe700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c072810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005910 tx=0x7fbf2c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:57.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.675+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c01d070 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.675+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c1a4680 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.675+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf2c022470 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.675+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c017650 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.675+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c1a4aa0 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.676+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c067290 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.678+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbf2c0225e0 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.679+0000 7fbf38ff9700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fbf240a12c0 0x7fbf240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:57.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.679+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9623+0+0 (secure 0 0 0) 0x7fbf2c0ce4d0 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.679+0000 7fbf3affd700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fbf240a12c0 0x7fbf240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:57.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.680+0000 7fbf3affd700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fbf240a12c0 0x7fbf240a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c074000 tx=0x7fbf30009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:57.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.683+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbf2c0974c0 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.868 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-mds[153713]: -- 172.21.15.104:0/148466871 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0cc8011e0 con 0x55a0cc7d7000 2023-07-21T22:37:57.868 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e[153688]: starting mds.e at 2023-07-21T22:37:57.869 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h[154032]: ignoring --setuser ceph since I am not root 2023-07-21T22:37:57.869 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h[154032]: ignoring --setgroup ceph since I am not root 2023-07-21T22:37:57.869 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-mds[154057]: -- 172.21.15.104:0/2138285121 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5654c8f8b1e0 con 0x5654c8f61000 2023-07-21T22:37:57.869 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h[154032]: starting mds.h at 2023-07-21T22:37:57.870 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: ignoring --setuser ceph since I am not root 2023-07-21T22:37:57.870 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: ignoring --setgroup ceph since I am not root 2023-07-21T22:37:57.870 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-mds[154397]: -- 172.21.15.104:0/402960561 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56282a8b71e0 con 0x56282a88cc00 2023-07-21T22:37:57.871 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:37:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: starting mds.b at 2023-07-21T22:37:57.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a[126355]: 2023-07-21T22:37:57.428+0000 7f9d83e5e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:37:57.983 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:57.938+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbf3c1a4dc0 con 0x7fbf3c0ffae0 2023-07-21T22:37:57.989 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c[155099]: ignoring --setuser ceph since I am not root 2023-07-21T22:37:57.989 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c[155099]: ignoring --setgroup ceph since I am not root 2023-07-21T22:37:57.989 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-mds[155123]: -- 172.21.15.149:0/3159392744 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a75b4ad1e0 con 0x55a75b483000 2023-07-21T22:37:57.989 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c[155099]: starting mds.c at 2023-07-21T22:37:57.990 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f[154757]: ignoring --setuser ceph since I am not root 2023-07-21T22:37:57.990 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f[154757]: ignoring --setgroup ceph since I am not root 2023-07-21T22:37:57.990 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-mds[154781]: -- 172.21.15.149:0/2663310406 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56152bc9f1e0 con 0x56152bc74c00 2023-07-21T22:37:57.991 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f[154757]: starting mds.f at 2023-07-21T22:37:57.991 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: ignoring --setuser ceph since I am not root 2023-07-21T22:37:57.991 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: ignoring --setgroup ceph since I am not root 2023-07-21T22:37:57.991 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-mds[155463]: -- 172.21.15.149:0/3372868329 <== mon.1 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558dce21f1e0 con 0x558dce1f4800 2023-07-21T22:37:57.992 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:37:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: starting mds.i at 2023-07-21T22:37:58.480 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.436+0000 7fbf38ff9700 1 -- 172.21.15.66:0/3278168010 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf2c09aae0 con 0x7fbf3c0ffae0 2023-07-21T22:37:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fbf240a12c0 msgr2=0x7fbf240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:58.484 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fbf240a12c0 0x7fbf240a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c074000 tx=0x7fbf30009260 comp rx=0 tx=0).stop 2023-07-21T22:37:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 msgr2=0x7fbf3c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c072810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c005910 tx=0x7fbf2c005720 comp rx=0 tx=0).stop 2023-07-21T22:37:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 shutdown_connections 2023-07-21T22:37:58.485 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf3c105af0 0x7fbf3c1a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fbf240a12c0 0x7fbf240a3770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fbf3c0ffae0 0x7fbf3c072810 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 --2- 172.21.15.66:0/3278168010 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf3c100490 0x7fbf3c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.486 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 >> 172.21.15.66:0/3278168010 conn(0x7fbf3c0fb5e0 msgr2=0x7fbf3c0fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 shutdown_connections 2023-07-21T22:37:58.487 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.439+0000 7fbf41bff700 1 -- 172.21.15.66:0/3278168010 wait complete. 2023-07-21T22:37:58.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 162 KiB/s wr, 9 op/s 2023-07-21T22:37:58.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3573133420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:37:58.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:37:58.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:37:58.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:37:58.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-21T22:37:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T22:37:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:37:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-21T22:37:58.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:58 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3278168010' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:37:58.504 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 162 KiB/s wr, 9 op/s 2023-07-21T22:37:58.504 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3573133420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:37:58.504 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:37:58.505 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:37:58.505 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:37:58.505 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-21T22:37:58.505 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T22:37:58.505 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:37:58.506 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-21T22:37:58.506 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:58 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3278168010' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:37:58.507 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-21T22:37:58.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 162 KiB/s wr, 9 op/s 2023-07-21T22:37:58.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3573133420' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:37:58.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-07-21T22:37:58.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-07-21T22:37:58.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-21T22:37:58.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-07-21T22:37:58.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T22:37:58.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-07-21T22:37:58.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-07-21T22:37:58.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:58 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3278168010' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T22:37:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.676+0000 7f4590bff700 1 -- 172.21.15.66:0/784809027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 msgr2=0x7f458c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:58.721 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.676+0000 7f4590bff700 1 --2- 172.21.15.66:0/784809027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c072c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f457c009990 tx=0x7f457c009ca0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 -- 172.21.15.66:0/784809027 shutdown_connections 2023-07-21T22:37:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 --2- 172.21.15.66:0/784809027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c10c310 0x7f458c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 --2- 172.21.15.66:0/784809027 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c072c80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.722 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 --2- 172.21.15.66:0/784809027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f458c0731c0 0x7f458c10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 -- 172.21.15.66:0/784809027 >> 172.21.15.66:0/784809027 conn(0x7f458c0fb5f0 msgr2=0x7f458c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 -- 172.21.15.66:0/784809027 shutdown_connections 2023-07-21T22:37:58.723 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.677+0000 7f4590bff700 1 -- 172.21.15.66:0/784809027 wait complete. 2023-07-21T22:37:58.724 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 Processor -- start 2023-07-21T22:37:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 -- start start 2023-07-21T22:37:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f458c0731c0 0x7f458c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c10c310 0x7f458c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:58.725 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19ce20 con 0x7f458c0728b0 2023-07-21T22:37:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19cf90 con 0x7f458c10c310 2023-07-21T22:37:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.678+0000 7f4590bff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c19d150 con 0x7f458c0731c0 2023-07-21T22:37:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54796/0 (socket says 172.21.15.66:54796) 2023-07-21T22:37:58.726 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 -- 172.21.15.66:0/850933938 learned_addr learned my addr 172.21.15.66:0/850933938 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458ad9d700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c10c310 0x7f458c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f4589d9b700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f458c0731c0 0x7f458c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 -- 172.21.15.66:0/850933938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f458c0731c0 msgr2=0x7f458c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f458c0731c0 0x7f458c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.727 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 -- 172.21.15.66:0/850933938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c10c310 msgr2=0x7f458c19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c10c310 0x7f458c19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 -- 172.21.15.66:0/850933938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c0095f0 con 0x7f458c0728b0 2023-07-21T22:37:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f458a59c700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c197e40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f457c005910 tx=0x7f457c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.679+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c01d070 con 0x7f458c0728b0 2023-07-21T22:37:58.728 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.680+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c19d3d0 con 0x7f458c0728b0 2023-07-21T22:37:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.680+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f457c022470 con 0x7f458c0728b0 2023-07-21T22:37:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.680+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c017650 con 0x7f458c0728b0 2023-07-21T22:37:58.729 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.680+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c1a4c70 con 0x7f458c0728b0 2023-07-21T22:37:58.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.681+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c067290 con 0x7f458c0728b0 2023-07-21T22:37:58.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.683+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f457c00fad0 con 0x7f458c0728b0 2023-07-21T22:37:58.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.684+0000 7f457b7fe700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f45740a12c0 0x7f45740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:58.730 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.684+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9623+0+0 (secure 0 0 0) 0x7f457c0ce9f0 con 0x7f458c0728b0 2023-07-21T22:37:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.684+0000 7f4589d9b700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f45740a12c0 0x7f45740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.685+0000 7f4589d9b700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f45740a12c0 0x7f45740a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f458c199450 tx=0x7f4580009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:58.731 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.686+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f457c0979e0 con 0x7f458c0728b0 2023-07-21T22:37:58.940 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:58.896+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f458c199190 con 0x7f458c0728b0 2023-07-21T22:37:59.488 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.444+0000 7f457b7fe700 1 -- 172.21.15.66:0/850933938 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7f457c09b000 con 0x7f458c0728b0 2023-07-21T22:37:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:37:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:37:59.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: Cluster is now healthy 2023-07-21T22:37:59.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6834/304165541,v1:172.21.15.104:6835/304165541] up:boot 2023-07-21T22:37:59.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6837/831169842,v1:172.21.15.104:6839/831169842] up:boot 2023-07-21T22:37:59.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.104:6836/2179879939,v1:172.21.15.104:6838/2179879939] up:boot 2023-07-21T22:37:59.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6836/3929990676,v1:172.21.15.149:6838/3929990676] up:boot 2023-07-21T22:37:59.490 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6834/881812108,v1:172.21.15.149:6835/881812108] up:boot 2023-07-21T22:37:59.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: mds.? [v2:172.21.15.149:6837/2500097872,v1:172.21.15.149:6839/2500097872] up:boot 2023-07-21T22:37:59.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3278168010' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:37:59.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: fsmap 12 up:standby 2023-07-21T22:37:59.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:37:59.491 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:37:59.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:37:59.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:37:59.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:37:59.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:37:59.492 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:37:59 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/850933938' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:37:59.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f45740a12c0 msgr2=0x7f45740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:59.493 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f45740a12c0 0x7f45740a3770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f458c199450 tx=0x7f4580009260 comp rx=0 tx=0).stop 2023-07-21T22:37:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 msgr2=0x7f458c197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c197e40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f457c005910 tx=0x7f457c004730 comp rx=0 tx=0).stop 2023-07-21T22:37:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 shutdown_connections 2023-07-21T22:37:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f458c10c310 0x7f458c19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.494 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f45740a12c0 0x7f45740a3770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f458c0728b0 0x7f458c197e40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 --2- 172.21.15.66:0/850933938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f458c0731c0 0x7f458c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.448+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 >> 172.21.15.66:0/850933938 conn(0x7f458c0fb5f0 msgr2=0x7f458c10e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.449+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 shutdown_connections 2023-07-21T22:37:59.495 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.449+0000 7f4590bff700 1 -- 172.21.15.66:0/850933938 wait complete. 2023-07-21T22:37:59.496 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_metadata' removed 2023-07-21T22:37:59.521 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:37:59.521 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:37:59.521 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: Cluster is now healthy 2023-07-21T22:37:59.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6834/304165541,v1:172.21.15.104:6835/304165541] up:boot 2023-07-21T22:37:59.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6837/831169842,v1:172.21.15.104:6839/831169842] up:boot 2023-07-21T22:37:59.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.104:6836/2179879939,v1:172.21.15.104:6838/2179879939] up:boot 2023-07-21T22:37:59.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6836/3929990676,v1:172.21.15.149:6838/3929990676] up:boot 2023-07-21T22:37:59.522 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6834/881812108,v1:172.21.15.149:6835/881812108] up:boot 2023-07-21T22:37:59.523 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: mds.? [v2:172.21.15.149:6837/2500097872,v1:172.21.15.149:6839/2500097872] up:boot 2023-07-21T22:37:59.523 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3278168010' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:37:59.523 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: fsmap 12 up:standby 2023-07-21T22:37:59.523 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:37:59.523 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:37:59.523 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:37:59.524 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:37:59.524 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:37:59.524 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:37:59.524 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:37:59 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/850933938' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:37:59.525 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-21T22:37:59.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-07-21T22:37:59.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-07-21T22:37:59.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: Cluster is now healthy 2023-07-21T22:37:59.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6834/304165541,v1:172.21.15.104:6835/304165541] up:boot 2023-07-21T22:37:59.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6837/831169842,v1:172.21.15.104:6839/831169842] up:boot 2023-07-21T22:37:59.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.104:6836/2179879939,v1:172.21.15.104:6838/2179879939] up:boot 2023-07-21T22:37:59.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6836/3929990676,v1:172.21.15.149:6838/3929990676] up:boot 2023-07-21T22:37:59.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6834/881812108,v1:172.21.15.149:6835/881812108] up:boot 2023-07-21T22:37:59.620 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: mds.? [v2:172.21.15.149:6837/2500097872,v1:172.21.15.149:6839/2500097872] up:boot 2023-07-21T22:37:59.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3278168010' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-07-21T22:37:59.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: fsmap 12 up:standby 2023-07-21T22:37:59.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T22:37:59.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T22:37:59.621 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T22:37:59.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T22:37:59.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T22:37:59.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='mgr.15483 172.21.15.66:0/4223004714' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T22:37:59.622 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:37:59 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/850933938' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:37:59.945 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.900+0000 7fd40b1ff700 1 -- 172.21.15.66:0/4003912156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40406d3a0 msgr2=0x7fd40411b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:59.946 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.900+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/4003912156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40406d3a0 0x7fd40411b470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec01a9a0 tx=0x7fd3ec01acb0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 -- 172.21.15.66:0/4003912156 shutdown_connections 2023-07-21T22:37:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/4003912156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd40410e280 0x7fd40410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.948 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/4003912156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40406d3a0 0x7fd40411b470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/4003912156 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd40406ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 -- 172.21.15.66:0/4003912156 >> 172.21.15.66:0/4003912156 conn(0x7fd40406bf70 msgr2=0x7fd404070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:37:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 -- 172.21.15.66:0/4003912156 shutdown_connections 2023-07-21T22:37:59.949 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.903+0000 7fd40b1ff700 1 -- 172.21.15.66:0/4003912156 wait complete. 2023-07-21T22:37:59.950 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.905+0000 7fd40b1ff700 1 Processor -- start 2023-07-21T22:37:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.905+0000 7fd40b1ff700 1 -- start start 2023-07-21T22:37:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.906+0000 7fd40b1ff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd404116d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.906+0000 7fd40b1ff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd40406d3a0 0x7fd404117240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:59.952 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.906+0000 7fd40b1ff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40410e280 0x7fd4041a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.906+0000 7fd40b1ff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041a8480 con 0x7fd40406c9a0 2023-07-21T22:37:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.906+0000 7fd40b1ff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041a85f0 con 0x7fd40406d3a0 2023-07-21T22:37:59.953 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.906+0000 7fd40b1ff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041a87b0 con 0x7fd40410e280 2023-07-21T22:37:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd40979c700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40410e280 0x7fd4041a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd404116d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd403fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd40406d3a0 0x7fd404117240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:59.954 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd404116d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.66:3300/0 says I am v2:172.21.15.66:54814/0 (socket says 172.21.15.66:54814) 2023-07-21T22:37:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 -- 172.21.15.66:0/1256932897 learned_addr learned my addr 172.21.15.66:0/1256932897 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:37:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 -- 172.21.15.66:0/1256932897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40410e280 msgr2=0x7fd4041a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40410e280 0x7fd4041a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 -- 172.21.15.66:0/1256932897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd40406d3a0 msgr2=0x7fd404117240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:37:59.955 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd40406d3a0 0x7fd404117240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:37:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.908+0000 7fd408f9b700 1 -- 172.21.15.66:0/1256932897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec01a600 con 0x7fd40406c9a0 2023-07-21T22:37:59.956 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.910+0000 7fd408f9b700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd404116d00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00ada0 tx=0x7fd3fc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:59.958 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.914+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00f070 con 0x7fd40406c9a0 2023-07-21T22:37:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.914+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3fc00ae90 con 0x7fd40406c9a0 2023-07-21T22:37:59.959 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.914+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0053e0 con 0x7fd40406c9a0 2023-07-21T22:37:59.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.918+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4041a8a90 con 0x7fd40406c9a0 2023-07-21T22:37:59.963 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.919+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041a8f20 con 0x7fd40406c9a0 2023-07-21T22:37:59.966 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.922+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd404067290 con 0x7fd40406c9a0 2023-07-21T22:37:59.975 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.927+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fd3fc00c980 con 0x7fd40406c9a0 2023-07-21T22:37:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.929+0000 7fd401ffb700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd3f00a14d0 0x7fd3f00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:37:59.976 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.929+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9098+0+0 (secure 0 0 0) 0x7fd3fc0cc0b0 con 0x7fd40406c9a0 2023-07-21T22:37:59.977 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.933+0000 7fd403fff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd3f00a14d0 0x7fd3f00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:37:59.978 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.934+0000 7fd403fff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd3f00a14d0 0x7fd3f00a3980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd404118310 tx=0x7fd3f4009190 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:37:59.985 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:37:59.936+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd3fc04c770 con 0x7fd40406c9a0 2023-07-21T22:38:00.239 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:00.194+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd404118050 con 0x7fd40406c9a0 2023-07-21T22:38:00.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:00 smithi149 ceph-mon[129268]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-07-21T22:38:00.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:00 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/850933938' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:38:00.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:00 smithi149 ceph-mon[129268]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T22:38:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:00 smithi104 ceph-mon[129021]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-07-21T22:38:00.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:00 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/850933938' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:38:00.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:00 smithi104 ceph-mon[129021]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T22:38:00.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:38:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:38:00.289+0000 7f75bd582700 -1 client.25549: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2023-07-21T22:38:00.695 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:38:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:38:00.290+0000 7f75bd582700 -1 client.25549: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2023-07-21T22:38:00.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:00 smithi066 ceph-mon[126379]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 195 KiB/s wr, 10 op/s 2023-07-21T22:38:00.696 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:00 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/850933938' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:38:00.697 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:00 smithi066 ceph-mon[126379]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T22:38:01.215 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.171+0000 7fd401ffb700 1 -- 172.21.15.66:0/1256932897 <== mon.0 v2:172.21.15.66:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd3fc095220 con 0x7fd40406c9a0 2023-07-21T22:38:01.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.175+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd3f00a14d0 msgr2=0x7fd3f00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:01.219 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd3f00a14d0 0x7fd3f00a3980 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd404118310 tx=0x7fd3f4009190 comp rx=0 tx=0).stop 2023-07-21T22:38:01.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 msgr2=0x7fd404116d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:01.220 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd404116d00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00ada0 tx=0x7fd3fc00c330 comp rx=0 tx=0).stop 2023-07-21T22:38:01.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 shutdown_connections 2023-07-21T22:38:01.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7fd3f00a14d0 0x7fd3f00a3980 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.222 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd40410e280 0x7fd4041a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7fd40406c9a0 0x7fd404116d00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.223 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 --2- 172.21.15.66:0/1256932897 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd40406d3a0 0x7fd404117240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 >> 172.21.15.66:0/1256932897 conn(0x7fd40406bf70 msgr2=0x7fd40406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:01.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 shutdown_connections 2023-07-21T22:38:01.224 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.176+0000 7fd40b1ff700 1 -- 172.21.15.66:0/1256932897 wait complete. 2023-07-21T22:38:01.225 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data' removed 2023-07-21T22:38:01.244 DEBUG:teuthology.orchestra.run.smithi066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-07-21T22:38:01.458 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 -- 172.21.15.66:0/2813615866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 msgr2=0x7f5d88105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:01.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/2813615866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d88105ec0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009990 tx=0x7f5d84009ca0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 -- 172.21.15.66:0/2813615866 shutdown_connections 2023-07-21T22:38:01.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/2813615866 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88100490 0x7f5d8810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.459 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/2813615866 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d88105ec0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/2813615866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d880fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.414+0000 7f5d8fdff700 1 -- 172.21.15.66:0/2813615866 >> 172.21.15.66:0/2813615866 conn(0x7f5d880fb5e0 msgr2=0x7f5d880fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.415+0000 7f5d8fdff700 1 -- 172.21.15.66:0/2813615866 shutdown_connections 2023-07-21T22:38:01.460 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.415+0000 7f5d8fdff700 1 -- 172.21.15.66:0/2813615866 wait complete. 2023-07-21T22:38:01.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.415+0000 7f5d8fdff700 1 Processor -- start 2023-07-21T22:38:01.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 -- start start 2023-07-21T22:38:01.462 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d88197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:01.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88100490 0x7f5d881983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:01.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d8819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:01.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 -- --> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819ce40 con 0x7f5d88105af0 2023-07-21T22:38:01.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819cfb0 con 0x7f5d880ffae0 2023-07-21T22:38:01.463 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8fdff700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8819d170 con 0x7f5d88100490 2023-07-21T22:38:01.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d88197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:01.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8e39c700 1 --2- >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d8819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:01.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8db9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d88197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.66:47176/0 (socket says 172.21.15.66:47176) 2023-07-21T22:38:01.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8db9b700 1 -- 172.21.15.66:0/3640353017 learned_addr learned my addr 172.21.15.66:0/3640353017 (peer_addr_for_me v2:172.21.15.66:0/0) 2023-07-21T22:38:01.464 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.416+0000 7f5d8d39a700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88100490 0x7f5d881983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:01.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8db9b700 1 -- 172.21.15.66:0/3640353017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88100490 msgr2=0x7f5d881983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:01.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8db9b700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88100490 0x7f5d881983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8db9b700 1 -- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 msgr2=0x7f5d8819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:01.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8db9b700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d8819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:01.465 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8db9b700 1 -- 172.21.15.66:0/3640353017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d840095f0 con 0x7f5d880ffae0 2023-07-21T22:38:01.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8e39c700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d8819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T22:38:01.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8db9b700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d88197e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5d84005910 tx=0x7f5d84005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:01.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8401d070 con 0x7f5d880ffae0 2023-07-21T22:38:01.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d84022470 con 0x7f5d880ffae0 2023-07-21T22:38:01.466 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8819d3f0 con 0x7f5d880ffae0 2023-07-21T22:38:01.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.417+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84017650 con 0x7f5d880ffae0 2023-07-21T22:38:01.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.418+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d881a4c90 con 0x7f5d880ffae0 2023-07-21T22:38:01.467 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.418+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88067290 con 0x7f5d880ffae0 2023-07-21T22:38:01.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.421+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 26) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5d840225e0 con 0x7f5d880ffae0 2023-07-21T22:38:01.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.422+0000 7f5d7effd700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5d740a12c0 0x7f5d740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T22:38:01.468 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.422+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8510+0+0 (secure 0 0 0) 0x7f5d840ce7a0 con 0x7f5d880ffae0 2023-07-21T22:38:01.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.422+0000 7f5d8d39a700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5d740a12c0 0x7f5d740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T22:38:01.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.423+0000 7f5d8d39a700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5d740a12c0 0x7f5d740a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5d88199470 tx=0x7f5d78009260 comp rx=0 tx=0).ready entity=mgr.15483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T22:38:01.469 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.424+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5d84097bf0 con 0x7f5d880ffae0 2023-07-21T22:38:01.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:01 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1256932897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:01.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:01 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1256932897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:01.675 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:01.631+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d881a4e20 con 0x7f5d880ffae0 2023-07-21T22:38:01.694 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:01 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1256932897' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.221 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.178+0000 7f5d7effd700 1 -- 172.21.15.66:0/3640353017 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v118) v1 ==== 178+0+0 (secure 0 0 0) 0x7f5d8409b210 con 0x7f5d880ffae0 2023-07-21T22:38:02.225 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5d740a12c0 msgr2=0x7f5d740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:02.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5d740a12c0 0x7f5d740a3770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5d88199470 tx=0x7f5d78009260 comp rx=0 tx=0).stop 2023-07-21T22:38:02.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 msgr2=0x7f5d88197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T22:38:02.226 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d88197e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5d84005910 tx=0x7f5d84005720 comp rx=0 tx=0).stop 2023-07-21T22:38:02.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 shutdown_connections 2023-07-21T22:38:02.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d880ffae0 0x7f5d88197e60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:02.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:6800/3475780841,v1:172.21.15.66:6801/3475780841] conn(0x7f5d740a12c0 0x7f5d740a3770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:02.227 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] conn(0x7f5d88105af0 0x7f5d8819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:02.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 --2- 172.21.15.66:0/3640353017 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d88100490 0x7f5d881983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T22:38:02.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 >> 172.21.15.66:0/3640353017 conn(0x7f5d880fb5e0 msgr2=0x7f5d880fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T22:38:02.228 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 shutdown_connections 2023-07-21T22:38:02.229 INFO:teuthology.orchestra.run.smithi066.stderr:2023-07-21T22:38:02.182+0000 7f5d8fdff700 1 -- 172.21.15.66:0/3640353017 wait complete. 2023-07-21T22:38:02.230 INFO:teuthology.orchestra.run.smithi066.stderr:pool 'cephfs_data_ec' removed 2023-07-21T22:38:02.251 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:38:02.323 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:38:02.370 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:38:02.419 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-21T22:38:02.420 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:38:02.465 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:38:02.482 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T22:38:02.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:02 smithi149 ceph-mon[129268]: pgmap v521: 97 pgs: 97 active+clean; 304 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:38:02.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:02 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/1256932897' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:38:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:02 smithi149 ceph-mon[129268]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T22:38:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:02 smithi149 ceph-mon[129268]: from='client.? 172.21.15.66:0/3640353017' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.489 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:02 smithi149 ceph-mon[129268]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.501 INFO:tasks.cephadm:Stopping all daemons... 2023-07-21T22:38:02.502 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-21T22:38:02.502 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a 2023-07-21T22:38:02.567 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-mon[126379]: pgmap v521: 97 pgs: 97 active+clean; 304 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:38:02.567 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/1256932897' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:38:02.568 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-mon[126379]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T22:38:02.568 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-mon[126379]: from='client.? 172.21.15.66:0/3640353017' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.568 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-mon[126379]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:02 smithi104 ceph-mon[129021]: pgmap v521: 97 pgs: 97 active+clean; 304 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T22:38:02.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:02 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/1256932897' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-07-21T22:38:02.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:02 smithi104 ceph-mon[129021]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T22:38:02.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:02 smithi104 ceph-mon[129021]: from='client.? 172.21.15.66:0/3640353017' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.619 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:02 smithi104 ceph-mon[129021]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-07-21T22:38:02.948 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 systemd[1]: Stopping Ceph mon.a for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:03.420 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a[126355]: 2023-07-21T22:38:02.981+0000 7f9d89669700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:03.420 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a[126355]: 2023-07-21T22:38:02.981+0000 7f9d89669700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-21T22:38:03.695 INFO:journalctl@ceph.mon.a.smithi066.stdout:Jul 21 22:38:03 smithi066 bash[192360]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-a 2023-07-21T22:38:03.800 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-21T22:38:03.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:03.905 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-21T22:38:03.906 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-21T22:38:03.906 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.b 2023-07-21T22:38:04.295 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:03 smithi104 systemd[1]: Stopping Ceph mon.b for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:04.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-b[128997]: 2023-07-21T22:38:04.251+0000 7ffa135cb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:04.618 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-b[128997]: 2023-07-21T22:38:04.251+0000 7ffa135cb700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-21T22:38:05.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Jul 21 22:38:04 smithi104 bash[161410]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-b 2023-07-21T22:38:05.054 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-21T22:38:05.105 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:05.106 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-21T22:38:05.106 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-21T22:38:05.107 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.c 2023-07-21T22:38:05.488 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:05 smithi149 systemd[1]: Stopping Ceph mon.c for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:05.784 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-c[129221]: 2023-07-21T22:38:05.481+0000 7f1dba9d4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:05.785 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-c[129221]: 2023-07-21T22:38:05.481+0000 7f1dba9d4700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-21T22:38:06.238 INFO:journalctl@ceph.mon.c.smithi149.stdout:Jul 21 22:38:05 smithi149 bash[163344]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mon-c 2023-07-21T22:38:06.282 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-21T22:38:06.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:06.350 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-21T22:38:06.350 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-21T22:38:06.351 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T22:38:06.694 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:38:06 smithi066 systemd[1]: Stopping Ceph mgr.x for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:07.156 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:38:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:38:06.838+0000 7f765c343700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:07.156 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:38:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x[126626]: 2023-07-21T22:38:06.838+0000 7f765c343700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-21T22:38:07.444 INFO:journalctl@ceph.mgr.x.smithi066.stdout:Jul 21 22:38:07 smithi066 bash[192502]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-x 2023-07-21T22:38:07.605 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-21T22:38:07.694 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:07.695 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-21T22:38:07.695 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-21T22:38:07.696 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.y 2023-07-21T22:38:08.117 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:38:07 smithi104 systemd[1]: Stopping Ceph mgr.y for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:08.368 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:38:08 smithi104 bash[161555]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-y 2023-07-21T22:38:08.368 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Jul 21 22:38:08 smithi104 systemd[1]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-21T22:38:08.584 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-21T22:38:08.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:08.620 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-21T22:38:08.621 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-21T22:38:08.621 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.z 2023-07-21T22:38:08.989 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:38:08 smithi149 systemd[1]: Stopping Ceph mgr.z for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:09.611 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:38:09 smithi149 bash[163491]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mgr-z 2023-07-21T22:38:09.612 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:38:09 smithi149 systemd[1]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-21T22:38:09.615 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-21T22:38:09.639 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:38:09 smithi149 systemd[1]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mgr.z.service: Failed with result 'exit-code'. 2023-07-21T22:38:09.640 INFO:journalctl@ceph.mgr.z.smithi149.stdout:Jul 21 22:38:09 smithi149 systemd[1]: Stopped Ceph mgr.z for 6bcb348c-2813-11ee-9b34-001a4aab830c. 2023-07-21T22:38:09.652 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:09.652 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-21T22:38:09.653 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-21T22:38:09.653 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.0 2023-07-21T22:38:10.195 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:38:09 smithi066 systemd[1]: Stopping Ceph osd.0 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:10.694 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:38:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-0[138663]: 2023-07-21T22:38:10.263+0000 7fd371260700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:10.695 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:38:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-0[138663]: 2023-07-21T22:38:10.263+0000 7fd371260700 -1 osd.0 118 *** Got signal Terminated *** 2023-07-21T22:38:10.695 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:38:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-0[138663]: 2023-07-21T22:38:10.263+0000 7fd371260700 -1 osd.0 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:16.445 INFO:journalctl@ceph.osd.0.smithi066.stdout:Jul 21 22:38:16 smithi066 bash[192647]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-0 2023-07-21T22:38:18.271 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-21T22:38:18.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:18.366 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-21T22:38:18.366 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-21T22:38:18.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.1 2023-07-21T22:38:18.724 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:38:18 smithi104 systemd[1]: Stopping Ceph osd.1 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:19.117 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:38:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1[134104]: 2023-07-21T22:38:18.679+0000 7f1ca4cd8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:19.118 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:38:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1[134104]: 2023-07-21T22:38:18.679+0000 7f1ca4cd8700 -1 osd.1 118 *** Got signal Terminated *** 2023-07-21T22:38:19.118 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:38:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1[134104]: 2023-07-21T22:38:18.679+0000 7f1ca4cd8700 -1 osd.1 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:24.867 INFO:journalctl@ceph.osd.1.smithi104.stdout:Jul 21 22:38:24 smithi104 bash[161701]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-1 2023-07-21T22:38:25.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-21T22:38:25.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:25.912 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-21T22:38:25.913 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-21T22:38:25.913 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.2 2023-07-21T22:38:26.238 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:38:25 smithi149 systemd[1]: Stopping Ceph osd.2 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:26.738 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:38:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2[135042]: 2023-07-21T22:38:26.252+0000 7f87eb3ac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:26.738 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:38:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2[135042]: 2023-07-21T22:38:26.252+0000 7f87eb3ac700 -1 osd.2 118 *** Got signal Terminated *** 2023-07-21T22:38:26.738 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:38:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2[135042]: 2023-07-21T22:38:26.252+0000 7f87eb3ac700 -1 osd.2 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:32.238 INFO:journalctl@ceph.osd.2.smithi149.stdout:Jul 21 22:38:32 smithi149 bash[163642]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-2 2023-07-21T22:38:33.619 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-21T22:38:33.660 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:33.660 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-21T22:38:33.661 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-21T22:38:33.661 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.3 2023-07-21T22:38:34.187 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:38:33 smithi066 systemd[1]: Stopping Ceph osd.3 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:34.444 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:38:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-3[144116]: 2023-07-21T22:38:34.144+0000 7f5be4151700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:34.444 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:38:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-3[144116]: 2023-07-21T22:38:34.144+0000 7f5be4151700 -1 osd.3 118 *** Got signal Terminated *** 2023-07-21T22:38:34.444 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:38:34 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-3[144116]: 2023-07-21T22:38:34.144+0000 7f5be4151700 -1 osd.3 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:37.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:37.595+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:37.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:37.530+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:38.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:38.533+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:38.989 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:38.580+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:38.989 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:38.633+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:39.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:39.514+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:39.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:39.590+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:39.989 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:39.640+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:40.135 INFO:journalctl@ceph.osd.3.smithi066.stdout:Jul 21 22:38:39 smithi066 bash[192932]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-3 2023-07-21T22:38:40.136 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:39 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:39.768+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:40.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:40 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:40.025+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:40.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:40.623+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:40.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:40.480+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:40.989 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:40.639+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:41.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:40 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:40.771+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:41.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:41 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:41.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:41.737 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:41.609+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:41.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:41.617+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:41.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:41.438+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:42.002 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-21T22:38:42.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:42.097 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-21T22:38:42.097 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-21T22:38:42.097 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.4 2023-07-21T22:38:42.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:41 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:41.811+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:42.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:42.053+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:42.368 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:42 smithi104 systemd[1]: Stopping Ceph osd.4 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:42.737 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:42 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:42.576+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:42.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:42 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:42.404+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:42.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:42 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:42.624+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:42.867 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:38:42.407+0000 7f0f671a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:42.868 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:38:42.407+0000 7f0f671a3700 -1 osd.4 118 *** Got signal Terminated *** 2023-07-21T22:38:42.868 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:42 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:38:42.407+0000 7f0f671a3700 -1 osd.4 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:43.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:42 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:42.858+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:43.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:43 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:43.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:43.737 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:43.555+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:43.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:43.577+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:43.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:43 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:43.434+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:44.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:43 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:43.828+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:44.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:44 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:44.014+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:44.987 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:44 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:44.540+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:44.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:44 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:44.453+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:44.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:44 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:44.551+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:45.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:44 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:44.864+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:45.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:44 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:44.864+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:45.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:44 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:44.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:45.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:44 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:44.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:45.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:45.192+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:45.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:45.409+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:45.738 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:45.566+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:45.738 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:45.566+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:45.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:45 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:45.539+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:45.867 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:38:45.325+0000 7f0f60793700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.719432+0000 front 2023-07-21T22:38:22.719408+0000 (oldest deadline 2023-07-21T22:38:45.019581+0000) 2023-07-21T22:38:46.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:45.814+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:46.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:45 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:45.814+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:46.350 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:45.942+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:46.351 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:45 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:45.942+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:46.617 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:46 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:38:46.307+0000 7f0f60793700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.719432+0000 front 2023-07-21T22:38:22.719408+0000 (oldest deadline 2023-07-21T22:38:45.019581+0000) 2023-07-21T22:38:46.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:46.183+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:46.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:46 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:46.563+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:46.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:46 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:46.434+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:46.739 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:46 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:46.609+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:46.739 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:46 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:46.609+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:47.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:46.845+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:47.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:46 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:46.845+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:47.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:46 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:46.906+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:47.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:46 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:46.906+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:47.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:47 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:47.229+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:47.867 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4[139526]: 2023-07-21T22:38:47.331+0000 7f0f60793700 -1 osd.4 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.719432+0000 front 2023-07-21T22:38:22.719408+0000 (oldest deadline 2023-07-21T22:38:45.019581+0000) 2023-07-21T22:38:47.987 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:47 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:47.528+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:47.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:47 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:47.453+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:47.989 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:47 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:47.609+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:47.989 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:47 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:47.609+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:48.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:47 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:47.837+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:48.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:47 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:47.837+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:48.256 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:48 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:48.192+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:48.257 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:47.948+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:48.257 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:47 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:47.948+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:48.617 INFO:journalctl@ceph.osd.4.smithi104.stdout:Jul 21 22:38:48 smithi104 bash[161982]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-4 2023-07-21T22:38:48.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:48 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:48.218+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:48.987 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:48 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:48.574+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:48.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:48 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:48.567+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:48.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:48 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:48.567+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:48.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:48 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:48.493+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:49.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:48 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:48.796+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:49.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:48 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:48.796+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:49.343 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:49 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:49.168+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:49.343 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:48 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:48.983+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:49.344 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:48 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:48.983+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:49.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:49 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:49.185+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:49.784 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-21T22:38:49.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:49.825 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-21T22:38:49.826 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-21T22:38:49.826 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.5 2023-07-21T22:38:49.857 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:49 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:49.576+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:49.857 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:49 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:49.532+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:49.858 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:49 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:49.596+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:49.858 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:49 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:49.596+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:50.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:49 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:49.808+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:50.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:49 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:49.808+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:50.200 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:49 smithi149 systemd[1]: Stopping Ceph osd.5 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:50.200 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:50.157+0000 7f654d8c3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:50.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:50 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:50.160+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:50.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:49 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:49.989+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:50.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:49 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:49.989+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:50.487 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:50.157+0000 7f654d8c3700 -1 osd.5 118 *** Got signal Terminated *** 2023-07-21T22:38:50.488 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:50.157+0000 7f654d8c3700 -1 osd.5 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:50.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:50.157+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:50.987 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:50.531+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:50.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:50.490+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:50.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:50.591+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:50.989 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:50 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:50.591+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:51.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:50.825+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:51.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:50.825+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:51.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:51 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:51.201+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:51.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:50 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:50.978+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:51.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:50 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:50.978+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:51.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:51 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:51.159+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:51.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:51 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:51.499+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:51.988 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:51 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:51.514+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:51.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:51 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:51.571+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:51.989 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:51 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:51.571+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:52.194 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:52.111+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:52.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:51 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:51.871+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:52.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:51 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:51.871+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:52.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:52 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:52.196+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:52.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:52 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:52.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:52.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:52 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:52.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:52.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:52.454+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:52.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:52.454+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:52.988 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:52.607+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:52.989 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:52.607+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:52.989 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:52.516+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:52.989 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:52 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:52.516+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:53.191 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:52.830+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:53.191 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:52.830+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:53.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:53.216+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:53.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:53.216+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:53.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:53.029+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:53.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:53.029+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:53.443 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:53 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:53.148+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:53.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:53.415+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:53.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:53.415+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:53.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:53.525+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:53.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:53.525+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:53.739 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:53.641+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:53.739 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:53.641+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:53.739 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:53 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:53.641+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.247770+0000 front 2023-07-21T22:38:28.247688+0000 (oldest deadline 2023-07-21T22:38:52.947812+0000) 2023-07-21T22:38:54.164 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:53 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:53.843+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:54.165 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:53 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:53.843+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:54.298 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:53.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:54.298 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:53.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:54.298 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:53 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:53.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:38:54.444 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:54 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:54.122+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:54.444 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:54 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:54.122+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:38:54.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:54 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:54.254+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:54.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:54 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:54.254+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:54.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:54.496+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:54.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:54.496+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:54.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:54.400+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:54.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:54.400+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:54.739 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:54.622+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.543633+0000 front 2023-07-21T22:38:11.543411+0000 (oldest deadline 2023-07-21T22:38:37.443449+0000) 2023-07-21T22:38:54.740 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:54.622+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.446540+0000 front 2023-07-21T22:38:22.446760+0000 (oldest deadline 2023-07-21T22:38:44.746598+0000) 2023-07-21T22:38:54.740 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:54 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5[140400]: 2023-07-21T22:38:54.622+0000 7f6549eb9700 -1 osd.5 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.247770+0000 front 2023-07-21T22:38:28.247688+0000 (oldest deadline 2023-07-21T22:38:52.947812+0000) 2023-07-21T22:38:55.116 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:54 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:54.818+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:55.117 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:54 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:54.818+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:55.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:55 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:55.235+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:55.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:55 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:55.235+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:55.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:55 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:55.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:55.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:55 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:55.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:55.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:55 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:55.012+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:38:55.443 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:55 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:55.073+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:55.444 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:55 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:55.073+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:38:55.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:55 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:55.454+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:55.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:55 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:55.454+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:55.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:55 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:55.436+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:55.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:55 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:55.436+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:56.136 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:55 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:55.833+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:56.136 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:55 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:55.833+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:56.136 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:55 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:55.833+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:38:56.237 INFO:journalctl@ceph.osd.5.smithi149.stdout:Jul 21 22:38:55 smithi149 bash[163913]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-5 2023-07-21T22:38:56.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:56.266+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:56.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:56.266+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:56.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:56.025+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:56.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:56.025+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:56.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:56 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:56.025+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:38:56.443 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:56 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:56.093+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:56.444 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:56 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:56.093+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:38:56.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:56 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:56.429+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:56.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:56 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:56.429+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:56.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:56 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:56.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:56.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:56 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:56.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:57.132 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-21T22:38:57.180 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:56 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:56.790+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:57.181 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:56 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:56.790+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:57.181 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:56 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:56.790+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:38:57.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:38:57.213 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-21T22:38:57.214 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-21T22:38:57.214 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.6 2023-07-21T22:38:57.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:57.218+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:57.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:57.218+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:57.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:57.021+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:57.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:57.021+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:57.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:57.021+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:38:57.443 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:57.138+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:57.444 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:57.138+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:38:57.444 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:57 smithi066 systemd[1]: Stopping Ceph osd.6 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:38:57.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:57.405+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:57.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:57.405+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:57.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:57.383+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:57.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:57 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:57.383+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:58.178 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:57.832+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:58.178 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:57.832+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:58.179 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:57.832+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:38:58.179 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:57.699+0000 7f4435315700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:38:58.179 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:57.699+0000 7f4435315700 -1 osd.6 118 *** Got signal Terminated *** 2023-07-21T22:38:58.180 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:57 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:57.699+0000 7f4435315700 -1 osd.6 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:38:58.180 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:58 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:58.135+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:58.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:57.983+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:58.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:57.983+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:58.305 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:57 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:57.983+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:38:58.443 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:58 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:58.135+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:38:58.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:58.261+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:58.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:58.261+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:58.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:58 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:58.385+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:58.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:58 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:58.385+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:58.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:58 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:58.420+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:58.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:58 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:58.420+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:38:59.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:58 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:58.793+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:38:59.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:58 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:58.793+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:38:59.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:58 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:58.793+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:38:59.297 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:58.986+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:38:59.298 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:58.986+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:38:59.298 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:38:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:38:58.986+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:38:59.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:59 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:59.254+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:38:59.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:38:59 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:38:59.254+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:38:59.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:59 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:59.185+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:38:59.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:38:59 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:38:59.185+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:38:59.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:59 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:59.410+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:38:59.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:38:59 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:38:59.410+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:38:59.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:59 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:59.422+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:38:59.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:38:59 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:38:59.422+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:00.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:59 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:59.745+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:00.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:59 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:59.745+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:00.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:38:59 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:38:59.745+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:00.306 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.001+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:00.307 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.001+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:00.307 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.001+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:00.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:00.263+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:00.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:00.263+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:00.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:00.164+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:39:00.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:00.164+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:39:00.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:00 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:00.433+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:00.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:00 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:00.433+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:00.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:00 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:00.443+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:00.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:00 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:00.443+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:01.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:00.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:01.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:00.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:01.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:00 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:00.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:01.321 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.998+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:01.322 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.998+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:01.322 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.998+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:01.323 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:00 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:00.998+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:01.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:01 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:01.277+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:01.618 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:01 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:01.277+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:01.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:01.157+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:39:01.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:01.157+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:39:01.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:01.157+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.547244+0000 front 2023-07-21T22:38:37.547281+0000 (oldest deadline 2023-07-21T22:39:00.948531+0000) 2023-07-21T22:39:01.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:01 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:01.413+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:01.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:01 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:01.413+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:01.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:01 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:01.407+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:01.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:01 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:01.407+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:02.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:01.764+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:02.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:01.764+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:02.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:01.764+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:02.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:01 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:01.764+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:02.298 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:01 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:01.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:02.299 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:01 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:01.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:02.299 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:01 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:01.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:02.299 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:01 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:01.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:02.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:02.255+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:02.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:02.255+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:02.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:02.255+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:02.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:02.203+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.842074+0000 front 2023-07-21T22:38:21.842134+0000 (oldest deadline 2023-07-21T22:38:44.742382+0000) 2023-07-21T22:39:02.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:02.203+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.444416+0000 front 2023-07-21T22:38:29.444736+0000 (oldest deadline 2023-07-21T22:38:53.544632+0000) 2023-07-21T22:39:02.694 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6[149585]: 2023-07-21T22:39:02.203+0000 7f442f106700 -1 osd.6 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.547244+0000 front 2023-07-21T22:38:37.547281+0000 (oldest deadline 2023-07-21T22:39:00.948531+0000) 2023-07-21T22:39:02.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:02 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:02.462+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:02.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:02 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:02.462+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:02.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:02 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:02.412+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:02.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:02 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:02.412+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:03.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:02.726+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:03.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:02.726+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:03.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:02.726+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:03.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:02 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:02.726+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:03.275 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:02.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:03.275 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:02.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:03.276 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:02.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:03.276 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:02.957+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:03.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:03 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:03.232+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:03.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:03 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:03.232+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:03.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:03 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:03.232+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:03.693 INFO:journalctl@ceph.osd.6.smithi066.stdout:Jul 21 22:39:03 smithi066 bash[193208]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-6 2023-07-21T22:39:03.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:03 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:03.491+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:03.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:03 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:03.491+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:03.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:03 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:03.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:03.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:03 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:03.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:03.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:03 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:03.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:04.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:03 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:03.751+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:04.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:03 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:03.751+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:04.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:03 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:03.751+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:04.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:03 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:03.751+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:04.320 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:04.002+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:04.320 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:04.002+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:04.321 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:04.002+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:04.321 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:04.002+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:04.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:04.276+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:04.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:04.276+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:04.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:04 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:04.276+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:04.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:04 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:04.485+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:04.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:04 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:04.485+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:04.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:04 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:04.362+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:04.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:04 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:04.362+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:04.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:04 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:04.362+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:04.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:04 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:04.707+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:04.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:04 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:04.707+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:04.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:04 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:04.707+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:04.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:04 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:04.707+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:05.352 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:05.024+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:05.352 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:05.024+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:05.353 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:05.024+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:05.353 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:05.024+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:05.516 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-21T22:39:05.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:05.610 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-21T22:39:05.611 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-21T22:39:05.611 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.7 2023-07-21T22:39:05.616 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:05.309+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:05.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:05.309+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:05.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:05.309+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:05.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:05.439+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:05.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:05.439+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:05.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:05.336+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:05.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:05.336+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:05.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:05 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:05.336+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:05.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:05.687+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:05.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:05.687+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:05.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:05.687+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:05.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:05 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:05.687+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:05.971 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 systemd[1]: Stopping Ceph osd.7 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:05.972 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:05.928+0000 7f5d61873700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:06.342 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:05.928+0000 7f5d61873700 -1 osd.7 118 *** Got signal Terminated *** 2023-07-21T22:39:06.343 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:05 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:05.928+0000 7f5d61873700 -1 osd.7 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:06.343 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:06.045+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:06.344 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:06.045+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:06.344 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:06.045+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:06.344 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:06.045+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:06.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:06.300+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:06.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:06.300+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:06.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:06.300+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:06.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:06.404+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:06.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:06.404+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:06.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:06.404+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:06.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:06.377+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:06.740 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:06.377+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:06.740 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:06 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:06.377+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:07.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:06.691+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:07.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:06.691+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:07.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:06.691+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:07.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:06 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:06.691+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:07.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:07.312+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:07.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:07.312+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:07.618 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:07.312+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:07.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:07.078+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:07.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:07.078+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:07.619 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:07.078+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:07.619 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:07 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:07.078+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:07.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:07 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:07.432+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:07.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:07 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:07.432+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:07.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:07 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:07.432+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:07.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:07 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:07.377+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:07.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:07 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:07.377+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:07.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:07 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:07.377+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:08.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:07 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:07.720+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:08.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:07 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:07.720+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:08.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:07 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:07.720+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:08.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:07 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:07.720+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:08.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:08.292+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:08.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:08.292+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:08.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:08.292+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:08.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:08.102+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:08.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:08.102+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:08.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:08.102+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:08.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:08 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:08.102+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:08.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:08.401+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:08.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:08.401+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:08.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:08.401+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:08.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:08.416+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:08.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:08.416+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:08.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:08 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:08.416+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:09.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:08.738+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:09.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:08.738+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:09.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:08.738+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:09.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:08 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:08.738+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:09.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:09.287+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:09.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:09.287+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:09.367 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:09.287+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:09.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:09.057+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:09.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:09.057+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:09.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:09.057+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:09.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:09 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:09.057+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:09.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:09 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:09.369+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:09.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:09 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:09.369+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:09.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:09 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:09.369+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:09.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:09 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:09.426+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:09.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:09 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:09.426+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:09.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:09 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:09.426+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:10.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:09 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:09.697+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:10.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:09 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:09.697+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:10.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:09 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:09.697+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:10.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:09 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:09.697+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:10.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:10.282+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:22.772528+0000 front 2023-07-21T22:38:22.772555+0000 (oldest deadline 2023-07-21T22:38:47.472747+0000) 2023-07-21T22:39:10.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:10.282+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.574188+0000 front 2023-07-21T22:38:28.574234+0000 (oldest deadline 2023-07-21T22:38:52.674393+0000) 2023-07-21T22:39:10.617 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7[145048]: 2023-07-21T22:39:10.282+0000 7f5d5b664700 -1 osd.7 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.175782+0000 front 2023-07-21T22:38:36.176020+0000 (oldest deadline 2023-07-21T22:39:02.075775+0000) 2023-07-21T22:39:10.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:10.100+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:10.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:10.100+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:10.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:10.100+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:10.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:10.100+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:10.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:10.418+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:10.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:10.418+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:10.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:10.418+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:10.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:10.456+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:10.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:10.456+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:10.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:10.456+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:10.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:10 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:10.456+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:11.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:10.683+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:11.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:10.683+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:11.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:10.683+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:11.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:10.683+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:11.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:10 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:10.683+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:11.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:11 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:11.056+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:11.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:11 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:11.056+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:11.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:11 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:11.056+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:11.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:11 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:11.056+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:11.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:11 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:11.056+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:11.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:11.413+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:11.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:11.413+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:11.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:11.413+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:11.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:11.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:11.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:11.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:11.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:11.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:11.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:11 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:11.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:12.117 INFO:journalctl@ceph.osd.7.smithi104.stdout:Jul 21 22:39:11 smithi104 bash[162262]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-7 2023-07-21T22:39:12.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:11 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:11.710+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:12.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:11 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:11.710+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:12.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:11 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:11.710+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:12.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:11 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:11.710+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:12.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:11 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:11.710+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:12.387 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:12 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:12.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:12.387 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:12 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:12.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:12.387 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:12 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:12.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:12.388 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:12 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:12.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:12.388 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:12 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:12.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:12.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:12.394+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:12.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:12.394+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:12.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:12.394+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:12.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:12.394+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:12.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:12.503+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:12.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:12.503+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:12.740 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:12.503+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:12.740 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:12 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:12.503+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:13.091 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-21T22:39:13.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:13.172 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-21T22:39:13.173 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-21T22:39:13.174 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.8 2023-07-21T22:39:13.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:12 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:12.749+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:13.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:12 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:12.749+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:13.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:12 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:12.749+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:13.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:12 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:12.749+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:13.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:12 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:12.749+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:13.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:13 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:13.050+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:13.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:13 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:13.050+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:13.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:13 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:13.050+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:13.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:13 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:13.050+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:13.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:13 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:13.050+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:13.487 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 systemd[1]: Stopping Ceph osd.8 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:13.488 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.368+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:13.488 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.368+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:13.488 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.368+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:13.488 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.368+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:13.987 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.503+0000 7fb288a30700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:13.988 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.503+0000 7fb288a30700 -1 osd.8 118 *** Got signal Terminated *** 2023-07-21T22:39:13.988 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:13.503+0000 7fb288a30700 -1 osd.8 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:13.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:13.474+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:13.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:13.474+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:13.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:13.474+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:13.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:13 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:13.474+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:14.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:13 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:13.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:14.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:13 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:13.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:14.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:13 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:13.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:14.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:13 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:13.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:14.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:13 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:13.718+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:14.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:14 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:14.003+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:14.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:14 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:14.003+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:14.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:14 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:14.003+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:14.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:14 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:14.003+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:14.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:14 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:14.003+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:14.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:14.466+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:14.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:14.466+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:14.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:14.466+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:14.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:14.466+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:14.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:14.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:14.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:14.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:14.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:14.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:14.739 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:14 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:14.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:15.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:14.696+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:15.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:14.696+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:15.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:14.696+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:15.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:14.696+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:15.195 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:14 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:14.696+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:15.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.036+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:15.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.036+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:15.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.036+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:15.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.036+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:15.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.036+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:15.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:15.335+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:15.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:15.335+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:15.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:15.335+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:15.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:15.335+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:15.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:15.486+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:15.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:15.486+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:15.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:15.486+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:15.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:15.486+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:16.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:15 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:15.664+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:16.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:15 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:15.664+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:16.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:15 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:15.664+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:16.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:15 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:15.664+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:16.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:15 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:15.664+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:16.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.992+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:16.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.992+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:16.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.992+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:16.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.992+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:16.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:15 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:15.992+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:16.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:16.317+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:16.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:16.317+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:16.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:16.317+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:16.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:16.317+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:16.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:16.476+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:16.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:16.476+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:16.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:16.476+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:16.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:16.476+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:16.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:16 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:16.476+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:16.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:16 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:16.624+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:16.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:16 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:16.624+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:16.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:16 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:16.624+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:16.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:16 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:16.624+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:16.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:16 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:16.624+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:16.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:16 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:16.624+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:17.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:16 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:16.984+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:17.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:16 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:16.984+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:17.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:16 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:16.984+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:17.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:16 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:16.984+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:17.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:16 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:16.984+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:17.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:17.326+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:17.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:17.326+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:17.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:17.326+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:17.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:17.326+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:17.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:17.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:17.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:17.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:17.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:17.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:17.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:17.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:17.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:17 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:17.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:17.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:17.575+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:17.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:17.575+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:17.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:17.575+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:17.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:17.575+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:17.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:17.575+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:17.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:17 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:17.575+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:18.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:17 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:17.951+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:18.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:17 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:17.951+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:18.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:17 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:17.951+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:18.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:17 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:17.951+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:18.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:17 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:17.951+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:18.737 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:18.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.037112+0000 front 2023-07-21T22:38:14.037353+0000 (oldest deadline 2023-07-21T22:38:38.137116+0000) 2023-07-21T22:39:18.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:18.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:26.338777+0000 front 2023-07-21T22:38:26.338799+0000 (oldest deadline 2023-07-21T22:38:52.139796+0000) 2023-07-21T22:39:18.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:18.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:36.141880+0000 front 2023-07-21T22:38:36.141833+0000 (oldest deadline 2023-07-21T22:39:05.542458+0000) 2023-07-21T22:39:18.738 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8[145913]: 2023-07-21T22:39:18.353+0000 7fb282821700 -1 osd.8 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:45.543071+0000 front 2023-07-21T22:38:45.543136+0000 (oldest deadline 2023-07-21T22:39:11.443126+0000) 2023-07-21T22:39:18.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:18.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:18.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:18.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:18.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:18.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:18.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:18.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:18.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:18.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:18.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:18 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:18.595+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:18.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:18 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:18.595+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:18.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:18 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:18.595+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:18.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:18 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:18.595+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:18.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:18 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:18.595+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:18.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:18 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:18.595+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:19.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:18.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:19.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:18.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:19.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:18.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:19.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:18.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:19.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:18.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:19.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:18 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:18.977+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:19.487 INFO:journalctl@ceph.osd.8.smithi149.stdout:Jul 21 22:39:19 smithi149 bash[164176]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-8 2023-07-21T22:39:19.874 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:19.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:19.874 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:19.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:19.875 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:19.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:19.875 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:19.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:19.875 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:19 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:19.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:19.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:19.573+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:19.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:19.573+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:19.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:19.573+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:19.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:19.573+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:19.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:19.573+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:19.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:19 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:19.573+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:20.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:19.946+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:20.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:19.946+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:20.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:19.946+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:20.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:19.946+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:20.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:19.946+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:20.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:19 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:19.946+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:20.682 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-21T22:39:20.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:20.723 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-21T22:39:20.724 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-21T22:39:20.724 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.9 2023-07-21T22:39:20.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:20.621+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:20.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:20.621+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:20.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:20.621+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:20.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:20.621+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:20.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:20.621+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:20.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:20.621+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:20.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:20 smithi066 systemd[1]: Stopping Ceph osd.9 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:20.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:20.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:20.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:20.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:20.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:20.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:20.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:20.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:20.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:20 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:20.482+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:21.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:20.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:21.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:20.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:21.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:20.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:21.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:20.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:21.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:20.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:21.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:20 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:20.995+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:21.655 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.211+0000 7f5248f55700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:21.656 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.211+0000 7f5248f55700 -1 osd.9 118 *** Got signal Terminated *** 2023-07-21T22:39:21.656 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.211+0000 7f5248f55700 -1 osd.9 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:21.656 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.612+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:21.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.612+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:21.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.612+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:21.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.612+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:21.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.612+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:21.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:21 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:21.612+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:21.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:21.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:21.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:21.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:21.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:21.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:21.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:21.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:21.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:21 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:21.473+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:22.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.007+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:22.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.007+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:22.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.007+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:22.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.007+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:22.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.007+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:22.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.007+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:22.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:22.594+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:22.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:22.594+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:22.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:22.594+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:22.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:22.594+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:22.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:22.594+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:22.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:22 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:22.594+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:22.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:22.446+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:22.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:22.446+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:22.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:22.446+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:22.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:22.446+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:22.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:22.446+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:23.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.994+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:23.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.994+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:23.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.994+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:23.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.994+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:23.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.994+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:23.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:22 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:22.994+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:23.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:23.644+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:23.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:23.644+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:23.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:23.644+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:23.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:23.644+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:23.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:23.644+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:23.945 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:23 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:23.644+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:23.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:23.488+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:23.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:23.488+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:23.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:23.488+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:23.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:23.488+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:23.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:23 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:23.488+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:24.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:24.018+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:24.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:24.018+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:24.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:24.018+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:24.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:24.018+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:24.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:24.018+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:24.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:24 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:24.018+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:24.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:24.484+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:24.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:24.484+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:24.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:24.484+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:24.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:24.484+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:24.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:24.484+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:24.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:24 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:24.484+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:25.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:24.694+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:25.193 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:24.694+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:25.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:24.694+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:25.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:24.694+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:25.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:24.694+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:25.194 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:24 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:24.694+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:25.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:25.049+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:25.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:25.049+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:25.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:25.049+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:25.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:25.049+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:25.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:25.049+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:25.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:25 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:25.049+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:25.943 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:25.650+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:13.365899+0000 front 2023-07-21T22:38:13.365938+0000 (oldest deadline 2023-07-21T22:38:39.266191+0000) 2023-07-21T22:39:25.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:25.650+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:21.568089+0000 front 2023-07-21T22:38:21.568123+0000 (oldest deadline 2023-07-21T22:38:44.468395+0000) 2023-07-21T22:39:25.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:25.650+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:30.871467+0000 front 2023-07-21T22:38:30.871606+0000 (oldest deadline 2023-07-21T22:38:55.571518+0000) 2023-07-21T22:39:25.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:25.650+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.373773+0000 front 2023-07-21T22:38:38.373798+0000 (oldest deadline 2023-07-21T22:39:01.273899+0000) 2023-07-21T22:39:25.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:25.650+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:46.377102+0000 front 2023-07-21T22:38:46.377203+0000 (oldest deadline 2023-07-21T22:39:09.877291+0000) 2023-07-21T22:39:25.944 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:25 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9[155057]: 2023-07-21T22:39:25.650+0000 7f5242d46700 -1 osd.9 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.579068+0000 front 2023-07-21T22:38:54.578844+0000 (oldest deadline 2023-07-21T22:39:15.678708+0000) 2023-07-21T22:39:25.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:25.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:25.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:25.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:25.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:25.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:25.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:25.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:25.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:25.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:25.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:25 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:25.455+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:26.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:26.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:26.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:26.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:26.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:26.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:26.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:26 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:26.097+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:26.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:26.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:26.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:26.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:26.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:26.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:26.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:26.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:26.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:26.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:26.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:26.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:27.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:27.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:27.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:27.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:27.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:27.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:27.369 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:27 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:27.059+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:27.443 INFO:journalctl@ceph.osd.9.smithi066.stdout:Jul 21 22:39:27 smithi066 bash[193485]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-9 2023-07-21T22:39:27.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:27 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:27.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:27.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:27 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:27.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:27.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:27 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:27.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:27.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:27 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:27.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:27.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:27 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:27.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:27.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:27 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:27.398+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:28.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:28.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:28.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:28.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:28.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:28.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:28.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:28 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:28.016+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:28.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:28 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:28.413+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:28.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:28 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:28.413+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:28.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:28 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:28.413+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:28.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:28 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:28.413+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:28.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:28 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:28.413+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:28.739 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:28 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:28.413+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:29.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-21T22:39:29.300 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:29.301 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-21T22:39:29.301 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-21T22:39:29.302 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.10 2023-07-21T22:39:29.334 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:29.334 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:29.334 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:29.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:29.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:29.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:29.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.034+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:29.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 systemd[1]: Stopping Ceph osd.10 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:29.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:29 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:29.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:29.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:29 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:29.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:29.737 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:29 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:29.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:29.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:29 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:29.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:29.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:29 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:29.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:29.738 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:29 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:29.423+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:30.088 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.693+0000 7fef854a8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:30.089 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.693+0000 7fef854a8700 -1 osd.10 118 *** Got signal Terminated *** 2023-07-21T22:39:30.089 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:29 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:29.693+0000 7fef854a8700 -1 osd.10 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:30.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:30.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:30.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:30.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:30.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:30.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:30.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:30 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:30.046+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:30.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:30 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:30.471+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:30.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:30 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:30.471+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:30.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:30 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:30.471+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:30.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:30 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:30.471+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:30.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:30 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:30.471+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:30.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:30 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:30.471+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:31.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:31.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:31.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:31.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:31.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:31.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:31.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:31 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:31.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:31.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:31 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:31.495+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:31.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:31 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:31.495+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:31.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:31 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:31.495+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:31.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:31 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:31.495+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:31.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:31 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:31.495+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:31.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:31 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:31.495+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:32.366 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:32.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:32.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:32.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:32.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:32.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:32.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:32 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:32.071+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:32.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:32.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:32.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:32.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:32.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:32.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:32.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:32 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:32.489+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:33.616 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:33.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:33.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:33.617 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:33.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:33.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:33.618 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:33 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:33.092+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:33.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:33.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:33.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:33.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:33.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:33.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:33.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:33 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:33.509+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:34.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:14.846588+0000 front 2023-07-21T22:38:14.846568+0000 (oldest deadline 2023-07-21T22:38:39.546673+0000) 2023-07-21T22:39:34.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2023-07-21T22:38:23.548620+0000 front 2023-07-21T22:38:23.548597+0000 (oldest deadline 2023-07-21T22:38:44.648746+0000) 2023-07-21T22:39:34.367 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:29.751451+0000 front 2023-07-21T22:38:29.751424+0000 (oldest deadline 2023-07-21T22:38:53.251534+0000) 2023-07-21T22:39:34.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:37.754381+0000 front 2023-07-21T22:38:37.754272+0000 (oldest deadline 2023-07-21T22:39:00.654276+0000) 2023-07-21T22:39:34.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.656041+0000 front 2023-07-21T22:38:44.656013+0000 (oldest deadline 2023-07-21T22:39:10.556177+0000) 2023-07-21T22:39:34.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:54.657125+0000 front 2023-07-21T22:38:54.657180+0000 (oldest deadline 2023-07-21T22:39:18.157227+0000) 2023-07-21T22:39:34.368 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:34 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10[150597]: 2023-07-21T22:39:34.074+0000 7fef81a9e700 -1 osd.10 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:02.158699+0000 front 2023-07-21T22:39:02.158767+0000 (oldest deadline 2023-07-21T22:39:25.658794+0000) 2023-07-21T22:39:34.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:34.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:34.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:34.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:34.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:34.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:34.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:34 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:34.555+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:35.616 INFO:journalctl@ceph.osd.10.smithi104.stdout:Jul 21 22:39:35 smithi104 bash[162539]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-10 2023-07-21T22:39:35.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:35.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:35.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:35.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:35.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:35.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:35.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:35 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:35.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:36.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-21T22:39:36.681 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:36.682 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-21T22:39:36.682 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-21T22:39:36.683 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.11 2023-07-21T22:39:36.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:36.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:36.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:36.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:36.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:36.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:36.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:36.507+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:36.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:36 smithi149 systemd[1]: Stopping Ceph osd.11 for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:37.487 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.000+0000 7ffac51b7700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:37.487 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.000+0000 7ffac51b7700 -1 osd.11 118 *** Got signal Terminated *** 2023-07-21T22:39:37.487 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.000+0000 7ffac51b7700 -1 osd.11 118 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T22:39:37.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:37.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:37.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:37.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:37.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:37.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:37.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:37 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:37.542+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:38.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:38.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:38.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:38.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:38.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:38.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:38.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:38 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:38.582+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:39.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:39.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:39.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:39.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:39.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:39.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:39.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:39.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:39 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:39.554+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2023-07-21T22:39:18.371036+0000 front 2023-07-21T22:39:18.371001+0000 (oldest deadline 2023-07-21T22:39:39.471073+0000) 2023-07-21T22:39:40.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:40.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:40.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:40.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:40.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:40.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:40.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:40.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:40 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:40.588+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2023-07-21T22:39:18.371036+0000 front 2023-07-21T22:39:18.371001+0000 (oldest deadline 2023-07-21T22:39:39.471073+0000) 2023-07-21T22:39:41.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2023-07-21T22:38:11.856270+0000 front 2023-07-21T22:38:11.856394+0000 (oldest deadline 2023-07-21T22:38:37.057209+0000) 2023-07-21T22:39:41.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6804 osd.2 since back 2023-07-21T22:38:28.561015+0000 front 2023-07-21T22:38:28.561039+0000 (oldest deadline 2023-07-21T22:38:52.061249+0000) 2023-07-21T22:39:41.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6814 osd.3 since back 2023-07-21T22:38:38.363636+0000 front 2023-07-21T22:38:38.363594+0000 (oldest deadline 2023-07-21T22:39:02.463483+0000) 2023-07-21T22:39:41.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2023-07-21T22:38:44.764631+0000 front 2023-07-21T22:38:44.764644+0000 (oldest deadline 2023-07-21T22:39:10.064666+0000) 2023-07-21T22:39:41.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6812 osd.5 since back 2023-07-21T22:38:51.765643+0000 front 2023-07-21T22:38:51.765661+0000 (oldest deadline 2023-07-21T22:39:15.865799+0000) 2023-07-21T22:39:41.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.66:6822 osd.6 since back 2023-07-21T22:39:01.667452+0000 front 2023-07-21T22:39:01.667511+0000 (oldest deadline 2023-07-21T22:39:23.867886+0000) 2023-07-21T22:39:41.988 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2023-07-21T22:39:10.269684+0000 front 2023-07-21T22:39:10.269600+0000 (oldest deadline 2023-07-21T22:39:31.969485+0000) 2023-07-21T22:39:41.989 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:41 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11[151391]: 2023-07-21T22:39:41.567+0000 7ffabefa8700 -1 osd.11 118 heartbeat_check: no reply from 172.21.15.149:6820 osd.8 since back 2023-07-21T22:39:18.371036+0000 front 2023-07-21T22:39:18.371001+0000 (oldest deadline 2023-07-21T22:39:39.471073+0000) 2023-07-21T22:39:42.987 INFO:journalctl@ceph.osd.11.smithi149.stdout:Jul 21 22:39:42 smithi149 bash[164436]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-osd-11 2023-07-21T22:39:44.176 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-21T22:39:44.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:44.217 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-21T22:39:44.217 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-21T22:39:44.218 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.a 2023-07-21T22:39:44.693 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:39:44 smithi066 systemd[1]: Stopping Ceph mds.a for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:45.193 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:39:44 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-a[160509]: 2023-07-21T22:39:44.708+0000 7fd96eea3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:45.193 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:39:44 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-a[160509]: 2023-07-21T22:39:44.708+0000 7fd96eea3700 -1 mds.a *** got signal Terminated *** 2023-07-21T22:39:47.193 INFO:journalctl@ceph.mds.a.smithi066.stdout:Jul 21 22:39:46 smithi066 bash[193758]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-a 2023-07-21T22:39:47.404 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-21T22:39:47.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:47.492 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-21T22:39:47.492 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-21T22:39:47.492 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.d 2023-07-21T22:39:47.693 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:39:47 smithi066 systemd[1]: Stopping Ceph mds.d for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:48.443 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:39:47 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-d[159503]: 2023-07-21T22:39:47.950+0000 7f9808600700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:48.443 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:39:47 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-d[159503]: 2023-07-21T22:39:47.950+0000 7f9808600700 -1 mds.d *** got signal Terminated *** 2023-07-21T22:39:49.443 INFO:journalctl@ceph.mds.d.smithi066.stdout:Jul 21 22:39:49 smithi066 bash[193903]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-d 2023-07-21T22:39:49.644 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-21T22:39:49.733 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:49.734 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-21T22:39:49.734 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-21T22:39:49.734 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.g 2023-07-21T22:39:49.943 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:39:49 smithi066 systemd[1]: Stopping Ceph mds.g for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:50.693 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:39:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-g[160116]: 2023-07-21T22:39:50.224+0000 7fe374a6f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:50.693 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:39:50 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-g[160116]: 2023-07-21T22:39:50.224+0000 7fe374a6f700 -1 mds.g *** got signal Terminated *** 2023-07-21T22:39:52.382 INFO:journalctl@ceph.mds.g.smithi066.stdout:Jul 21 22:39:51 smithi066 bash[194050]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-g 2023-07-21T22:39:52.413 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-21T22:39:52.497 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:52.498 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-21T22:39:52.498 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-21T22:39:52.499 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.j 2023-07-21T22:39:52.693 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:39:52 smithi066 systemd[1]: Stopping Ceph mds.j for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:53.443 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:39:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-j[159113]: 2023-07-21T22:39:52.965+0000 7f9ede279700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:53.443 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:39:52 smithi066 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-j[159113]: 2023-07-21T22:39:52.965+0000 7f9ede279700 -1 mds.j *** got signal Terminated *** 2023-07-21T22:39:57.501 INFO:journalctl@ceph.mds.j.smithi066.stdout:Jul 21 22:39:57 smithi066 bash[194191]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-j 2023-07-21T22:39:57.762 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-21T22:39:57.848 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:39:57.848 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-21T22:39:57.849 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-21T22:39:57.849 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.b 2023-07-21T22:39:58.251 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:39:57 smithi104 systemd[1]: Stopping Ceph mds.b for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:39:58.616 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:39:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: 2023-07-21T22:39:58.208+0000 7f7839353700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:39:58.616 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:39:58 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b[154373]: 2023-07-21T22:39:58.208+0000 7f7839353700 -1 mds.b *** got signal Terminated *** 2023-07-21T22:40:01.866 INFO:journalctl@ceph.mds.b.smithi104.stdout:Jul 21 22:40:01 smithi104 bash[162800]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-b 2023-07-21T22:40:02.125 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-21T22:40:02.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:02.163 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-21T22:40:02.164 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-21T22:40:02.164 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.e 2023-07-21T22:40:02.550 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:40:02 smithi104 systemd[1]: Stopping Ceph mds.e for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:02.866 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:40:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e[153688]: 2023-07-21T22:40:02.507+0000 7ff7eef5b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:02.866 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:40:02 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e[153688]: 2023-07-21T22:40:02.507+0000 7ff7eef5b700 -1 mds.e *** got signal Terminated *** 2023-07-21T22:40:06.059 INFO:journalctl@ceph.mds.e.smithi104.stdout:Jul 21 22:40:05 smithi104 bash[162959]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-e 2023-07-21T22:40:06.065 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-21T22:40:06.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:06.101 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-21T22:40:06.101 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-21T22:40:06.102 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.h 2023-07-21T22:40:06.366 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:40:06 smithi104 systemd[1]: Stopping Ceph mds.h for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:06.866 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:40:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h[154032]: 2023-07-21T22:40:06.431+0000 7f2664ae1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:06.867 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:40:06 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h[154032]: 2023-07-21T22:40:06.431+0000 7f2664ae1700 -1 mds.h *** got signal Terminated *** 2023-07-21T22:40:10.116 INFO:journalctl@ceph.mds.h.smithi104.stdout:Jul 21 22:40:09 smithi104 bash[163105]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-h 2023-07-21T22:40:10.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-21T22:40:10.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:10.171 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-21T22:40:10.171 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-21T22:40:10.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.k 2023-07-21T22:40:10.555 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:40:10 smithi104 systemd[1]: Stopping Ceph mds.k for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:10.866 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:40:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-k[153347]: 2023-07-21T22:40:10.512+0000 7f8bec6ea700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:10.866 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:40:10 smithi104 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-k[153347]: 2023-07-21T22:40:10.512+0000 7f8bec6ea700 -1 mds.k *** got signal Terminated *** 2023-07-21T22:40:15.105 INFO:journalctl@ceph.mds.k.smithi104.stdout:Jul 21 22:40:14 smithi104 bash[163251]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-k 2023-07-21T22:40:15.191 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-21T22:40:15.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:15.226 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-21T22:40:15.227 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-21T22:40:15.227 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.c 2023-07-21T22:40:15.603 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:40:15 smithi149 systemd[1]: Stopping Ceph mds.c for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:15.986 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:40:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c[155099]: 2023-07-21T22:40:15.560+0000 7f9242c75700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:15.986 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:40:15 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c[155099]: 2023-07-21T22:40:15.560+0000 7f9242c75700 -1 mds.c *** got signal Terminated *** 2023-07-21T22:40:17.986 INFO:journalctl@ceph.mds.c.smithi149.stdout:Jul 21 22:40:17 smithi149 bash[164712]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-c 2023-07-21T22:40:18.033 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-21T22:40:18.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:18.071 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-21T22:40:18.072 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-21T22:40:18.072 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.f 2023-07-21T22:40:18.460 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:40:18 smithi149 systemd[1]: Stopping Ceph mds.f for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:18.736 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:40:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f[154757]: 2023-07-21T22:40:18.417+0000 7f8c481f9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:18.736 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:40:18 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f[154757]: 2023-07-21T22:40:18.417+0000 7f8c481f9700 -1 mds.f *** got signal Terminated *** 2023-07-21T22:40:21.986 INFO:journalctl@ceph.mds.f.smithi149.stdout:Jul 21 22:40:21 smithi149 bash[164852]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-f 2023-07-21T22:40:22.155 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-21T22:40:22.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:22.190 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-21T22:40:22.190 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-21T22:40:22.191 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.i 2023-07-21T22:40:22.486 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:40:22 smithi149 systemd[1]: Stopping Ceph mds.i for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:22.986 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:40:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:40:22.513+0000 7fa6cbdd5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:22.986 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:40:22 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i[155439]: 2023-07-21T22:40:22.513+0000 7fa6cbdd5700 -1 mds.i *** got signal Terminated *** 2023-07-21T22:40:25.986 INFO:journalctl@ceph.mds.i.smithi149.stdout:Jul 21 22:40:25 smithi149 bash[164997]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-i 2023-07-21T22:40:26.138 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-21T22:40:26.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:26.173 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-21T22:40:26.173 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-21T22:40:26.173 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.l 2023-07-21T22:40:26.486 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:40:26 smithi149 systemd[1]: Stopping Ceph mds.l for 6bcb348c-2813-11ee-9b34-001a4aab830c... 2023-07-21T22:40:26.986 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:40:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-l[154410]: 2023-07-21T22:40:26.508+0000 7f34eb561700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T22:40:26.987 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:40:26 smithi149 ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-l[154410]: 2023-07-21T22:40:26.508+0000 7f34eb561700 -1 mds.l *** got signal Terminated *** 2023-07-21T22:40:29.115 INFO:journalctl@ceph.mds.l.smithi149.stdout:Jul 21 22:40:29 smithi149 bash[165143]: ceph-6bcb348c-2813-11ee-9b34-001a4aab830c-mds-l 2023-07-21T22:40:29.322 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-6bcb348c-2813-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-21T22:40:29.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T22:40:29.355 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-21T22:40:29.356 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:40:33.157 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:40:35.011 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T22:40:36.986 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:40:37.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:40:37.104 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T22:40:37.134 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-21T22:40:37.136 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi066/crash 2023-07-21T22:40:37.137 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:40:37.220 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:40:37.221 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:40:37.224 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi104/crash 2023-07-21T22:40:37.225 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:40:37.271 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:40:37.272 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:40:37.273 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi149/crash 2023-07-21T22:40:37.274 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T22:40:37.315 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T22:40:37.315 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2023-07-21T22:40:37.316 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-21T22:40:37.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-21T22:40:37.401 INFO:teuthology.orchestra.run.smithi066.stderr:grep: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-21T22:40:37.405 INFO:tasks.cephadm:Compressing logs... 2023-07-21T22:40:37.406 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:40:37.410 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:40:37.413 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T22:40:37.445 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-07-21T22:40:37.446 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:40:37.448 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:40:37.448 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2023-07-21T22:40:37.452 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-21T22:40:37.453 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-21T22:40:37.453 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-21T22:40:37.454 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-21T22:40:37.454 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.2.log: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-21T22:40:37.455 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-21T22:40:37.455 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-21T22:40:37.455 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-21T22:40:37.455 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-21T22:40:37.455 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.c.log: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:40:37.456 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-21T22:40:37.456 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.l.log: 94.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-21T22:40:37.457 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-21T22:40:37.457 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-21T22:40:37.458 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-21T22:40:37.458 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.10.log: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-21T22:40:37.459 INFO:teuthology.orchestra.run.smithi149.stderr: 92.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-21T22:40:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-21T22:40:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-21T22:40:37.460 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:40:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-21T22:40:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-21T22:40:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-21T22:40:37.461 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-21T22:40:37.462 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.b.log: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-21T22:40:37.465 INFO:teuthology.orchestra.run.smithi104.stderr: 92.2% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-21T22:40:37.471 INFO:teuthology.orchestra.run.smithi104.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:40:37.475 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T22:40:37.476 INFO:teuthology.orchestra.run.smithi066.stderr:: No such file or directory 2023-07-21T22:40:37.477 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-21T22:40:37.478 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-21T22:40:37.479 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-21T22:40:37.481 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-21T22:40:37.482 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-21T22:40:37.484 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-21T22:40:37.485 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.3.log: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:40:37.485 INFO:teuthology.orchestra.run.smithi066.stderr: 94.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-21T22:40:37.486 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-21T22:40:37.487 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-21T22:40:37.487 INFO:teuthology.orchestra.run.smithi066.stderr: 94.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-21T22:40:37.488 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T22:40:37.490 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-21T22:40:37.492 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-21T22:40:37.493 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-21T22:40:37.500 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-21T22:40:37.500 INFO:teuthology.orchestra.run.smithi066.stderr: 91.4% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:40:37.501 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-07-21T22:40:37.505 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/ceph-client.admin.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-07-21T22:40:37.877 INFO:teuthology.orchestra.run.smithi149.stderr: 91.4% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:40:37.890 INFO:teuthology.orchestra.run.smithi066.stderr: 89.7% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-21T22:40:38.570 INFO:teuthology.orchestra.run.smithi104.stderr: 91.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T22:40:38.593 INFO:teuthology.orchestra.run.smithi066.stderr: 90.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-21T22:40:38.968 INFO:teuthology.orchestra.run.smithi149.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T22:40:39.342 INFO:teuthology.orchestra.run.smithi104.stderr: 91.8% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-21T22:40:43.950 INFO:teuthology.orchestra.run.smithi149.stderr: 93.2% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-21T22:40:53.791 INFO:teuthology.orchestra.run.smithi104.stderr: 95.2% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-21T22:41:00.473 INFO:teuthology.orchestra.run.smithi149.stderr: 95.2% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-21T22:41:02.493 INFO:teuthology.orchestra.run.smithi066.stderr: 91.6% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-21T22:41:03.381 INFO:teuthology.orchestra.run.smithi149.stderr: 95.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-21T22:41:08.349 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-21T22:41:08.399 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-21T22:41:08.755 INFO:teuthology.orchestra.run.smithi066.stderr: 91.2% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-21T22:41:08.758 INFO:teuthology.orchestra.run.smithi066.stderr: 2023-07-21T22:41:08.758 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m31.311s 2023-07-21T22:41:08.758 INFO:teuthology.orchestra.run.smithi066.stderr:user 1m50.998s 2023-07-21T22:41:08.759 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m8.693s 2023-07-21T22:41:27.149 INFO:teuthology.orchestra.run.smithi104.stderr: 91.6% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-21T22:41:29.517 INFO:teuthology.orchestra.run.smithi104.stderr: 91.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz 2023-07-21T22:41:39.042 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-21T22:41:44.952 INFO:teuthology.orchestra.run.smithi104.stderr: 91.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-21T22:41:47.836 INFO:teuthology.orchestra.run.smithi104.stderr: 91.2% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-21T22:41:48.942 INFO:teuthology.orchestra.run.smithi149.stderr: 91.0% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-21T22:41:49.201 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-21T22:42:08.387 INFO:teuthology.orchestra.run.smithi149.stderr: 91.1% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-21T22:43:07.926 INFO:teuthology.orchestra.run.smithi149.stderr: 93.3% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-21T22:43:07.959 INFO:teuthology.orchestra.run.smithi149.stderr: 2023-07-21T22:43:07.959 INFO:teuthology.orchestra.run.smithi149.stderr:real 2m30.533s 2023-07-21T22:43:07.959 INFO:teuthology.orchestra.run.smithi149.stderr:user 3m23.699s 2023-07-21T22:43:07.960 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m8.176s 2023-07-21T22:43:10.904 INFO:teuthology.orchestra.run.smithi104.stderr: 94.1% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-21T22:44:15.260 INFO:teuthology.orchestra.run.smithi104.stderr: 94.5% -- replaced with /var/log/ceph/6bcb348c-2813-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-21T22:44:15.278 INFO:teuthology.orchestra.run.smithi104.stderr: 2023-07-21T22:44:15.278 INFO:teuthology.orchestra.run.smithi104.stderr:real 3m37.852s 2023-07-21T22:44:15.278 INFO:teuthology.orchestra.run.smithi104.stderr:user 5m39.259s 2023-07-21T22:44:15.279 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m13.396s 2023-07-21T22:44:15.279 INFO:tasks.cephadm:Archiving logs... 2023-07-21T22:44:15.280 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi066/log 2023-07-21T22:44:15.281 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:44:30.028 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi104/log 2023-07-21T22:44:30.030 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:45:11.928 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi149/log 2023-07-21T22:45:11.930 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T22:45:39.848 INFO:tasks.cephadm:Removing cluster... 2023-07-21T22:45:39.849 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --force 2023-07-21T22:45:41.454 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --force 2023-07-21T22:45:43.094 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 6bcb348c-2813-11ee-9b34-001a4aab830c --force 2023-07-21T22:45:44.829 INFO:tasks.cephadm:Removing cephadm ... 2023-07-21T22:45:44.830 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:45:44.881 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:45:44.981 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T22:45:45.007 INFO:tasks.cephadm:Teardown complete 2023-07-21T22:45:45.008 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-21T22:45:45.021 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-21T22:45:45.022 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:45:45.028 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:45:45.030 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T22:45:45.186 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-21T22:45:45.196 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-21T22:45:45.197 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:45:45.202 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:45:45.204 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T22:45:45.220 INFO:teuthology.orchestra.run.smithi149.stderr:bash: ntpq: command not found 2023-07-21T22:45:45.221 INFO:teuthology.orchestra.run.smithi104.stderr:bash: ntpq: command not found 2023-07-21T22:45:45.237 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:45:45.237 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2023-07-21T22:45:45.238 INFO:teuthology.orchestra.run.smithi149.stdout:^- hv01.front.sepia.ceph.com 2 6 377 23 -746us[ -746us] +/- 268ms 2023-07-21T22:45:45.238 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1210us[-1141us] +/- 67ms 2023-07-21T22:45:45.238 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +1493us[+1493us] +/- 82ms 2023-07-21T22:45:45.238 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:45:45.241 INFO:teuthology.orchestra.run.smithi066.stderr:bash: ntpq: command not found 2023-07-21T22:45:45.250 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:45:45.250 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2023-07-21T22:45:45.250 INFO:teuthology.orchestra.run.smithi066.stdout:^- hv01.front.sepia.ceph.com 2 6 377 24 -714us[ -676us] +/- 268ms 2023-07-21T22:45:45.251 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -1202us[-1164us] +/- 67ms 2023-07-21T22:45:45.251 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +1518us[+1556us] +/- 82ms 2023-07-21T22:45:45.251 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:45:45.254 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T22:45:45.254 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2023-07-21T22:45:45.254 INFO:teuthology.orchestra.run.smithi104.stdout:^- hv01.front.sepia.ceph.com 2 6 377 23 -735us[ -724us] +/- 268ms 2023-07-21T22:45:45.254 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv02.front.sepia.ceph.com 3 6 377 23 -1206us[-1195us] +/- 67ms 2023-07-21T22:45:45.255 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 22 +1499us[+1499us] +/- 82ms 2023-07-21T22:45:45.255 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T22:45:45.256 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-21T22:45:45.266 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-21T22:45:45.267 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-21T22:45:45.278 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:45:45.480 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:45:45.643 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T22:45:45.854 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:45:45.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:45.933 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:45:45.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:45.973 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T22:45:46.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:46.007 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-21T22:45:46.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-21T22:45:46.036 INFO:teuthology.task.internal:Duration was 3379.530998 seconds 2023-07-21T22:45:46.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-21T22:45:46.047 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-21T22:45:46.048 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:45:46.053 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:45:46.055 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T22:45:46.133 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:45:46.133 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:45:46.170 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T22:45:46.524 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-21T22:45:46.525 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2023-07-21T22:45:46.525 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:45:46.587 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2023-07-21T22:45:46.588 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:45:46.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2023-07-21T22:45:46.620 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T22:45:46.648 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-21T22:45:46.649 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:45:46.653 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:45:46.661 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T22:45:46.753 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-21T22:45:46.754 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:45:46.758 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:45:46.760 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T22:45:46.930 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-21T22:45:46.947 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-21T22:45:46.948 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:45:47.019 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:45:47.061 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T22:45:47.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-21T22:45:47.104 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:47.109 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:47.112 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:47.151 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2023-07-21T22:45:47.162 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2023-07-21T22:45:47.174 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2023-07-21T22:45:47.212 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:47.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:47.255 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:47.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:47.272 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T22:45:47.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T22:45:47.289 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-21T22:45:47.302 INFO:teuthology.task.internal:Transferring archived files... 2023-07-21T22:45:47.303 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi066 2023-07-21T22:45:47.304 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:45:47.392 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi104 2023-07-21T22:45:47.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:45:47.430 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716/remote/smithi149 2023-07-21T22:45:47.431 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T22:45:47.469 INFO:teuthology.task.internal:Removing archive directory... 2023-07-21T22:45:47.470 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:45:47.474 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:45:47.477 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T22:45:47.520 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-21T22:45:47.534 INFO:teuthology.task.internal:Not uploading archives. 2023-07-21T22:45:47.534 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-21T22:45:47.545 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-21T22:45:47.546 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:45:47.559 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:45:47.561 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T22:45:47.575 INFO:teuthology.orchestra.run.smithi104.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:45 /home/ubuntu/cephtest 2023-07-21T22:45:47.577 INFO:teuthology.orchestra.run.smithi149.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:45 /home/ubuntu/cephtest 2023-07-21T22:45:47.599 INFO:teuthology.orchestra.run.smithi066.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 22:45 /home/ubuntu/cephtest 2023-07-21T22:45:47.603 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-21T22:45:47.619 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-21T22:45:47.644 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T22:45:47.672 DEBUG:teuthology.nuke:shortname: smithi066 2023-07-21T22:45:47.673 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:45:47.696 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:36:15.023638', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:45:47.700 DEBUG:teuthology.nuke:shortname: smithi104 2023-07-21T22:45:47.701 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:45:47.725 DEBUG:teuthology.nuke:shortname: smithi149 2023-07-21T22:45:47.726 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T22:45:47.730 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:36:15.022389', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T22:45:47.765 INFO:teuthology.orchestra.console:Power off smithi066 2023-07-21T22:45:47.766 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:45:47.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_21:30:53-fs-wip-rishabh-2023Jul13-base-2-testing-default-smithi/7347716', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 21:36:15.020912', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T22:45:47.790 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:47.823 INFO:teuthology.orchestra.console:Power off smithi104 2023-07-21T22:45:47.823 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:45:47.847 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:47.866 INFO:teuthology.orchestra.console:Power off smithi149 2023-07-21T22:45:47.867 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T22:45:47.889 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:51.814 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:51.877 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:51.913 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:55.928 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:55.993 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:45:56.026 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:46:00.045 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi066.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:46:00.106 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:46:00.137 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T22:46:00.258 INFO:teuthology.orchestra.console:Power off for smithi066 completed 2023-07-21T22:46:00.318 INFO:teuthology.orchestra.console:Power off for smithi104 completed 2023-07-21T22:46:00.349 INFO:teuthology.orchestra.console:Power off for smithi149 completed 2023-07-21T22:46:00.457 INFO:teuthology.run:Summary data: client.0-kernel-sha1: c685da6f56cc77f2cfb9c4221be3960522616caf description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 3379.530998468399 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi066 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bffe049d41f6256b5f1cd35b2ea5c7f5a9a2e7ea TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6cec760fdafc45fa8d9a524f1eaffce0 status: fail success: false 2023-07-21T22:46:00.458 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T22:46:00.545 INFO:teuthology.run:FAIL